On 2023-02-26 12:22, Dave Howorth wrote:

On Sun, 26 Feb 2023 09:25:26 +0100 "Carlos E. R." <> wrote:

...

On 2023-02-26 07:04, Felix Miata wrote:

...

# journalctl -b --no-hostname| grep "Feb 25" | egrep -v 'smartd|rpc' Feb 25 00:00:47 systemd[1]: Starting Do daily mandb update... Feb 25 00:00:47 systemd[1]: Starting Rotate log files... Feb 25 00:00:47 systemd[1]: Starting Update locate database... Feb 25 00:00:47 systemd[1]: logrotate.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Rotate log files. Feb 25 00:00:47 systemd[1]: mandb.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Do daily mandb update.

What is mystifiying to me is that the three tasks start simultaneously, and the three end in zero seconds.

Well if multiple services are started at the same time and are short-running it must be more likely than not, surely? If you had timestamps to milliseconds or more precisely then you might see a difference.

But I would expect to see: Starting Do daily mandb update... Finished Do daily mandb update. Starting Rotate log files... Finished Rotate log files. Starting Update locate database... Finished Update locate database. The three jobs are demanding on the hard disk.

And three services start but only two finish, unless I'm misreading something? The one substantial task takes over a second!

...

The rest is similar to other versions.

-- Cheers / Saludos, Carlos E. R. (from 15.4 x86_64 at Telcontar)

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 El 2023-02-26 a las 13:36 -0000, Dave Howorth escribió:

On Sun, 26 Feb 2023 12:46:42 +0100 "Carlos E. R." <> wrote:

...

On 2023-02-26 12:22, Dave Howorth wrote:

...

On Sun, 26 Feb 2023 09:25:26 +0100 "Carlos E. R." <> wrote:

...

On 2023-02-26 07:04, Felix Miata wrote:

...

# journalctl -b --no-hostname| grep "Feb 25" | egrep -v 'smartd|rpc' Feb 25 00:00:47 systemd[1]: Starting Do daily mandb update... Feb 25 00:00:47 systemd[1]: Starting Rotate log files... Feb 25 00:00:47 systemd[1]: Starting Update locate database... Feb 25 00:00:47 systemd[1]: logrotate.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Rotate log files. Feb 25 00:00:47 systemd[1]: mandb.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Do daily mandb update.

What is mystifiying to me is that the three tasks start simultaneously, and the three end in zero seconds.

Well if multiple services are started at the same time and are short-running it must be more likely than not, surely? If you had timestamps to milliseconds or more precisely then you might see a difference.

But I would expect to see:

Starting Do daily mandb update... Finished Do daily mandb update. Starting Rotate log files... Finished Rotate log files. Starting Update locate database... Finished Update locate database.

Why, if the jobs are started in parallel?

That's the problem, they shouldn't. Before systemd, they happened in sequence.

...

The three jobs are demanding on the hard disk.

Apparently not particularly except for the locate databse task.

On my computer, they are very demanding. Specially if the machine is rotating rust. There have been times when these jobs started that the machine became unresponsive for a while. cer@Telcontar:~> grep "Starting Do daily mandb update\|Finished Do daily mandb update" /var/log/messages <3.6> 2023-02-24T00:00:02.256852+01:00 Telcontar systemd 1 - - Finished Do daily mandb update. <3.6> 2023-02-25T00:00:01.561800+01:00 Telcontar systemd 1 - - Starting Do daily mandb update... <3.6> 2023-02-25T00:00:03.518819+01:00 Telcontar systemd 1 - - Finished Do daily mandb update. <3.6> 2023-02-26T00:00:01.766809+01:00 Telcontar systemd 1 - - Starting Do daily mandb update... <3.6> 2023-02-26T00:00:03.002047+01:00 Telcontar systemd 1 - - Finished Do daily mandb update. cer@Telcontar:~> 2". It is an M2 disk. cer@Telcontar:~> grep "Starting Rotate log files\|Finished Rotate log files" /var/log/messages <3.6> 2023-02-24T00:00:02.272908+01:00 Telcontar systemd 1 - - Finished Rotate log files. <3.6> 2023-02-25T00:00:01.563275+01:00 Telcontar systemd 1 - - Starting Rotate log files... <3.6> 2023-02-25T00:00:03.519266+01:00 Telcontar systemd 1 - - Finished Rotate log files. <3.6> 2023-02-26T00:00:01.768163+01:00 Telcontar systemd 1 - - Starting Rotate log files... <3.6> 2023-02-26T00:00:03.940346+01:00 Telcontar systemd 1 - - Finished Rotate log files. cer@Telcontar:~> 2". It is an M2 disk. cer@Telcontar:~> grep "Starting Update locate database...\|Finished Update locate database." /var/log/messages <3.6> 2023-02-24T00:00:03.318721+01:00 Telcontar systemd 1 - - Finished Update locate database. <3.6> 2023-02-25T00:00:01.564561+01:00 Telcontar systemd 1 - - Starting Update locate database... <3.6> 2023-02-25T00:00:29.706891+01:00 Telcontar systemd 1 - - Finished Update locate database. <3.6> 2023-02-26T00:00:01.769485+01:00 Telcontar systemd 1 - - Starting Update locate database... <3.6> 2023-02-26T00:00:59.713500+01:00 Telcontar systemd 1 - - Finished Update locate database. cer@Telcontar:~> Almost one minute, depends on the day. There are 4 rotating rust disks to explore. - -- Cheers, Carlos E. R. (from openSUSE 15.4 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCY/uaYBwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfV5Y0AnjV1WLIx2ilVr3V9+Hg4 Qb9eTYyPAJ9096yxzPaOUzFYShay2dbQLr/Oug== =jKvi -----END PGP SIGNATURE-----

On Sun, 26 Feb 2023 18:44:00 +0100 (CET) "Carlos E. R." wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

El 2023-02-26 a las 13:36 -0000, Dave Howorth escribió:

...

On Sun, 26 Feb 2023 12:46:42 +0100 "Carlos E. R." <> wrote:

...

On 2023-02-26 12:22, Dave Howorth wrote:

...

On Sun, 26 Feb 2023 09:25:26 +0100 "Carlos E. R." <> wrote:

...

On 2023-02-26 07:04, Felix Miata wrote:

...

# journalctl -b --no-hostname| grep "Feb 25" | egrep -v 'smartd|rpc' Feb 25 00:00:47 systemd[1]: Starting Do daily mandb update... Feb 25 00:00:47 systemd[1]: Starting Rotate log files... Feb 25 00:00:47 systemd[1]: Starting Update locate database... Feb 25 00:00:47 systemd[1]: logrotate.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Rotate log files. Feb 25 00:00:47 systemd[1]: mandb.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Do daily mandb update.

What is mystifiying to me is that the three tasks start simultaneously, and the three end in zero seconds.

Well if multiple services are started at the same time and are short-running it must be more likely than not, surely? If you had timestamps to milliseconds or more precisely then you might see a difference.

But I would expect to see:

Starting Do daily mandb update... Finished Do daily mandb update. Starting Rotate log files... Finished Rotate log files. Starting Update locate database... Finished Update locate database.

Why, if the jobs are started in parallel?

That's the problem, they shouldn't. Before systemd, they happened in sequence.

Err, you're complaining because systemd improved things and runs jobs in parallel!!? It's called progress, not problem.

...

...

The three jobs are demanding on the hard disk.

Apparently not particularly except for the locate databse task.

On my computer, they are very demanding. Specially if the machine is rotating rust. There have been times when these jobs started that the machine became unresponsive for a while.

OK, but you were not commenting about your computer and we're not discussing that. You were commenting about Felix's situation. I didn't claim the condition would always be true on all machines! [snip]

Carlos E. R. composed on 2023-02-26 19:27 (UTC+0100):

I simply noticed the situation in Felix output, then came to realize that my machine does the same, and that is why it was so severely impacted when the daily jobs trigger.

So customize! Don't fire multiple timed tasks at the same time. Move the times to when you're normally not awake. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata

On Sun, 26 Feb 2023, 20:03:44 +0100, Carlos E. R. wrote:

On 2023-02-26 19:46, Felix Miata wrote:

...

Carlos E. R. composed on 2023-02-26 19:27 (UTC+0100):

...

I simply noticed the situation in Felix output, then came to realize that my machine does the same, and that is why it was so severely impacted when the daily jobs trigger.

So customize! Don't fire multiple timed tasks at the same time. Move the times to when you're normally not awake.

That's what I am thinking to do, but I do not know how they are fired. Have not started the investigation.

Oh, wait, move to when I am not awake, you say. No, that does not happen, I hibernate the machine when I go to sleep.

Carlos, we had a long discussion about this when systemd timers should take over the behaviour of cron to run 15 minutes after starting/resuming. The result of that was to install cronie-anacron to enable a similar behaviour. Systemd timers are not usable for machines not running 24 hrs a day.

The logrotate job should happen precisely at 00 hours. Or perhaps not? Maybe it should happen before or way after all timer jobs run. At 23:50, maybe. But do these job run of a single timer, or several? Before systemd, it as a single cron task, and there was a common config file.

See above. Disable the systemd timers and put a suitable script into /etc/cron.daily and you're done. HTH, cheers. l8er manfred

On Sun, 26 Feb 2023, 22:58:56 +0100, Carlos E. R. wrote:

On 2023-02-26 20:44, Manfred Hollstein wrote:

...

On Sun, 26 Feb 2023, 20:03:44 +0100, Carlos E. R. wrote:

...

On 2023-02-26 19:46, Felix Miata wrote:

...

Carlos E. R. composed on 2023-02-26 19:27 (UTC+0100):

...

I simply noticed the situation in Felix output, then came to realize that my machine does the same, and that is why it was so severely impacted when the daily jobs trigger.

So customize! Don't fire multiple timed tasks at the same time. Move the times to when you're normally not awake.

That's what I am thinking to do, but I do not know how they are fired. Have not started the investigation.

Oh, wait, move to when I am not awake, you say. No, that does not happen, I hibernate the machine when I go to sleep.

Carlos, we had a long discussion about this when systemd timers should take over the behaviour of cron to run 15 minutes after starting/resuming. The result of that was to install cronie-anacron to enable a similar behaviour. Systemd timers are not usable for machines not running 24 hrs a day.

If I have to go that way, I would simply restore from backup the old suse cron scripts.

My machine is certainly not running 24/7, yet the jobs are running. Somewhere they must be configured.

Sure, it's all documented in systemd.timer(5). Cheers. l8er manfred

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 El 2023-02-27 a las 10:01 +0100, Carlos E. R. escribió:

On 2023-02-27 09:27, Manfred Hollstein wrote:

...

On Sun, 26 Feb 2023, 22:58:56 +0100, Carlos E. R. wrote:

...

On 2023-02-26 20:44, Manfred Hollstein wrote:

...

On Sun, 26 Feb 2023, 20:03:44 +0100, Carlos E. R. wrote:

...

On 2023-02-26 19:46, Felix Miata wrote:

...

Carlos E. R. composed on 2023-02-26 19:27 (UTC+0100):

...

...

My machine is certainly not running 24/7, yet the jobs are running. Somewhere they must be configured.

Sure, it's all documented in systemd.timer(5).

Of course, that is documented. How *SUSE uses that, is not documented.

Same as cron is documented, but how *SUSE used that and built the scripts, was not.

Ok, playing. Telcontar:~ # systemctl list-timers --all NEXT LEFT LAST PASSED UNIT ACTIVATES Mon 2023-02-27 10:18:44 CET 4min 42s left Mon 2023-02-27 10:13:44 CET 17s ago leafnode-hourly.timer leafnode-hourly.service Mon 2023-02-27 11:00:00 CET 45min left Mon 2023-02-27 10:00:01 CET 14min ago snapper-timeline.timer snapper-timeline.service Mon 2023-02-27 15:25:23 CET 5h 11min left Sat 2023-02-25 15:29:10 CET 1 day 18h ago snapper-cleanup.timer snapper-cleanup.service Mon 2023-02-27 15:30:23 CET 5h 16min left Sat 2023-02-25 15:34:10 CET 1 day 18h ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago logrotate.timer logrotate.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago mandb.timer mandb.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago mlocate.timer mlocate.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago seccheck-daily.timer seccheck-daily.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago storeBackup.timer storeBackup.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago unbound-anchor.timer unbound-anchor.service Tue 2023-02-28 00:02:34 CET 13h left Mon 2023-02-27 01:35:36 CET 8h ago check-battery.timer check-battery.service Tue 2023-02-28 01:34:58 CET 15h left Mon 2023-02-27 01:38:36 CET 8h ago backup-rpmdb.timer backup-rpmdb.service Tue 2023-02-28 01:56:53 CET 15h left Mon 2023-02-27 00:03:52 CET 10h ago backup-sysconfig.timer backup-sysconfig.service Tue 2023-02-28 04:26:00 CET 18h left Mon 2023-02-27 09:45:49 CET 28min ago leafnode-daily.timer leafnode-daily.service Wed 2023-03-01 00:00:00 CET 1 day 13h left Mon 2023-02-27 00:00:01 CET 10h ago btrfs-balance.timer btrfs-balance.service Wed 2023-03-01 00:00:00 CET 1 day 13h left Wed 2023-02-01 00:00:01 CET 3 weeks 5 days ago btrfs-scrub.timer btrfs-scrub.service Wed 2023-03-01 00:00:00 CET 1 day 13h left Mon 2023-02-27 00:00:01 CET 10h ago btrfs-trim.timer btrfs-trim.service Wed 2023-03-01 00:00:00 CET 1 day 13h left Wed 2023-02-01 00:00:01 CET 3 weeks 5 days ago seccheck-monthly.timer seccheck-monthly.service Mon 2023-03-06 00:00:00 CET 6 days left Mon 2023-02-27 00:00:01 CET 10h ago seccheck-weekly.timer seccheck-weekly.service Mon 2023-03-06 00:00:00 CET 6 days left Mon 2023-02-27 00:00:01 CET 10h ago seccheck-weekly.timer seccheck-weekly.service Mon 2023-03-06 00:16:25 CET 6 days left Mon 2023-02-27 00:31:36 CET 9h ago fstrim.timer fstrim.service n/a n/a n/a n/a mdadm-last-resort@md0.timer mdadm-last-resort@md0.service 21 timers listed. Starting with mlocate.timer Telcontar:~ # systemctl cat mlocate.timer # /usr/lib/systemd/system/mlocate.timer [Unit] Description=Daily locate database update Documentation=man:updatedb [Timer] OnCalendar=daily AccuracySec=12h Unit=mlocate.service Persistent=true [Install] WantedBy=timers.target I can add an override: systemctl edit mlocate.timer and add: [Unit] After=logrotate.service And repeat that for all daily jobs: Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago mandb.timer mandb.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago mlocate.timer mlocate.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago seccheck-daily.timer seccheck-daily.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago storeBackup.timer storeBackup.service Tue 2023-02-28 00:00:00 CET 13h left Mon 2023-02-27 00:00:01 CET 10h ago unbound-anchor.timer unbound-anchor.service Tue 2023-02-28 00:02:34 CET 13h left Mon 2023-02-27 01:35:36 CET 8h ago check-battery.timer check-battery.service Tue 2023-02-28 01:34:58 CET 15h left Mon 2023-02-27 01:38:36 CET 8h ago backup-rpmdb.timer backup-rpmdb.service Tue 2023-02-28 01:56:53 CET 15h left Mon 2023-02-27 00:03:52 CET 10h ago backup-sysconfig.timer backup-sysconfig.service Huh, I don't know why the last two are set for two hours later. Or I can wait till tonight and see if the mlocate.timer does run after logrotate.timer - -- Cheers, Carlos E. R. (from openSUSE 15.4 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCY/x29hwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfV5xsAoJYnDiqt9rsVuwjn9cKn A+SjOy0uAJ0Tu/yOLZzOVXs6jIPj0KCwjKAjcw== =DNfw -----END PGP SIGNATURE-----

On 28.02.2023 02:43, Carlos E. R. wrote:

...

systemctl edit mlocate.timer

...

and add:

...

[Unit] After=logrotate.service

Did not work.

Of course not. It does not matter when timer unit is started - you need to add ordering dependencies to the services started by these timers.

Отправлено с iPhone

28 февр. 2023 г., в 21:23, Carlos E. R. написал(а):

On 2023-02-28 18:32, Andrei Borzenkov wrote:

...

...

On 28.02.2023 02:43, Carlos E. R. wrote:

...

systemctl edit mlocate.timer

...

and add:

...

[Unit] After=logrotate.service

Did not work.

Of course not. It does not matter when timer unit is started - you need to add ordering dependencies to the services started by these timers.

Ok, could you tell me how to do that?

I googled for "ordering dependencies in systemd" and did not find it.

Do you mean to add:

After=logrotate.service

in mlocate.service instead?

Yes (assuming these are the services started by timers).

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 El 2023-02-28 a las 21:45 +0100, Carlos E. R. escribió:

On 2023-02-28 20:40, Andrei Borzenkov wrote:

...

Отправлено с iPhone

...

28 февр. 2023 г., в 21:23, Carlos E. R. <> написал(а): On 2023-02-28 18:32, Andrei Borzenkov wrote:

...

...

On 28.02.2023 02:43, Carlos E. R. wrote:

...

...

...

Do you mean to add:

After=logrotate.service

in mlocate.service instead?

Yes (assuming these are the services started by timers).

Ok, I have that in place already, we'll see what it does at midnight. Thanks.

cer@Telcontar:~> grep "Backup tool\|Do daily mandb update\|Rotate log files\|Update locate database\|Daily seccheck run" /var/log/messages ... <3.6> 2023-03-01T00:00:01.205125+01:00 Telcontar systemd 1 - - Starting Backup tool... <3.6> 2023-03-01T00:00:01.207306+01:00 Telcontar systemd 1 - - Starting Do daily mandb update... <3.6> 2023-03-01T00:00:01.208574+01:00 Telcontar systemd 1 - - Starting Rotate log files... <3.6> 2023-03-01T00:00:01.209534+01:00 Telcontar systemd 1 - - Starting Daily seccheck run... <3.6> 2023-03-01T00:00:01.214831+01:00 Telcontar systemd 1 - - Finished Backup tool. <3.6> 2023-03-01T00:00:01.274576+01:00 Telcontar systemd 1 - - Finished Rotate log files. <3.6> 2023-03-01T00:00:01.277098+01:00 Telcontar systemd 1 - - Starting Update locate database... <3.6> 2023-03-01T00:00:02.659882+01:00 Telcontar systemd 1 - - Finished Do daily mandb update. <3.6> 2023-03-01T00:00:03.419551+01:00 Telcontar systemd 1 - - Finished Daily seccheck run. <3.6> 2023-03-01T00:00:37.104963+01:00 Telcontar systemd 1 - - Finished Update locate database. Right, this is working. The "Starting Update locate database" happened after "Finished Rotate log files". Now I only have to figure out the proper ordering of things and implement them. Another day :-) - -- Cheers, Carlos E. R. (from openSUSE 15.4 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCY/6dEhwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfV1W0An0FV1/8wHoPpPHNPXB/s 6O2t7zyTAJ9n9yWnj6hUkoEcOF6O1+TcYkGlCw== =tpcr -----END PGP SIGNATURE-----

On 2023-02-26 19:30, Felix Miata wrote:

Carlos E. R. composed on 2023-02-26 09:25 (UTC+0100):

...

Felix Miata wrote:

...

...

# journalctl -b --no-hostname| grep "Feb 25" | egrep -v 'smartd|rpc' Feb 25 00:00:47 systemd[1]: Starting Do daily mandb update... Feb 25 00:00:47 systemd[1]: Starting Rotate log files... Feb 25 00:00:47 systemd[1]: Starting Update locate database... Feb 25 00:00:47 systemd[1]: logrotate.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Rotate log files. Feb 25 00:00:47 systemd[1]: mandb.service: Deactivated successfully. Feb 25 00:00:47 systemd[1]: Finished Do daily mandb update.

...

What is mystifiying to me is that the three tasks start simultaneously, and the three end in zero seconds.

Why have multiple cores dividing up one job at a time when several jobs can each run on one core without need to manage separation and recombination?

Sure, the CPU part can be divided. But these jobs are about shufling files or seeking files. The updatedb job in particular has to explore every directory in every hard disk, meaning a log of disk head movement, impeding any other hard disk job and impacting the entire machine down. There is no parallelization in hard disk. -- Cheers / Saludos, Carlos E. R. (from 15.4 x86_64 at Telcontar)