On Monday, 22 January 2018 07:43:17 GMT Oliver Kurz wrote:

On Sunday, 21 January 2018 23:16:17 CET CnZhx wrote:

...

Dear all,

I am wondering does anyone else have encounter a boot problem after upgrading to Snapshot 20180120.

I have skipped several snapshots and upgrade my system from Snapshot 20180109 to 20180120 by issuing,

$ sudo zypper dup

zypper did pop up some information but I thought that was ok. There were two kinds of messages: 1) Complaining that "recode" is obsoleted ( I chose to install the one from "base" repository instead); 2) Complaining that there are 3 files from kernel 4.14.12 will be overwritten by those from kernel 4.14.14. ( I chose "yes" for this overwriting)

After it finished, I reboot the system but it stopped at the line "Started Command Scheduler".

Pressing Power button to restart results the same situation. I can boot into the "pre" snapshot though. I have not issued a rollback in case I need this state.

In case it helps, my main system packages are from Tumbleweed repos with several applications such as qTox, kde-telepathy, dkms_acpi_call, acpi_call and some media related packages from other repos such as packman. Here is the list of enabled repos, ``` # | Alias | Name | Enabled | GPG Check | Refresh ---+----------------+-----------------------------+---------+----------- +--------

3 | base | base | Yes | (r ) Yes |

Yes 4 | dkms_acpi_call | dkms_acpi_call | Yes | (r ) Yes

| Yes 7 | kde_fw | kde_apps | Yes | (r ) | Yes | | Yes 13 | packman | packman | Yes | (r )

Yes | Yes 14 | qTox | qTox | Yes | (r ) Yes | Yes 16 | repo-non-oss | openSUSE-Tumbleweed-Non-Oss | Yes | (r ) Yes | Yes 17 | repo-oss | openSUSE-Tumbleweed-Oss | Yes

| (r ) Yes | Yes 19 | repo-update | openSUSE-Tumbleweed-Update | Yes | | (r ) Yes | Yes 20 | ring | ring | | Yes | | (r ) Yes | Yes ```

I wonder what the "ring" repo contains. Without the URL displayed I don't know which repo that is. You can call `zypper lr --uri` to show that as well.

Are you sure your system really "stopped" or is it just taking very very long to come up with the next point?

Hi, Oliver, Thank you for your response :-) Sorry that I hadn't made it clear with my situation. My hardware is a ThinkPad T470s with an Intel Core i7-7600U. The repos with uri are, ```# zypper lr -E --uri Repository priorities in effect: (See 'zypper lr -P' for details) 99 (default priority) : 4 repositories 100 (lowered priority) : 5 repositories # | Alias | Name | Enabled | GPG Check | Refresh | URI ---+----------------+-----------------------------+---------+----------- +--------- +------------------------------------------------------------------------------------------------- 3 | base | base | Yes | (r ) Yes | Yes | https://download.opensuse.org/repositories/Base:/System/openSUSE_Factory/ 4 | dkms_acpi_call | dkms_acpi_call | Yes | (r ) Yes | Yes | https://download.opensuse.org/repositories/home:/ramax/openSUSE_Tumbleweed/ 7 | kde_fw | kde_apps | Yes | (r ) Yes | Yes | http://download.opensuse.org/repositories/KDE:/Applications/ KDE_Frameworks5_openSUSE_Tumbleweed/ 13 | packman | packman | Yes | (r ) Yes | Yes | http://ftp.gwdg.de/pub/linux/misc/packman/suse/openSUSE_Tumbleweed/ 14 | qTox | qTox | Yes | (r ) Yes | Yes | http://download.opensuse.org/repositories/home:/ecsos:/messenger:/tox/ openSUSE_Tumbleweed/ 16 | repo-non-oss | openSUSE-Tumbleweed-Non-Oss | Yes | (r ) Yes | Yes | http://download.opensuse.org/tumbleweed/repo/non-oss/ 17 | repo-oss | openSUSE-Tumbleweed-Oss | Yes | (r ) Yes | Yes | http://download.opensuse.org/tumbleweed/repo/oss/ 19 | repo-update | openSUSE-Tumbleweed-Update | Yes | (r ) Yes | Yes | http://download.opensuse.org/update/tumbleweed/ 20 | ring | ring | Yes | (r ) Yes | Yes | http://download.opensuse.org/repositories/home:/szotsaki/openSUSE_Factory/ ``` So the "ring" is for ring-client-KDE of GNU Ring. It's the first time I ran into this situation in the 2-year time or so running Tumbleweed as my main OS. I have waited not too long (less than 3 minutes) for the boot. But I will try to wait a long time for the boot after I send this email out. But this machine has been very fast on booting. Regards, Haoxian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Yes, one drawback to using Btrfs is that it has these unpredictable bursts of high IO usage. If these occur at boot time, you're in for a long boot, and if they occur inside of a running system, everything that involves disk IO will suddenly become super-slow... Ideally, there would be a way to give these scheduled filesystem maintenance operations lower priority, so that they don't kill your system like that. I'm not familiar with Btrfs and kernel internals, so I don't know how easy / hard going in that direction would be... Cheers, Hadrien   Message original   De: zhx@cnzhx.net Envoyé: 22 janvier 2018 10:29 AM À: opensuse-factory@opensuse.org Répondre à: zhx@cnzhx.net Objet: Re: [opensuse-factory] Cannot boot after upgrading to Snapshot 20180120 On Monday, 22 January 2018 07:43:17 GMT Oliver Kurz wrote:

On Sunday, 21 January 2018 23:16:17 CET CnZhx wrote:

...

Dear all,

I am wondering does anyone else have encounter a boot problem after upgrading to Snapshot 20180120.

Are you sure your system really "stopped" or is it just taking very very long to come up with the next point?

Thanks everyone for your attention. And special thanks to Oliver for the remind. Now the machine boots right into desktop after a few seconds delay at the login prompt in booting shell. This time, it has no stop after showing the message "Started Command Scheduler" and gives the shell login prompt right away after the message. It is very awkward that the system actually has no problem booting. I booted several times last night any finally decided to boot into a read-only snapshot for checking the mailing list. So I made a false alarm. As I added later, this machine usually takes no more than 20 seconds to boot into desktop (KF5). But this time, it takes a long time to do `btrfs-balance` and other stuff making me think it actually dead. Here is the first several entries of `blame`, ``` $ systemd-analyze blame     1min 37.267s btrfs-balance.service          33.055s purge-kernels.service          33.015s dkms.service          20.642s fstrim.service           1.620s postfix.service           1.429s firewalld.service           1.298s btrfsmaintenance-refresh.service           1.153s vboxdrv.service           1.141s tor.service           1.089s apparmor.servic ``` Maybe the OS now performs maintenance at boot because even the BtrFS now uses systemd-timer for scheduling actions. Best wishes, Haoxian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Monday, 22 January 2018 09:55:53 GMT Frederic Crozat wrote:

Latest btrfsmaintenance package has switched from cron job to systemd timer, this should allow you to easily "tune" the condition on when the balancing is done, for your own need (thanks to systemd drop-in).

Thanks for the heads-up. Do you happen to know is it possible to add an option of `OnBootSec=1h` in [Timer] section of .timer file to make the process start after 1 hour after boot? Current `/usr/lib/systemd/system/btrfs-balance.timer` looks like, ``` [Unit] Description=Balance block groups on a btrfs filesystem Documentation=man:btrfs-balance [Timer] OnCalendar=monthly AccuracySec=1h Persistent=true [Install] WantedBy=timers.target ``` And because btrfs-trim.service, btrfs-scrub.service, and btrfs-balance.service are set to run after fstrim.service, maybe we could add this to `fstrim.timer`. But I am not sure whether these four service are meant to run before user login. Cheers, Haoxian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Maybe we should deliver this as a standard/default setting Schöne Grüße Axel -- Written from cell phone - excuses for typos Am 22. Januar 2018 12:43:32 MEZ schrieb Frederic Crozat :

Le lundi 22 janvier 2018 à 10:58 +0000, CnZhx a écrit :

...

On Monday, 22 January 2018 09:55:53 GMT Frederic Crozat wrote:

...

Latest btrfsmaintenance package has switched from cron job to systemd timer, this should allow you to easily "tune" the condition on when the balancing is done, for your own need (thanks to systemd drop-in).

Thanks for the heads-up. Do you happen to know is it possible to add an option  of `OnBootSec=1h` in [Timer] section of .timer file to make the process start  after 1 hour after boot?

Try creating  (untested)

/etc/systemd/system/btrfs-balance.timer.d/later.conf

containing:

[Timer] OnBootSec=1h

and run systemctl daemon-reload -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Monday, 22 January 2018 11:43:32 GMT Frederic Crozat wrote:

Le lundi 22 janvier 2018 à 10:58 +0000, CnZhx a écrit :

...

Thanks for the heads-up. Do you happen to know is it possible to add an option of `OnBootSec=1h` in [Timer] section of .timer file to make the process start after 1 hour after boot?

Try creating (untested)

/etc/systemd/system/btrfs-balance.timer.d/later.conf

containing:

[Timer] OnBootSec=1h

and run systemctl daemon-reload

I have created this file and reloaded the confs by running `sudo systemctl daemon-reload`. Do you know a way to trigger it to load on next reboot? I do not want to wait for 6 days to check it because when I check the timers, it says, ``` $ systemctl list-timers NEXT LEFT LAST PASSED UNIT ACTIVATES Mon 2018-01-22 16:00:00 GMT 56min left Mon 2018-01-22 15:00:01 GMT 3min 35s ago snapper-timeline.timer snapper-t Tue 2018-01-23 00:00:00 GMT 8h left Mon 2018-01-22 08:59:46 GMT 6h ago logrotate.timer logrotate Tue 2018-01-23 09:04:52 GMT 18h left Mon 2018-01-22 09:04:52 GMT 5h 58min ago snapper-cleanup.timer snapper-c Tue 2018-01-23 09:09:52 GMT 18h left Mon 2018-01-22 09:09:52 GMT 5h 53min ago systemd-tmpfiles-clean.timer systemd-t Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 08:59:46 GMT 6h ago btrfs-balance.timer btrfs-bal Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 08:59:46 GMT 6h ago fstrim.timer fstrim.se Thu 2018-02-01 00:00:00 GMT 1 weeks 2 days left Sun 2018-01-07 19:40:45 GMT 2 weeks 0 days ago btrfs-scrub.timer btrfs-scr 7 timers listed. Pass --all to see loaded but inactive timers, too. lines 1-11/11 (END) ``` Cheers, Haoxian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Monday, 22 January 2018 15:16:44 GMT Andrei Borzenkov wrote:

On Mon, Jan 22, 2018 at 6:07 PM, CnZhx wrote:

...

On Monday, 22 January 2018 11:43:32 GMT Frederic Crozat wrote:

...

Le lundi 22 janvier 2018 à 10:58 +0000, CnZhx a écrit :

...

Thanks for the heads-up. Do you happen to know is it possible to add an option of `OnBootSec=1h` in [Timer] section of .timer file to make the process start after 1 hour after boot?

Try creating (untested)

/etc/systemd/system/btrfs-balance.timer.d/later.conf

containing:

[Timer] OnBootSec=1h

and run systemctl daemon-reload

Note that timers are triggered when either specification expires. Which means it will run 1 hour after boot *or* as otherwise specified.

This was discussed recently, the only way to do it is to have second timer that triggers 1 hour after boot and starts service that starts fstrim.timer while leaving fstrim.timer itself disabled.

Before this comment, I set 10 minutes for the `/etc/systemd/system/btrfs- balance.timer.d/later.conf`, and reload by running `systemctl daemon-reload`. Then I reboot the machine. Observation gives has follows.

...

I have created this file and reloaded the confs by running `sudo systemctl daemon-reload`. Do you know a way to trigger it to load on next reboot? I do not want to wait for 6 days to check it because when I check the timers, it says, ``` $ systemctl list-timers NEXT LEFT LAST PASSED UNIT ACTIVATES Mon 2018-01-22 16:00:00 GMT 56min left Mon 2018-01-22 15:00:01 GMT 3min 35s ago snapper-timeline.timer snapper-t Tue 2018-01-23 00:00:00 GMT 8h left Mon 2018-01-22 08:59:46 GMT 6h ago logrotate.timer logrotate Tue 2018-01-23 09:04:52 GMT 18h left Mon 2018-01-22 09:04:52 GMT 5h 58min ago snapper-cleanup.timer snapper-c Tue 2018-01-23 09:09:52 GMT 18h left Mon 2018-01-22 09:09:52 GMT 5h 53min ago systemd-tmpfiles-clean.timer systemd-t Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 08:59:46 GMT 6h ago btrfs-balance.timer btrfs-bal Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 08:59:46 GMT 6h ago fstrim.timer fstrim.se Thu 2018-02-01 00:00:00 GMT 1 weeks 2 days left Sun 2018-01-07 19:40:45 GMT 2 weeks 0 days ago btrfs-scrub.timer btrfs-scr

7 timers listed. Pass --all to see loaded but inactive timers, too. lines 1-11/11 (END) ``` Now, the list-timers looks like this,

systemctl list-timers
NEXT                         LEFT                LAST                         
PASSED             UNIT                         ACTIVATES
Mon 2018-01-22 16:14:52 GMT  3min 46s left       Mon 2018-01-22 08:59:46 GMT  
7h ago             btrfs-balance.timer          btrfs-balance.service
Mon 2018-01-22 16:14:52 GMT  3min 46s left       n/a                          
n/a                snapper-cleanup.timer        snapper-cleanup.service
Mon 2018-01-22 16:19:52 GMT  8min left           n/a                          
n/a                systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Mon 2018-01-22 17:00:00 GMT  48min left          n/a                          
n/a                snapper-timeline.timer       snapper-timeline.service
Tue 2018-01-23 00:00:00 GMT  7h left             Mon 2018-01-22 08:59:46 GMT  
7h ago             logrotate.timer              logrotate.service
Mon 2018-01-29 00:00:00 GMT  6 days left         Mon 2018-01-22 08:59:46 GMT  
7h ago             fstrim.timer                 fstrim.service
Thu 2018-02-01 00:00:00 GMT  1 weeks 2 days left Sun 2018-01-07 19:40:45 GMT  
2 weeks 0 days ago btrfs-scrub.timer            btrfs-scrub.service

7 timers listed.
Pass --all to see loaded but inactive timers, too.

It seems that snapper-cleanup.timer, systemd-tmpfiles-clean.timer and snapper- timeline.timer are different. And after several minutes, `sudo journalctl -f -u btrfs-balance.service` gives me this, ```-- Logs begin at Mon 2018-01-22 14:53:27 GMT. -- Jan 22 16:15:01 ostp.cnzhx.net btrfs-balance.sh[9598]: Done, had to relocate 0 out of 19 chunks ... (truncated similar messages)... ``` Then, `$ systemctl list-timers` shows this, ``` NEXT LEFT LAST PASSED UNIT ACTIVATES Mon 2018-01-22 17:00:00 GMT 40min left n/a n/a snapper-timeline.timer snapper-timeline.service Tue 2018-01-23 00:00:00 GMT 7h left Mon 2018-01-22 08:59:46 GMT 7h ago logrotate.timer logrotate.service Tue 2018-01-23 16:15:01 GMT 23h left Mon 2018-01-22 16:15:01 GMT 4min 52s ago snapper-cleanup.timer snapper-cleanup.service Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 16:15:01 GMT 4min 52s ago btrfs-balance.timer btrfs-balance.service Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 08:59:46 GMT 7h ago fstrim.timer fstrim.service Thu 2018-02-01 00:00:00 GMT 1 weeks 2 days left Sun 2018-01-07 19:40:45 GMT 2 weeks 0 days ago btrfs-scrub.timer btrfs-scrub.service n/a n/a Mon 2018-01-22 16:19:53 GMT 30ms ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service 7 timers listed. Pass --all to see loaded but inactive timers, too. ``` So, `btrfs-balance` does run at 10min after boot. But I do not know what does other information indicate. This is beyond my knowledge. I guess, if the `btrfs-balance` does run regularly, maybe it will not take too much long time every time. As long as users know this could happen, it won't cause problem except for delaying the login screen sometimes. Then, a message saying that BtrFS maintenance is running is enough. Does this deserve a bug report? Regards, Haoxian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Monday, 22 January 2018 18:17:30 GMT Andrei Borzenkov wrote:

22.01.2018 19:26, CnZhx пишет: ...

...

...

...

...

...

>>> Try creating (untested)

/etc/systemd/system/btrfs-balance.timer.d/later.conf

containing:

[Timer] OnBootSec=1h

and run systemctl daemon-reload

Note that timers are triggered when either specification expires. Which means it will run 1 hour after boot *or* as otherwise specified.

...

...

Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 16:15:01 GMT 4min 52s ago btrfs-balance.timer btrfs-balance.service ...

...

So, `btrfs-balance` does run at 10min after boot. But I do not know what does other information indicate. This is beyond my knowledge.

It is run 10 minutes after boot *and* is scheduled to run in 7 days (or rather "weekly" which is Monday 00:00). So you simply added additional points in time when balance runs. Before it would run only on Monday (also if you reboot on Monday) now it will *additionally* run on every reboot :)

I am not sure what happens if you boot on Monday, will systemd "merge" these two timer runs or you get balance run twice.

I won't know because I have already delete that `later.conf` after the reboot and I rarely reboot but just sleep the laptop. But I guess it's true given that it had run after the reboot. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Monday, 29 January 2018 09:32:10 GMT H Zeng wrote:

On Monday, 22 January 2018 18:17:30 GMT Andrei Borzenkov wrote:

...

22.01.2018 19:26, CnZhx пишет: ...

...

...

...

...

>>>> Try creating (untested)

/etc/systemd/system/btrfs-balance.timer.d/later.conf

containing:

[Timer] OnBootSec=1h

and run systemctl daemon-reload

Note that timers are triggered when either specification expires. Which means it will run 1 hour after boot *or* as otherwise specified.

...

...

Mon 2018-01-29 00:00:00 GMT 6 days left Mon 2018-01-22 16:15:01 GMT 4min 52s ago btrfs-balance.timer btrfs-balance.service

...

...

So, `btrfs-balance` does run at 10min after boot. But I do not know what does other information indicate. This is beyond my knowledge.

It is run 10 minutes after boot *and* is scheduled to run in 7 days (or rather "weekly" which is Monday 00:00). So you simply added additional points in time when balance runs. Before it would run only on Monday (also if you reboot on Monday) now it will *additionally* run on every reboot :)

I am not sure what happens if you boot on Monday, will systemd "merge" these two timer runs or you get balance run twice.

...

I am going to re-enable the "1 hour delay" "later.timer" as suggested by Andrei Borzenkov in above quote and reboot right after this. I will report back two days later.

29th January, 7 days after last test on 22nd January, `btrfs-balance` ran after the laptop was resumed from sleep. During that, I had no problem logging into desktop. The `btrfs-balance` takes one full core of the CPU but not hinders the login process. The "btrfs-balance.timer" was set to run at "Thu 2018-02-01 00:00:00 GMT" after this. After resetting the "1 hour delay" configuration `/etc/systemd/system/btrfs- balance.timer.d/later.conf` as suggested by Andrei Borzenkov, I reloaded the configurations by issuing `systemctl daemon-reload`. The time of the "btrfs- balance.timer" did not change. After reboot, the time of the "btrfs-balance.timer" was "Mon 2018-01-29 11:05:35 GMT". So it was set to run "1 hour after the boot" during/after the boot process. I guess, this is not what we want. Additionally, after it ran in 1 hour later (this time it took only 1 second to finish), the "btrfs- balance.timer" was set to "Thu 2018-02-01 00:00:00 GMT" again. Obviously, the "later.conf" adds more btrfs-balance to the system even if it could prevent btrfs-balance from running during boot. Two days later, after I boot the laptop at 8:00 AM, `jounalctl` shows the btrfs-balance has run during boot. But I feel it has run after I input my password on the KDE login screen (or at least it has not prevented the login 1min 36s left btrfs-balance.timer ``` Now, the `systemd list-timers` shows that btrfs-balance has been scheduled to run at about 1 hour later. So the "1 hour delay" configuration `/etc/systemd/ system/btrfs-balance.timer.d/later.conf` does not prevent the btrfs-balance from running during boot. It also adds this process after every boot. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Mon, Jan 22, CnZhx wrote:

Maybe the OS now performs maintenance at boot because even the BtrFS now uses systemd-timer for scheduling actions.

During the first boot with a new timer systemd finds out this timer did never run before and starts it immeaditly. If somebody knows how to tell systemd that btrfs-balance.timer should run once a month, but not during the first 2 hours of a reboot ... Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & CaaSP SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Monday, 22 January 2018 11:10:05 CET Thorsten Kukuk wrote:

On Mon, Jan 22, CnZhx wrote:

...

Maybe the OS now performs maintenance at boot because even the BtrFS now uses systemd-timer for scheduling actions.

During the first boot with a new timer systemd finds out this timer did never run before and starts it immeaditly. If somebody knows how to tell systemd that btrfs-balance.timer should run once a month, but not during the first 2 hours of a reboot ...

Thorsten

I found no way to specify this directly, but what might work is using a "blocking" target, see e.g. the time-sync.target. For the beginning, putting "After=default.target" should mitigate the problem. It may slow down the login, but with some luck the user already had the display manager idle around a bit. Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen home: +49 241 53809034 mobile: +49 151 50412019