http://bugzilla.suse.com/show_bug.cgi?id=925873 Thomas Blume changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |schuetzm@gmx.net, | |thomas.blume@suse.com Flags| |needinfo?(schuetzm@gmx.net) --- Comment #3 from Thomas Blume --- (In reply to Marc Schütz from comment #2)

Created attachment 630006 [details] screenshot with debug info

Here's a new screenshot including debug info. Not very helpful AFAICS...

Seems that /bin/dracut-pre-mount hangs. Please add: rd.break=pre-mount rd.debug debug to the kernel command line and boot. When the system reached the emergency shell, get: /run/initramfs/rdsosreport.txt and attach it. Afterwards, please execute: /bin/dracut-pre-mount manually from the initrd. Does this work? If so, just type: exit. Does your machine boot then? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 --- Comment #5 from Marc Schütz --- (In reply to Thomas Blume from comment #3) (see attachment)

Afterwards, please execute:

/bin/dracut-pre-mount

manually from the initrd. Does this work?

It prints tons of debug messages to the screen and hanges, but I can kill it with Ctrl+C; I then get back to the emergency shell. Will have to experiment where exactly it hangs.

If so, just type: exit. Does your machine boot then?

Yes, it resumes correctly then, even when I don't try to run dracut-pre-mount. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Thomas Blume changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(schuetzm@gmx.net) --- Comment #7 from Thomas Blume --- (In reply to Marc Schütz from comment #6)

Hmm... I ran the commands from dracut-pre-mount manually one by one. This is the line that needs to be interrupted by Ctrl+C to get back to the shell:

getarg 'rd.break=pre-mount' 'rdbreak=pre-mount' && emergency_shell -n pre-mount "Break pre-mount"

I don't think that's the cause, because it shouldn't even be triggered without the rd.break arg. Except if "getarg" is the culprit?

The getarg function is from /lib/dracut-lib.sh and calls _dogetarg(). It must hang there somehwere. When you are in the emergency shell, please run: . /lib/dracut-lib.sh _dogetarg 'rd.break=pre-mount' 'rdbreak=pre-mount' and attach the output.

Anyway, the next command is:

source_hook pre-mount

When I run this one, it starts resuming and finishes successfully.

ok -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 --- Comment #9 from Marc Schütz --- (In reply to Thomas Blume from comment #7)

When you are in the emergency shell, please run:

. /lib/dracut-lib.sh _dogetarg 'rd.break=pre-mount' 'rdbreak=pre-mount'

and attach the output.

Unfortunately the screenshot became blurred, but it didn't hang anyway. I will try to source /dracut-state.sh first, maybe this will trigger the bug... -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Thomas Blume changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hare@suse.com, | |trenn@suse.com --- Comment #11 from Thomas Blume --- (In reply to Marc Schütz from comment #10)

Nope, the hang happens in start_emergency_shell (or similar). The last command is "systemctl start dracut-emergency.service". When I press Ctrl+C, it says that "systemd-tty-ask" was killed. (I don't see any indication that it actually asks for anything, and I wouldn't know why - the passphrase for the encrypted swap partition had already been entered before going into the emergency shell.)

Argh, sorry, I overlooked something in the rdsosreport. It shows that the kernel couldn't find the hibernation image: -->-- [ 16.883030] kaim kernel: PM: Starting manual resume from disk [ 16.884657] kaim kernel: PM: Hibernation image partition 254:0 present [ 16.884658] kaim kernel: PM: Looking for hibernation image. [ 16.886335] kaim kernel: PM: Image not found (code -22) [ 16.886335] kaim kernel: PM: Hibernation image not present or could not be loaded. --<-- Only after this I can see dracut reporting the initqueue hooks as finished: -->-- 16.883146] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@432(check_finished): . /lib/dracut/hooks/initqueue/finished/00resume.sh [ 16.894094] kaim dracut-initqueue[365]: //lib/dracut/hooks/initqueue/finished/00resume.sh@1(source): echo 254:0 [ 16.903208] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@430(check_finished): for f in '$hookdir/initqueue/finished/*.sh' [ 16.914483] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@431(check_finished): '[' /lib/dracut/hooks/initqueue/finished/90-crypt.sh = '/lib/dracut/hooks/initqueue/finished/*.sh' ']' [ 16.927973] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@432(check_finished): '[' -e /lib/dracut/hooks/initqueue/finished/90-crypt.sh ']' [ 16.939852] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@432(check_finished): . /lib/dracut/hooks/initqueue/finished/90-crypt.sh [ 16.951343] kaim dracut-initqueue[365]: //lib/dracut/hooks/initqueue/finished/90-crypt.sh@1(source): '[' -e /dev/disk/by-id/dm-uuid-CRYPT-LUKS1-425fd9fafeb94526b10feaa733605791-swap ']' [ 16.964492] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@430(check_finished): for f in '$hookdir/initqueue/finished/*.sh' [ 16.976379] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@431(check_finished): '[' '/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2f4c63085f-b79c-427d-bd4f-46ad072de4af.sh' = '/lib/dracut/hooks/initqueue/finished/*.sh' ']' [ 16.988222] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@432(check_finished): '[' -e '/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2f4c63085f-b79c-427d-bd4f-46ad072de4af.sh' ']' [ 16.999956] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@432(check_finished): . '/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2f4c63085f-b79c-427d-bd4f-46ad072de4af.sh' [ 17.011505] kaim dracut-initqueue[365]: //lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2f4c63085f-b79c-427d-bd'[' -e /dev/disk/by-uuid/4c63085f-b79c-427d-bd4f-46ad072de4af ']' [ 17.022788] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@430(check_finished): for f in '$hookdir/initqueue/finished/*.sh' [ 17.032039] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@431(check_finished): '[' '/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2fswap.sh' = '/lib/dracut/hooks/initqueue/finished/*.sh' ']' [ 17.039594] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@432(check_finished): '[' -e '/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2fswap.sh' ']' [ 17.043625] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@432(check_finished): . '/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2fswap.sh' [ 17.043752] kaim dracut-initqueue[365]: //lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2fswap.sh@1(source): '[' -e /dev/mapper/swap ']' [ 17.043872] kaim dracut-initqueue[365]: /lib/dracut-lib.sh@434(check_finished): return 0 -->-- I guess the hibernation image is found afterwards. That's why it works if you just type 'exit' from the emergency shell. Obviously, the kernel shouldn't look for the hibernate image before dracut returns from check_finished. We have a race condition here. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 --- Comment #13 from Thomas Blume --- (In reply to Marc Schütz from comment #12)

(In reply to Thomas Blume from comment #11)

...

Argh, sorry, I overlooked something in the rdsosreport. It shows that the kernel couldn't find the hibernation image:

-->-- [ 16.883030] kaim kernel: PM: Starting manual resume from disk [ 16.884657] kaim kernel: PM: Hibernation image partition 254:0 present [ 16.884658] kaim kernel: PM: Looking for hibernation image. [ 16.886335] kaim kernel: PM: Image not found (code -22) [ 16.886335] kaim kernel: PM: Hibernation image not present or could not be loaded. --<--

I actually saw this, but thought it's ok because the image is encrypted...

Does this mean that the image is encrypted separately from the swap device? If so, the initial password request from plymouth would only decrypt the swap device, but not the hibernation image on this device. Later on, plymouth seems to be blocked or stopped, so it doesn't show the prompt for decrypting the image. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Thomas Blume changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(schuetzm@gmx.net) --- Comment #15 from Thomas Blume --- (In reply to Marc Schütz from comment #14)

Now, there must be some other difference between "echo disk > /sys/power/state" and hibernating from the Cinnamon menu, because the former works. It's probably this difference that triggers the race condition.

While trying to reproduce the issue, I've found something else in the rdsosreport log. You pass the resume parameter as: resume=/dev/mapper/swap Does this really match your encrypted swap device? If configured via YaST, the encrypted swap should rather look like this: /dev/mapper/cr_swap Can you please check? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Thomas Blume changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(schuetzm@gmx.net) --- Comment #17 from Thomas Blume --- I could probably reproduce the issue now in a virtual machine. Resume after hibernate hung with encrypted swap. When I afterwards resetted the machine, it came up correctly. The hang happened, after sysroot mount, when the system was trying to activate the swap (swapon). Commenting out the swap entry in /etc/fstab fixed the hang on resume. When doing a manual swapon after resume has been finished, I see this: -->-- # swapon /dev/mapper/cr_swap swapon: /dev/mapper/cr_swap: software suspend data detected. Rewriting the swap signature. --<-- Thinking of it, it is obviously not a goot idea to modify a swap device during an resume operation. The swap device shouldn't be changed at all while a resume is in progress. The swap activation should be blocked until resume has finished. Could you please check wheter commenting out the swap entry in fstab also fixes it on your machine? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Marc Schütz changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(schuetzm@gmx.net) | --- Comment #19 from Marc Schütz --- (In reply to Thomas Blume from comment #17)

Could you please check wheter commenting out the swap entry in fstab also fixes it on your machine?

Can you tell me exactly what I should change? Just commenting out the swap entry from the running system and then suspending resulted in the hang again. Then I rebooted, and ran `mkinitrd`, but after suspend it just rebooted normally and didn't even try to resume. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Marc Schütz changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(schuetzm@gmx.net) | --- Comment #21 from Marc Schütz --- Created attachment 634497 --> http://bugzilla.suse.com/attachment.cgi?id=634497&action=edit journal of boot with debugging after successful resume Sorry for the delay, here is the output of journalctl -b of a boot with systemd.log_level=debug systemd.log_target=console rd.debug debug after a suspend/resume with echo disk > /sys/power/state -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 --- Comment #23 from Marc Schütz --- (In reply to Thomas Blume from comment #22)

So, the difference between your (working) method and suspend from Cinnamon is, that you are not using the systemd commands. Can you please now suspend the machine via the commands:

/usr/bin/systemd-sleep-grub pre

INFO: running prepare-grub Skipping grub entry #2, because it has the noresume option Skipping grub entry #4, because it has the noresume option running kernel is grub menu entry 0 (vmlinuz-3.16.7-21-desktop) preparing boot-loader: selecting entry 0, kernel /boot/3.16.7-21-desktop grub-once: running '/usr/sbin/grub2-once 0' time needed for sync: 0.3 seconds, time needed for grub: 0.1 seconds.

and then:

/usr/lib/systemd/systemd-sleep hibernate

I couldn't capture the output, but I think it complained about /etc/systemd/sleep.conf missing. Apart from that, it hibernated successfully ...

and check wheter this works?

... but hung again on boot :-(

If not, please send me the output of the above commands as well as journalctl -b after reboot.

(going to attach the journal output) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Thomas Blume changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(schuetzm@gmx.net) --- Comment #25 from Thomas Blume --- (In reply to Marc Schütz from comment #24)

Created attachment 634857 [details] journal of boot with debugging after failed resume

I can see the suspend event here: -->-- May 16 11:53:43 kaim.site kernel: PM: Syncing filesystems ... May 16 11:53:43 kaim.site kernel: done. May 16 11:54:35 kaim.site kernel: Freezing user space processes ... (elapsed 0.002 seconds) done. May 16 11:54:35 kaim.site kernel: PM: Marking nosave pages: [mem 0x0009d000-0x000fffff] May 16 11:54:35 kaim.site kernel: PM: Marking nosave pages: [mem 0xbeff6000-0xbf4f6fff] May 16 11:54:35 kaim.site kernel: PM: Marking nosave pages: [mem 0xbf781000-0xbfef2fff] May 16 11:54:35 kaim.site kernel: PM: Marking nosave pages: [mem 0xbff00000-0x100000fff] May 16 11:54:35 kaim.site kernel: PM: Marking nosave pages: [mem 0xb4000000-0xb7ffffff] May 16 11:54:35 kaim.site kernel: PM: Basic memory bitmaps created May 16 11:54:35 kaim.site kernel: PM: Preallocating image memory... done (allocated 257448 pages) May 16 11:54:35 kaim.site kernel: PM: Allocated 1029792 kbytes in 0.32 seconds (3218.10 MB/s) May 16 11:54:35 kaim.site kernel: Freezing remaining freezable tasks ... (elapsed 0.000 seconds) done. May 16 11:54:35 kaim.site kernel: Suspending console(s) (use no_console_suspend to debug) May 16 11:54:35 kaim.site kernel: i8042 kbd 00:07: System wakeup enabled by ACPI May 16 11:54:35 kaim.site kernel: serial 00:03: disabled May 16 11:54:35 kaim.site kernel: PM: freeze of devices complete after 630.432 msecs May 16 11:54:35 kaim.site kernel: PM: late freeze of devices complete after 0.580 msecs May 16 11:54:35 kaim.site kernel: PM: noirq freeze of devices complete after 0.945 msecs May 16 11:54:35 kaim.site kernel: ACPI: Preparing to enter system sleep state S4 May 16 11:54:35 kaim.site kernel: [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored May 16 11:54:35 kaim.site kernel: PM: Saving platform NVS memory May 16 11:54:35 kaim.site kernel: Disabling non-boot CPUs ... May 16 11:54:35 kaim.site kernel: kvm: disabling virtualization on CPU1 May 16 11:54:35 kaim.site kernel: smpboot: CPU 1 is now offline May 16 11:54:35 kaim.site kernel: kvm: disabling virtualization on CPU2 May 16 11:54:35 kaim.site kernel: smpboot: CPU 2 is now offline May 16 11:54:35 kaim.site kernel: kvm: disabling virtualization on CPU3 May 16 11:54:35 kaim.site kernel: smpboot: CPU 3 is now offline May 16 11:54:35 kaim.site kernel: PM: Creating hibernation image: May 16 11:54:35 kaim.site kernel: PM: Need to copy 263999 pages May 16 11:54:35 kaim.site kernel: PM: Normal pages needed: 263999 + 1024, available pages: 1808971 May 16 11:54:35 kaim.site kernel: PM: Restoring platform NVS memory May 16 11:54:35 kaim.site kernel: PCI-DMA: Resuming GART IOMMU May 16 11:54:35 kaim.site kernel: PCI-DMA: Restoring GART aperture settings May 16 11:54:35 kaim.site kernel: LVT offset 0 assigned for vector 0x400 May 16 11:54:35 kaim.site kernel: Enabling non-boot CPUs ... [....] May 16 11:54:35 kaim.site kernel: PM: restore of devices complete after 896.637 msecs May 16 11:54:35 kaim.site kernel: PM: Image restored successfully. May 16 11:54:35 kaim.site kernel: PM: Basic memory bitmaps freed May 16 11:54:35 kaim.site kernel: Restarting tasks ... done. -->-- It looks quite ok. No hint to the root cause of the problem. I've also tried on some test laptop but couldn't reproduce the issue. We need to continue narrowing it down. Please do the following tests: 1. run '/usr/bin/systemd-sleep-grub pre' afterwards suspend via 'echo disk > /sys/power/state' 2. run '/usr/lib/systemd/systemd-sleep hibernate' without a previous /usr/bin/systemd-sleep-grub Please report wheter the tests fail/succeed and provide the journal debug log from the tests. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 --- Comment #27 from Marc Schütz --- Created attachment 636694 --> http://bugzilla.suse.com/attachment.cgi?id=636694&action=edit journal of boot after /usr/bin/systemd-sleep-grub pre -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 --- Comment #29 from Marc Schütz --- Created attachment 636696 --> http://bugzilla.suse.com/attachment.cgi?id=636696&action=edit journal of next boot after /usr/lib/systemd/systemd-sleep hibernate (normal boot) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Marc Schütz changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(schuetzm@gmx.net) | --- Comment #31 from Marc Schütz --- (In reply to Thomas Blume from comment #25)

1. run '/usr/bin/systemd-sleep-grub pre' afterwards suspend via 'echo disk > /sys/power/state'

This succeeded. I attached the output of systemd-sleep-grub-pre and the journal of the entire successful boot.

2. run '/usr/lib/systemd/systemd-sleep hibernate' without a previous /usr/bin/systemd-sleep-grub

Resume failed. I could not capture the output of the command, but I attached the journal of the hanging boot (resp. the part before the suspend), the journal of the next normal boot, and a screenshot taken during the hanging resume. The last two lines on the screenshot are interesting. I pressed <Esc> after I entered the encryption passphrase to see the messages, and those lines appeared a few seconds later. It looks like the system attempts to hibernate again? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Marc Schütz changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(schuetzm@gmx.net) | --- Comment #33 from Marc Schütz --- Created attachment 637329 --> http://bugzilla.suse.com/attachment.cgi?id=637329&action=edit initrd as requested -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c36 Marc Schütz changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(schuetzm@gmx.net) | --- Comment #36 from Marc Schütz --- (In reply to Thomas Blume from comment #35)

There was a thread on the opensuse-factory mailing list about the same issue. It turned out that the solution was to uninstall the package:

suspend

Can you please check wheter you have this package installed? If so, does it help uninstalling the package?

Yes! I removed that package, and now it works fine! -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c39 Dr. Werner Fink changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kstreitova@suse.com, | |tiwai@suse.com --- Comment #39 from Dr. Werner Fink --- (In reply to Stanislav Brabec from comment #38) Good question: What are the pro/cons within the matrix entries? Is there hardware out which works only with e.g. suspend or pm-utils -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c43 Wolfgang Bauer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wbauer@tmo.at --- Comment #43 from Wolfgang Bauer --- (In reply to Stanislav Brabec from comment #42)

We already dropped pm-utils quirks from systemd, so everybody, who still depends on pm-utils quirks, should have already broken hibernation e. g. when closing the lid or using desktop menu.

But nobody complains. That's not quite true. systemd calls pm-utils again if installed since November: https://build.opensuse.org/request/show/262325

See also bug#904828. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c45 Dr. Werner Fink changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gber@opensuse.org Flags| |needinfo?(tiwai@suse.com) --- Comment #45 from Dr. Werner Fink --- @ Takashi : What do you think? Should we drop pm-utils and suspend from openSUSE Factory? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c48 Guido Berhoerster changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(gber@opensuse.org | |) | --- Comment #48 from Guido Berhoerster --- (In reply to Dr. Werner Fink from comment #46)

@ Guido : As you have re-added the pm-utils support and removed your own pm-utils-hooks-compat.sh script, I'd like to ask you if there is a newer script which can handle the users scripts below /etc/pm/sleep.d ... or does this require the pm-utils package ... or at least a pm-utils-compat package?

The script was OK, the problem was that not going through pm-utils and just relying on the script would run certain actions twice, once via systemd natively and once via my script and in no particular order. (In reply to Takashi Iwai from comment #47)

OTOH, dropping the whole pm-utils has to be done carefully. People may still use own hooks. Also, some hooks like Z99grub look really useful.

There are also packages using the hooks, I know of storage-fixup.

That said, I'm for dropping pm-utils, but not now. We need the careful evaluation of each hook and port to systemd (and eventually upstreaming), put a warning to pm-utils that it'll be deprecated soon later, and after some time, we can drop pm-utils finally.

I fully agree with that. One big problem with migrating pm-utils package and user hooks to systemd hooks is that systemd executes everything in parallel whereas the pm-utils hooks had a defined order. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c50 --- Comment #50 from Guido Berhoerster --- (In reply to Dr. Werner Fink from comment #49)

(In reply to Guido Berhoerster from comment #48)

...

One big problem with migrating pm-utils package and user hooks to systemd hooks is that systemd executes everything in parallel whereas the pm-utils hooks had a defined order.

If a master script is used for calling the others in serial order this should be solvable.

The docs are not very clear, it seems that hooks in /usr/lib/systemd/system-sleep/ are called immediately before sleeping and after all native systemd units have run and before all native services will run after waking up. Thus, if systemd takes over some of the functionality of pm-utils via native services, any user or package hooks executed via compatibility script in would be run in a different order wrt the native services than they did before. pm-utils had the convention that scripts starting with 00-49 could expect that the "usual services and userspace infrastructure is still running". I don't know how much of that is actually a problem in practice. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c53 --- Comment #53 from Wolfgang Bauer --- (In reply to Dr. Werner Fink from comment #52)

From changelog:

Wed Feb 18 05:01:38 UTC 2015 - crrodriguez@opensuse.org [...] - systemd-sleep-grub: moved to the grub2 package where it belongs as a suspend/resume hook (SR#286533) (drops prepare-suspend-to-disk.patch)

Ah, ok. But there's a reason why this has not been implemented as suspend/resume hook in the first place, to make sure it is called before/after all other hooks. See also bug 904828, comment 32 .

the question rises is this was ever accepted? It seems to be declined and revoked

https://build.opensuse.org/request/show/286533

which I'm not calling a solution!

Apparently not. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c55 --- Comment #55 from Takashi Iwai --- (In reply to Stanislav Brabec from comment #54)

Takashi_ Werner: Wolfgang: SLES12/SLED12 has no pm-utils and no pm-utils quirks. There is no regression report.

SLE12 isn't a good base for evaluation, unfortunately. How many people did test SLE12 with the very old laptop?

Also Fedora already dropped pm-utils, so it is very probable that most quirks there are obsolete.

Agreed.

Upstream of pm-utils died in 2010, openSUSE branch has only 26 commits since 2010. => No hardware newer than 5 years depends on pm-utils quirks.

So the real question is:

Does pm-utils still contain any requires quirks.

If yes, these should be ported to systemd quirks.

Right.

Look at these quirks. Evaluation is possible only for generic system quirks. Hardware quirks cannot be easily evaluated. They refer to hardware nobody of us owns and call binaries that don't even exist in openSUSE.

Yes, but what's your point? Just drop the package because we have no way to check? Common, there are just a dozen of scripts. We can review by ourselves, decide take or not, and ask on ML in doubt. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c57 --- Comment #57 from Stanislav Brabec --- We have no process to drop obsolete quirks. Most of hw quirks work-around a certain kernel bug. Most of these bugs are fixed, but nobody removed the quirk. I am just studying pm-utils in deep. It is even more complicated that I could imagine. pm-utils support very large matrix of combinations: Suspend methods: kernel: The fallback method using /sys/power/state tuxonice: If tuxonice or suspend2 kernel features are detected, then it switches to tuxonice. (I goess it is obsolete, isn't it?) suspend: (called uswsusp): If suspend binaries are detected, video and chvt quirks are disabled. suspend binaries are called instead. There is a convertor which converts list of proposed quirks to s2* command line arguments. It is expected that these quirks are executed by suspend package binaries. This mode is the default. => If suspend package is installed, pm-utils completely change its behavior. Hardware detection: DMI: Uses /sys/class/dmi/id/ (the fallback) HAL: Uses hal-get-property --udi /org/freedesktop/Hal/devices/computer --key dmidecode: Uses dmidecode -s. I hope that all of the return the same string, otherwise there is one another level of fragility. I will go through particular modules, try to understand them, check in kernel or systemd. I think that the behavior change is a nightmare for testing. I'll check suspend package, but I guess we can completely drop it, letting all quirks to be called by pm-utils (or systemd). -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c59 --- Comment #59 from Cristian Rodríguez --- (In reply to Stanislav Brabec from comment #58)

Takashi: I fully agree.

Porting of remaining quirks to systemd should be easy. I would create an extra legacy package.

No, this quirks will be rejected by systemd upstream..do not add them to the systemd package but to the relevant buggy components that need them, ideally try fixing the actual bugs instead. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c61 --- Comment #61 from Takashi Iwai --- (In reply to Cristian Rodríguez from comment #60)

(In reply to Takashi Iwai from comment #56)

...

02rtcwake: This is an own RTC alarm setup. Do we have an alternative in systemd or else?

http://joeyh.name/blog/entry/a_programmable_alarm_clock_using_systemd/ --> WakeSystem= timer setting..

How to script this at hibernation time?

...

06autofs: The autofs hook should be in systemd. I thought we have some hook in NetworkManager, though. Need to check.

Isn't this an autofs bug ? if it needs to be restarted after resume or does not respond.. maybe it misses an event ? a timer does not respond correctly ? is it using CLOCK_MONOTONIC where it should be using CLOCK_BOOTTIME ? is there a race condition ?

Well, we need to check what problem actually this has solved. I vaguely remember autofs issue in 11.x time, but haven't tracked since then.

...

75modules: This is an optional hook, and I guess some users have its own setup as a workaround. Any systemd alternative?

No, why would systemd have to unload kernel drivers before suspend ? this is to workaround buggy devices/drivers that do not come up correctly on system resume.. (usually devices that need reset-on-resume quirk afaik) Need to identify the buggy drivers ..

*We* as a distro need to provide a workaround until the kernel gets the fix. Sure, you can blame kernel, but it's no excuse to ignore the breakage. Show must go on. The system must keep working like before. (And why systemd? Because it ate others' cookies :)

...

90clock: This is also an optional hook. This might be interesting to have in systemd, too.

The top comment says:

"#!/bin/sh # Synchronize system time with hardware time. # Modern kernels handle this correctly so we skip this hook by default. " Yeah and if the kernel does not.. then there is a bug ..

True. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c63 --- Comment #63 from Dr. Werner Fink --- (In reply to Takashi Iwai from comment #61)

...

...

90clock: This is also an optional hook. This might be interesting to have in systemd, too.

The top comment says:

"#!/bin/sh # Synchronize system time with hardware time. # Modern kernels handle this correctly so we skip this hook by default. " Yeah and if the kernel does not.. then there is a bug ..

True.

This is not true for all use cases. If the kernel is in the eleven minutes mode then YES it is true (man:adjtimex(1)). This is the normal case for most systems running the ntp service. Nevertheless if the hardware clock is out of sync and is to fast/slow then this is not true even with ntp as the kernel does not touch the hardware it the offset of its system clock and the BIOS hardware clock is equal of more then 15 minutes -> /usr/src/linux/kernel/time/ntp.c in ntp_validate_timex(). Be aware that even modern hardware may have a hardware clock which is broken. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c65 Stefan Seyfried changed: What |Removed |Added ---------------------------------------------------------------------------- CC|seyfried@b1-systems.de |seife@novell.slipkontur.de --- Comment #65 from Stefan Seyfried --- not much more than "just use suspend-to-ram, everything else is too slow anyway" :-) But I will try to reproduce and debug. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c67 --- Comment #67 from Thomas Blume --- (In reply to Stefan Seyfried from comment #66)

(In reply to Thomas Blume from comment #34)

...

2. from there, mount your system root:

mount /dev/disk/by-uuid/4c63085f-b79c-427d-bd4f-46ad072de4af /sysroot

...

You can then umount /sysroot and try to continue the resume by typing: exit. If this doesn't work, you need to reset your machine.

This will possibly lead to data corruption. *NEVER EVER* touch any mounted file system between suspend and resume.

Uh, but we do an fsck on the root filesystem during resume. Would this be also bad? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c69 --- Comment #69 from Stanislav Brabec ---

90clock:

As far as I remember, this quirk causes only problems. When my machine was offline, it was much better do disable this quirk (and also the same in the shutdown sequence). It causes problems for two reasons: 1. The quirk itself expects system clock being better that hwclock. This is true for NTP, but not for many machines without NTP. 2. We have /etc/adjtime feature, that works much better for machines without NTP, as it has predictable results. If the suspend quirk saves an system time which went off over the time, user then probably wants to adjust the clock to the correct time. adjtime calculation thinks that the whole drift was caused by hwclock inaccuracy, and updates seconds per day compensation to an incorrect value. Next time, the clock will be more shifted by this time. Imagine that system clock went 2 minutes off while going to suspend. Quirk written this incorrect time to hwclock. Next day user discovers the shift, and adjusts time. /etc/adjtime will be set to 2 minutes per day. 5 das later, the system is booted, and /etc/adjtime computes need for 10 minutes correction. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c71 --- Comment #71 from Dr. Werner Fink --- To be more in detail: I had seen a lot of bug reports in both directions, that is do not touch hardware clock (as the reporters insist that it works for them) as well as the other side that in all cases the systems clock should be written back at reboot/halt/sleep (and also those reporters insist that it works for them). Hardware bugs can be evil ... -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c73 --- Comment #73 from Stefan Seyfried --- (In reply to Stanislav Brabec from comment #72)

...

06autofs:

According to the bug 916737 comment 15, the NetworkManager fix already exists.

...

90clock:

- --hctosys on resume is a work around. Good systems should always update system time.

And the kernel already did that before. It is useless IMO.

- --systohc on suspend/shutdown makes sense for systems with bad hwclock. But adjtime must be disabled then.

- --systohc on suspend/shutdown also makes sense on systems with NTP. But regular --systohc (e. g. once or twice daily) just after NTP update makes even more sense.

- regular --hctosys and regular --adjust makes sense for systems with bad system clock. But systohc on suspend/shutdown must be disabled then.

I guess we should create global sysconfig for them. This quirk can be easily transformed to systemd quirk, so the result will not be dependent on pm-utils.

Let's do that once someone is reporting a problem. Do not fix things that are not broken :-) Until then, just get rid of all the scripts, unless they are obviously still useful. I think they can all go away and we'll later fix what's broken. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c75 --- Comment #75 from Wolfgang Bauer --- (In reply to Stanislav Brabec from comment #74)

Things are broken. Right.

At the moment suspend cannot even work in Factory/Tumbleweed AFAICT, because dracut doesn't add the necessary things to the initrd any more. See http://lists.opensuse.org/opensuse-factory/2015-06/msg00258.html And it's mostly suspend that causes problems on 13.2 too, not pm-utils, as far as I can tell. This discussion is all good and nice (and necessary), but it won't help users with problems _now_. And this bug report was for 13.2, where the changes that are being discussed now would be too disruptive anyway I suppose. My proposal (again): change the defaults for pm-utils to _not_ use suspend, i.e. set SLEEP_MODULE="kernel" in /usr/lib/pm-utils/defaults (even on 13.2 with an update) to "fix" the problems at hand. The rest can still be discussed (and changed/fixed/moved/whatever) afterwards. May I go forward with this? IOW, are you ok if I create a submit request with this change of defaults? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c77 --- Comment #77 from Takashi Iwai --- (In reply to Wolfgang Bauer from comment #75)

My proposal (again): change the defaults for pm-utils to _not_ use suspend, i.e. set SLEEP_MODULE="kernel" in /usr/lib/pm-utils/defaults (even on 13.2 with an update) to "fix" the problems at hand.

Full agreed. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c79 --- Comment #79 from Stefan Seyfried --- (In reply to Stanislav Brabec from comment #74)

Stefan:

...

Let's do that once someone is reporting a problem. Do not fix things that are >> not broken :-)

Things are broken. A year ago I was debugging a customer problem, where these nice "clock improvement tricks" caused drift more than 20 years per day!

Yes, it's broken with this special hook. I'm proposing to just drop all the pm-utils tricks, *without* porting them to systemd or similar. If afterwards we find that one of them was actually doing something useful (I doubt it :-), and it being gone causes problems, then we can fix those potential problems once we see them. All of the above is for factory. For 13.2, I agree that changing the default seems sensible. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c81 --- Comment #81 from Dr. Werner Fink --- (In reply to Stanislav Brabec from comment #72)

Werner: It needs some way of global configuration:

- --hctosys on resume is a work around. Good systems should always update system time.

Normally the kernel does this at boot ... therefore in mkinitrd/dracut the kernels system clock has to warped if local time is or has to be used for the hardware clock *before* file system checks and mounting them.

- --systohc on suspend/shutdown makes sense for systems with bad hwclock. But adjtime must be disabled then.

ACK

- --systohc on suspend/shutdown also makes sense on systems with NTP. But regular --systohc (e. g. once or twice daily) just after NTP update makes even more sense.

Here the matrix has to expanded, that is if ntp service is running then the kernel may stay in elven minutes mode (bit 64 in the status line return by adjtimex is *not* set -> STA_UNSYNC[1]) there is no need to perfom system clock to hw clock.

- regular --hctosys and regular --adjust makes sense for systems with bad system clock. But systohc on suspend/shutdown must be disabled then.

I guess we should create global sysconfig for them. This quirk can be easily transformed to systemd quirk, so the result will not be dependent on pm-utils.

ACK [1] /usr/include/bits/timex.h /* Status codes (timex.status) */ #define STA_PLL 0x0001 /* enable PLL updates (rw) */ #define STA_PPSFREQ 0x0002 /* enable PPS freq discipline (rw) */ #define STA_PPSTIME 0x0004 /* enable PPS time discipline (rw) */ #define STA_FLL 0x0008 /* select frequency-lock mode (rw) */ #define STA_INS 0x0010 /* insert leap (rw) */ #define STA_DEL 0x0020 /* delete leap (rw) */ #define STA_UNSYNC 0x0040 /* clock unsynchronized (rw) */ #define STA_FREQHOLD 0x0080 /* hold frequency (rw) */ #define STA_PPSSIGNAL 0x0100 /* PPS signal present (ro) */ #define STA_PPSJITTER 0x0200 /* PPS signal jitter exceeded (ro) */ #define STA_PPSWANDER 0x0400 /* PPS signal wander exceeded (ro) */ #define STA_PPSERROR 0x0800 /* PPS signal calibration error (ro) */ #define STA_CLOCKERR 0x1000 /* clock hardware fault (ro) */ #define STA_NANO 0x2000 /* resolution (0 = us, 1 = ns) (ro) */ #define STA_MODE 0x4000 /* mode (0 = PLL, 1 = FLL) (ro) */ #define STA_CLK 0x8000 /* clock source (0 = A, 1 = B) (ro) */ -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 http://bugzilla.suse.com/show_bug.cgi?id=925873#c85 Benjamin Brunner changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(maintenance@opens | |use.org) | --- Comment #85 from Benjamin Brunner --- @Wolfgang, I'm sorry for the late reply. The update for 13.2 should be ok. Could you open a maintenancerequest please? Thanks. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=925873 Leonardo Chiquitto changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|916737 | -- You are receiving this mail because: You are on the CC list for the bug.