[Bug 937237] New: Timed out waiting for device dev-hvc0.device
http://bugzilla.suse.com/show_bug.cgi?id=937237 Bug ID: 937237 Summary: Timed out waiting for device dev-hvc0.device Classification: openSUSE Product: openSUSE Factory Version: 201505* Hardware: PowerPC-64 OS: openSUSE 13.2 Status: NEW Severity: Normal Priority: P5 - None Component: Installation Assignee: yast2-maintainers@suse.de Reporter: dvaleev@suse.com QA Contact: jsrain@suse.com CC: mjwolf@us.ibm.com, normand@linux.vnet.ibm.com Found By: --- Blocker: --- Trying to boot rescue mode from Tumbleweed snapshot,gives me: [ TIME ] Timed out waiting for device dev-hvc0.device. [DEPEND] Dependency failed for Serial Getty on hvc0. [ OK ] Reached target Login Prompts. [ OK ] Started /etc/init.d/after.local Compatibility. Starting /etc/init.d/after.local Compatibility... [FAILED] Failed to start System Logging Service. See "systemctl status syslogd.service" for details. [DEPEND] Dependency failed for System Kernel Logging Service. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c1
--- Comment #1 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c2
Arvin Schnell
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c3
Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
Chenzi Cao
http://bugzilla.suse.com/show_bug.cgi?id=937237
Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c5
Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c6
Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c7
Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c8
Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c9
--- Comment #9 from Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c10
Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c11
--- Comment #11 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c12
Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c14
--- Comment #14 from Dr. Werner Fink
Also I don't see system boot output during boot
Wrong order of console=hvc0,38400 console=tty0 switch the order to console=tty0 console=hvc0,38400 as with this the last one becomes the main console. Or use plymouth to see boot messages on both devices. There is no blogd anymore and systemd write to /dev/console which is printed on the main device only. Beside this I'd like to see systemctl status --all dev-hvc0.device (In reply to Dinar Valeev from comment #8)
I don't see the same behaviour on installed system.
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c15
--- Comment #15 from Dr. Werner Fink
I don't see the same behaviour on installed system.
.... this looks more like a problem in the installation system. Maybe the file /usr/lib/udev/rules.d/99-systemd.rules is missed in the installation system and/or the line: SUBSYSTEM=="tty", KERNEL=="tty[a-zA-Z]*|hvc*|xvc*|hvsi*|ttysclp*|sclp_line*|3270/tty*", TAG+="systemd" does miss `hvc*|' -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c16
--- Comment #16 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c17
--- Comment #17 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c18
--- Comment #18 from Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c19
--- Comment #19 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c20
--- Comment #20 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c21
--- Comment #21 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c22
Dr. Werner Fink
Oct 05 15:07:14 Rescue systemd[1]: systemd-udevd.service failed. Oct 05 15:07:14 Rescue systemd[1]: systemd-udevd.service has no holdoff time, scheduling restart.
This looks more like a broken library on the rescue system for PPC64. After asking google with "holdoff systemd" I found https://bugzilla.redhat.com/show_bug.cgi?id=1121419 which describes a bug in glibc in i686 with a similar behaviour. Related to this I have found https://bugzilla.redhat.com/show_bug.cgi?id=1120473#c1 -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c23
--- Comment #23 from Dr. Werner Fink
which describes a bug in glibc in i686 with a similar behaviour. Related to this I have found
the bugzillas I provided shows that the gcc had misscompiled the glibc, therefore I have added the maintainers of gcc to the carbon copy list. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c24
--- Comment #24 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c25
Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c26
--- Comment #26 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c27
--- Comment #27 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c28
--- Comment #28 from Jan Engelhardt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c29
--- Comment #29 from Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c30
--- Comment #30 from Dinar Valeev
If something like console=ttyS0 is on the command line, the getty generator kicks in, creating the particular "wants" link:
/run/systemd/generator/getty.target.wants/serial-getty@ttyS0.service
This ought to work similarly when having console=hvc0 on the line, and that is what it does, at least, the last time I checked with xvc0 (xen) on systemd-210.
I tried to boot resuce with console=hvc0, doesn't help with the currect installation-images code -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c31
--- Comment #31 from Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c32
--- Comment #32 from Jan Engelhardt
(systemd-getty-generator.8): It will also instantiate serial-getty@.service instances for virtualizer consoles, if execution in a virtualized environment is detected.
So maybe that virtual environment is not detected properly? Run systemd-detect-virt(8). -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c33
--- Comment #33 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c34
--- Comment #34 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c35
--- Comment #35 from Jan Engelhardt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c36
--- Comment #36 from Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c37
--- Comment #37 from Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c38
--- Comment #38 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c39
--- Comment #39 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c40
--- Comment #40 from Dr. Werner Fink
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c41
Thomas Blume
(In reply to Dinar Valeev from comment #39)
Hmmm ... then something is missed in the namespace/context for starting systemd-udevd like a sticky /tmp
The first error I see in the boot log is this: <7>[ 75.875642] systemd[1]: Failed to set up the root directory for shared mount propagation: Invalid argument seems to be in context with: https://github.com/jumpstarter-io/pkgbuilds/blob/master/systemd/0001-Revert-... Maybe there is a problem making the snapshot root shared? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c42
--- Comment #42 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c43
--- Comment #43 from Dr. Werner Fink
From comment
/* Mark the root directory as shared in regards to mount * propagation. The kernel defaults to "private", but we think * it makes more sense to have a default of "shared" so that * nspawn and the container tools work out of the box. If * specific setups need other settings they can reset the * propagation mode to private if needed. */ I like to guess that we need this feature as container are in use even for openSUSE... also from src/basic/virt.c function detect_container() I'd like to suppose that mkdir -p /run/systemd echo systemd-nspawn > /run/systemd/container could help in the rescue system -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c44
--- Comment #44 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c45
--- Comment #45 from Thomas Blume
Nope, doest help.
Could somebody please take a look at it?
vncviewer cabernet.arch:55
I still see the error message: Failed to set up the root directory for shared mount propagation: Invalid argument in the journal log. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c46
--- Comment #46 from Thomas Blume
I still see the error message:
Failed to set up the root directory for shared mount propagation: Invalid argument
in the journal log.
btw. I suspect a context with bug 902226. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c47
--- Comment #47 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c48
--- Comment #48 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c49
--- Comment #49 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c50
--- Comment #50 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c51
--- Comment #51 from Jan Engelhardt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c52
--- Comment #52 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c53
--- Comment #53 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c54
--- Comment #54 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c55
--- Comment #55 from Thomas Blume
Ok, rescue is now its own file system and the systemd error is gone.
HOWEVER, it now mounts the root-fs ro. Any idea why and how to stop this?
I assume that the rescue system doesn't have an fstab entry for system root mount. If so, systemd-fstab-generator would fail to remount system root rw after the initrd. You could try with the boot parameters: rw and/or: rootflags=rw -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c56
--- Comment #56 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c57
--- Comment #57 from Thomas Blume
I thought so but there is an entry; but probably a wrong one.
How would an entry for a tmpfs / look like?
Hm, not really sure either. When you are logged in as root, there is a systemd tmpfs mount for run-user-0.mount. On my machine, it has the following options: Options=rw,nosuid,nodev,relatime,size=203868k,mode=700 Don't know wheter all would apply for system root though. Can you paste the output of: systemctl cat / systemctl status / -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c58
--- Comment #58 from Thomas Blume
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c59
Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c60
--- Comment #60 from Jan Engelhardt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c61
--- Comment #61 from Thomas Blume
Hm, seems so.
Postponing the linuxrc changes until we have this in systemd.
As a short term solution: would adding a static getty symlink for hvc0 do?
I'm just building a systemd version with the (ported) patch from comment#58 here: https://build.opensuse.org/package/show/home:tsaupe:branches:Base:System:bsc... you might want to give it a try when it is finished. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c62
--- Comment #62 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c63
--- Comment #63 from Thomas Blume
Hm, I just checked with qemu and sle12-sp1.
I also saw the 'shared mount propagation error' in the journal.
But /dev/hdc0 _does_ exist. Also a getty was running there and I could login there after fixing /etc/securetty.
So why does that not work in openSUSE?
This is, because: http://cgit.freedesktop.org/systemd/systemd/commit/?id=b3ac5f8cb98757416d866... is not in systemd-210. Basically the same as if you apply the patch from comment#41. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c64
--- Comment #64 from Thomas Blume
This is, because:
http://cgit.freedesktop.org/systemd/systemd/commit/ ?id=b3ac5f8cb98757416d8660023d6564a7c411f0a0
is not in systemd-210. Basically the same as if you apply the patch from comment#41.
Sorry, my fault, it is in systemd-210, but it is limited to containers: -->-- if (detect_container(NULL) <= 0) if (mount(NULL, "/", NULL, MS_REC|MS_SHARED, NULL) < 0) log_warning("Failed to set up the root directory for shared mount propagation: %m"); --<-- -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c65
--- Comment #65 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c66
--- Comment #66 from Dinar Valeev
Hm, I just checked with qemu and sle12-sp1.
I also saw the 'shared mount propagation error' in the journal.
But /dev/hdc0 _does_ exist. Also a getty was running there and I could login there after fixing /etc/securetty.
So why does that not work in openSUSE?
Yes, it is only visible in openSUSE -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c67
--- Comment #67 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c68
--- Comment #68 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c69
--- Comment #69 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c70
--- Comment #70 from Thomas Blume
On tumbleweed I see comment 20; but /dev/hvc0 exists.
So, my question still is: why can systemd on sle12-sp1 cope with the situation but the one in tumbleweed can't? And, should systemd be fixed or the setup?
Checked the difference between SLES12SP1 and tumbleweed. /usr/lib/systemd/system/systemd-udevd.service on tumbleweed has this: MountFlags=slave and the manpage says: -->-- MountFlags= Takes a mount propagation flag: shared, slave or private, which control whether mounts in the file system namespace set up for this unit's processes will receive or propagate mounts or unmounts. --<-- I guess this makes systemd-udevd fail. The SLES12SP1 service file doesn't have this setting. The missing hvc0 device is most probably only a side effect of systemd-udevd.service failing to start. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c71
--- Comment #71 from Dinar Valeev
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c72
--- Comment #72 from Thomas Blume
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c73
--- Comment #73 from Jan Engelhardt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c74
--- Comment #74 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c75
--- Comment #75 from Jan Engelhardt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c76
--- Comment #76 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c77
--- Comment #77 from Thomas Blume
Tried that already; see comment 52.
I guess you will need the patched systemd version from comment#61 to make this work. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c78
--- Comment #78 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c79
--- Comment #79 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c80
--- Comment #80 from Steffen Winterfeldt
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c81
--- Comment #81 from Dr. Werner Fink
From man:pam_securetty(8)
.... It will also allow root logins on the tty specified with console= switch on the kernel command line and on ttys from the /sys/class/tty/console/active. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=937237
http://bugzilla.suse.com/show_bug.cgi?id=937237#c82
Steffen Winterfeldt
participants (1)
-
bugzilla_noreply@novell.com