[Bug 852652] New: Xen DomU, upgraded from Opensuse 12.3 -> 13.1, takes ~ 3mins to boot, up from ~ 20 secs in 12.3
https://bugzilla.novell.com/show_bug.cgi?id=852652 https://bugzilla.novell.com/show_bug.cgi?id=852652#c0 Summary: Xen DomU, upgraded from Opensuse 12.3 -> 13.1, takes ~ 3mins to boot, up from ~ 20 secs in 12.3 Classification: openSUSE Product: openSUSE 13.1 Version: Final Platform: x86-64 OS/Version: openSUSE 13.1 Status: NEW Severity: Normal Priority: P5 - None Component: Xen AssignedTo: jdouglas@suse.com ReportedBy: ar16@imapmail.org QAContact: qa-bugs@suse.de Found By: --- Blocker: --- User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0 I've an Opensuse 13.1 DomU on a 13.1 Dom0. Both host & guest were upgraded from 12.3. The Opensuse 13.1 Xen DomU Guest is booting successfully, but taking a long time (~3 mins; up from ~20sec on 12.3). Where the delay is is clear; here's img output from "systemd-analyze plot" -> http://pbrd.co/17UxVRS Also, systemd-analyze blame 1min 222ms kmod-static-nodes.service 1min 152ms dev-mqueue.mount 1min 142ms systemd-udev-root-symlink.service 1min 141ms systemd-vconsole-setup.service 1min 140ms systemd-modules-load.service 3.715s network.service 2.242s quotad.service 2.223s autofs.service 2.201s network@eth0.service 1.068s nfs.service 625ms nfsserver.service 489ms systemd-udev-settle.service 319ms rsyslog.service 312ms rpcbind.service 285ms systemd-logind.service 234ms lvm2-activation-early.service 213ms systemd-tmpfiles-clean.service 174ms systemd-fsck@dev-xvda1.service 107ms lvm2-activation.service 104ms sshd-custom.service 80ms systemd-tmpfiles-setup-dev.service 71ms boot.mount 66ms systemd-sysctl.service 65ms systemd-remount-fs.service 58ms user@0.service 56ms systemd-random-seed.service 47ms systemd-udevd.service 44ms systemd-readahead-done.service 41ms systemd-user-sessions.service 38ms systemd-udev-trigger.service 38ms sys-kernel-debug.mount 35ms systemd-update-utmp.service 32ms systemd-tmpfiles-setup.service 32ms systemd-journal-flush.service 27ms systemd-update-utmp-runlevel.service 26ms var-run.mount 22ms dev-xvdb1.swap 12ms rc-local.service 12ms var-lock.mount 5ms sys-fs-fuse-connections.mount WHY, the delay is occurring, isn't clear. Here's `journalctl -xb` -> http://pastebin.com/YGjAMnaP (~ 1700 lines). @ IRC #systemd, this was called out as a 'possible problem': ... Nov 25 10:45:31 os131guest systemd[1]: Job dev-xvc-1.device/start finished, result=timeout Nov 25 10:45:31 os131guest systemd[1]: Timed out waiting for device dev-xvc-1.device. Nov 25 10:45:31 os131guest systemd[1]: Job serial-getty@xvc-1.service/start finished, result=dependency ... The .cfg @ Dom0 that launches this Guest contains my usual: cat os131test.cfg name = 'template' builder = 'linux' bootloader = 'pygrub' bootargs = '' disk = [ 'phy:/dev/VG0/vm_os131boot,xvda,w','phy:/dev/VG0/vm_os131swap,xvdb,w','phy:/dev/VG0/vm_os131root,xvdc,w',] root = '/dev/xvdc1' extra = 'textmode=1 xencons=xvc0 noirqdebug elevator=noop' localtime = 0 on_shutdown = 'destroy' on_reboot = 'restart' on_crash = 'destroy' ... There are several delays/stalls seen in the referenced log. Atm, I'm only seeing these boot-up delays on Xen Guests. Dom0 has no similar delays, neither do any non-Xen machines. Reproducible: Always Steps to Reproduce: 1. 2. 3. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c
Charles Arnold
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c1
Olaf Hering
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c2
Dr. Werner Fink
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c3
--- Comment #3 from Dr. Werner Fink
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c4
--- Comment #4 from Dr. Werner Fink
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c5
--- Comment #5 from Olaf Hering
Also the setup of /dev/console can be viewed in the files
/proc/consoles
or
/sys/class/tty/console/active
please note that things like `tty-1' or `xvc-1' are nore device names but names of the console name and console index[1].
[1] http://www.mail-archive.com/systemd-devel@lists.freedesktop.org/msg04035.htm...
I get "xvc-1", which according to one of the mails in the thread above could be a wrong index when sprintf("xvc%d", idx) is used somewhere in the code. I think thats not new, but happens to trigger now. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c6
systemd maintainers
(In reply to comment #3)
I get "xvc-1", which according to one of the mails in the thread above could be a wrong index when sprintf("xvc%d", idx) is used somewhere in the code. I think thats not new, but happens to trigger now.
Hmmm ... please show me (Werner) the result of ls -l /sys/class/tty/xvc* thanks. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c7
--- Comment #7 from Olaf Hering
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c8
--- Comment #8 from Olaf Hering
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c9
--- Comment #9 from Dr. Werner Fink
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c10
--- Comment #10 from Olaf Hering
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c11
Olaf Hering
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c
Olaf Hering
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c12
--- Comment #12 from A R
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c13
--- Comment #13 from Jan Beulich
Whats appearently missing is a call to add_prefered_console. Otherwise ->index will not be updated in register_console. This is what pvops does, I just copied it verbatim. With this change booting a PV guest with empty cmdline shows xvc0. And in addition root login via console works as well because now systemd can appearently recognize the console device.
drivers/xen/console/console.c already has such a call. But it appearently only triggers when booted with xencons=hvc (which is probably also the workaround for the initial report for the time being).
Jan, what do you think about such a change?
I agree with the intention, but not with how it is being carried out. Instead I'd see the HVC-specific code in drivers/xen/console/console.c to be generalized (i.e. moved out of the switch) - the changeset having introduced this (http://xenbits.xen.org/hg/linux-2.6.18-xen.hg/rev/a070228ac76e) provides no clarification at all why this is being done for HVC only. Doing it there has the advantage of also properly taking xc_num into consideration and not adding consoles as preferred that won't ever get registered. And from your description I understand that all that is needed is that it's being done before calling register_console(). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c14
--- Comment #14 from Jan Beulich
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c15
--- Comment #15 from Dr. Werner Fink
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c16
Olaf Hering
(In reply to comment #14)
@ Olalf : does this really help systemd to detec the correct serial console and to enable it with the correct serivce link?
Yes, it does. There was just a disconnect between the kernel device name "xvc0" and yet another internal name "xcv-1". This looks like a shortcoming in the kernel to not use a single name for both. But other places also call also add_preferred_console before register_console. So for short: the patch fixes this for our kernel-xen. It will (most likely) also fix the first stage install which currently fails to run /sbin/inst_setup because showconsole or setcsid (?) fails. The workaround for this is to boot with xencons=hvc into the installer of released products. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c17
--- Comment #17 from Dr. Werner Fink
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c18
--- Comment #18 from Olaf Hering
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c19
--- Comment #19 from A R
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c20
Jan Beulich
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c21
--- Comment #21 from A R
Rather than inlining lots of not immediately necessary information
I can't recall when you've commented on a bug where you're not unhappy about something that's being provided. Try this: after updating, the system boots faster. What that means is up to you to decide. I'll not bother posting additional information/feedback unless you request very specific instructions details. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c
A R
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c22
Jan Beulich
I can't recall when you've commented on a bug where you're not unhappy about something that's being provided.
I sort of expected a response like this. I could return something along those lines on the style of bug reports that come from you (interestingly, iirc, under varying names/mail addresses, and apparently rarely with a proper name associated with it). But I won't...
Try this: after updating, the system boots faster.
What that means is up to you to decide.
That's not very helpful, as it doesn't make in any way clear whether there is still an issue remaining (i.e. things only got better, but not good enough), just like pushing out a table of numbers possibly having a meaning to you but not to me is not helpful. I appreciate you trying to find the cause of the problems you report, but you should try to report your findings in a way that someone lacking the context you have can understand.
I'll not bother posting additional information/feedback unless you request very specific instructions details.
Specific request: Is there still a reason to keep this bug open? If so, please explain what the remaining issue is in an understandable way (i.e. with the component set to Xen, you can't expect the person dealing with it to be a systemd expert - I need to know what the Xen aspects to it are if you continue to think this is a Xen problem). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c23
A R
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c24
--- Comment #24 from Olaf Hering
Created an attachment (id=570589) --> (http://bugzilla.novell.com/attachment.cgi?id=570589) [details] tentative fix
(against linux-2.6.18-xen.hg)
yes, this patch is working for me, it gets rid of xvc-1 in /proc/consoles -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c25
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c26
--- Comment #26 from Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c27
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c28
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c29
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c30
--- Comment #30 from Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c31
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c32
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c33
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c34
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c35
--- Comment #35 from Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c
Horst Venzke
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c
Swamp Workflow Management
https://bugzilla.novell.com/show_bug.cgi?id=852652
https://bugzilla.novell.com/show_bug.cgi?id=852652#c36
--- Comment #36 from Swamp Workflow Management
http://bugzilla.novell.com/show_bug.cgi?id=852652
Swamp Workflow Management
http://bugzilla.novell.com/show_bug.cgi?id=852652
Swamp Workflow Management
participants (1)
-
bugzilla_noreply@novell.com