[opensuse-virtual] Best Practices / Details for Serial Console debug-level logging when using Xen + systemd ?
I boot Opensuse 12.3 + Xen 4.3 using Grub1. Reading http://en.opensuse.org/SDB:Systemd#Getting_debug_from_systemd There's a note to ln -s /usr/lib/systemd/system/serial-getty@.service /etc/systemd/system/getty.target.wants/serial-getty@ttyS0.service On my system, I have just: find /usr/lib/systemd/system/ -name *serial-getty* /usr/lib/systemd/system/serial-getty@.service i.e., I do NOT have the symlink for startup in place. I currently have serial console(s) enabled using the config below. It seems to work, but it's evolved to this over time, tweaked as kernel- and xen-versions have changed. I suspect it needs some clean up. My goal is to have grub, early boot, Xen and detailed, debug-level, complete boot logged to both a serial console & video out. What, if anything, SHOULD be changed in the config below for correctness? Specifically, what's recommended for getting the 'right info' form boot process for informative reporting on bugs? cat /boot/grub/menu.lst ... serial --unit=0 --speed=57600 --word=8 --parity=no --stop=1 terminal --timeout=6 serial console ... title Xen root (hd0,0) kernel /xen.gz vga=gfx-1280x1024x16 ... systemd.log_level=debug systemd.log_target=syslog-or-kmsg loglvl=debug guest_loglvl=debug com1=57600,8n1,pci console=vga,com1 console_timestamps ... apic_verbosity=verbose iommu=verbose ... module /vmlinuz-xen log_buf_len=4M console=tty0 console=xvc0,57600n8 xencons=tty earlyprintk=xen vga=0x31a ... showopts noquiet module /initrd-xen ... -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
As I've been working on figuring out serial console setup for Xen, I now notice an OOPS on boot [ 34.971630] ------------[ cut here ]------------ [ 34.971639] WARNING: at /home/abuild/rpmbuild/BUILD/kernel-xen-3.7.10/linux-3.7/drivers/tty/tty_io.c:1430 tty_init_dev+0x19a/0x1d0() [ 34.971640] Hardware name: System Product Name [ 34.971642] tty_init_dev: tty driver does not set tty->port. This will crash the kernel later. Fix the driver! [ 34.971643] Modules linked in: xen_scsibk blkbk domctl blkback_pagemap netbk xenbus_be gntdev evtchn binfmt_misc snd_hda_codec_via snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sr_mod radeon ttm drm_kms_helper soundcore drm ppdev ata_generic i2c_algo_bit serio_raw k10temp snd_page_alloc pata_atiixp sp5100_tco i2c_piix4 i2c_core shpchp sg r8169 sky2 pci_hotplug parport_pc parport floppy 8250_core asus_atk0110 button n_hdlc slhc nfsd auth_rpcgss nfs_acl nfs fscache lockd sunrpc autofs4 hid_generic usbhid hid linear ohci_hcd ehci_hcd usbcore usb_common scsi_dh_alua scsi_dh_rdac scsi_dh_hp_sw scsi_dh_emc scsi_dh xenblk cdrom xennet sata_sil24 serial_core cpuid edd fan thermal processor thermal_sys hwmon dummy sha512_generic sha256_generic sha1_generic ablk_helper cryptd lrw aes_x86_64 xts gf128mul xfs btrfs zlib_deflate libcrc32c dm_snapshot dm_crypt dm_mod raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx raid10 raid1 raid0 [ 34.971684] Pid: 1057, comm: rs:main Q:Reg Not tainted 3.7.10-1.16-xen #1 [ 34.971685] Call Trace: [ 34.971697] [<ffffffff80008ae5>] dump_trace+0x85/0x1c0 [ 34.971702] [<ffffffff804f8449>] dump_stack+0x69/0x6f [ 34.971707] [<ffffffff8002edc9>] warn_slowpath_common+0x79/0xc0 [ 34.971711] [<ffffffff8002eec5>] warn_slowpath_fmt+0x45/0x50 [ 34.971714] [<ffffffff8031f38a>] tty_init_dev+0x19a/0x1d0 [ 34.971719] [<ffffffff8031fca8>] tty_open+0x318/0x580 [ 34.971723] [<ffffffff801376df>] chrdev_open+0xbf/0x1e0 [ 34.971727] [<ffffffff80131526>] do_dentry_open+0x206/0x290 [ 34.971731] [<ffffffff80140391>] do_last+0x2e1/0xed0 [ 34.971735] [<ffffffff80141f03>] path_openat+0xc3/0x4f0 [ 34.971738] [<ffffffff80142cf4>] do_filp_open+0x44/0xb0 [ 34.971741] [<ffffffff801326e3>] do_sys_open+0xf3/0x1e0 [ 34.971744] [<ffffffff8050ad0b>] system_call_fastpath+0x1a/0x1f [ 34.971751] [<00007f4d6b6f2b0d>] 0x7f4d6b6f2b0c [ 34.971753] ---[ end trace ca20eeab42859431 ]--- [ 34.971761] Warning: dev (tty10) tty->count(0) != #fd's(1) in tty_open [ 34.994123] usbcore: registered new interface driver usbback [ 35.016535] pciback: backend is vpci [ 35.042706] blktap_device_init: blktap device major 252 [ 35.042712] blktap_ring_init: blktap ring major: 250 [ 35.074257] nbd: registered device at major 43 [ 35.092721] systemd-logind[964]: Watching system buttons on /dev/input/event2 (Power Button) [ 35.092743] ------------[ cut here ]------------ The system continues to boot, and I can get to Dom0 + DomU working. I'm NOT sure if this is related to some `shutdown` issues I'm having; trying to figure out what's related, and what's not. Jan, I'm cc'ing you just to 1st ask -- does this ^^^ discussion/oops belong here in -virtual, or over on the -kernel side? Thanks. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 28.06.13 at 22:58, ar16 <ar16@imapmail.org> wrote: As I've been working on figuring out serial console setup for Xen, I now notice an OOPS on boot
[ 34.971630] ------------[ cut here ]------------ [ 34.971639] WARNING: at /home/abuild/rpmbuild/BUILD/kernel-xen-3.7.10/linux-3.7/drivers/tty/tty_io.c:14 30 tty_init_dev+0x19a/0x1d0() [ 34.971640] Hardware name: System Product Name [ 34.971642] tty_init_dev: tty driver does not set tty->port. This will crash the kernel later. Fix the driver! [ 34.971643] Modules linked in: xen_scsibk blkbk domctl blkback_pagemap netbk xenbus_be gntdev evtchn binfmt_misc snd_hda_codec_via snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sr_mod radeon ttm drm_kms_helper soundcore drm ppdev ata_generic i2c_algo_bit serio_raw k10temp snd_page_alloc pata_atiixp sp5100_tco i2c_piix4 i2c_core shpchp sg r8169 sky2 pci_hotplug parport_pc parport floppy 8250_core asus_atk0110 button n_hdlc slhc nfsd auth_rpcgss nfs_acl nfs fscache lockd sunrpc autofs4 hid_generic usbhid hid linear ohci_hcd ehci_hcd usbcore usb_common scsi_dh_alua scsi_dh_rdac scsi_dh_hp_sw scsi_dh_emc scsi_dh xenblk cdrom xennet sata_sil24 serial_core cpuid edd fan thermal processor thermal_sys hwmon dummy sha512_generic sha256_generic sha1_generic ablk_helper cryptd lrw aes_x86_64 xts gf128mul xfs btrfs zlib_deflate libcrc32c dm_snapshot dm_crypt dm_mod raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx raid10 raid1 raid0 [ 34.971684] Pid: 1057, comm: rs:main Q:Reg Not tainted 3.7.10-1.16-xen #1 [ 34.971685] Call Trace: [ 34.971697] [<ffffffff80008ae5>] dump_trace+0x85/0x1c0 [ 34.971702] [<ffffffff804f8449>] dump_stack+0x69/0x6f [ 34.971707] [<ffffffff8002edc9>] warn_slowpath_common+0x79/0xc0 [ 34.971711] [<ffffffff8002eec5>] warn_slowpath_fmt+0x45/0x50 [ 34.971714] [<ffffffff8031f38a>] tty_init_dev+0x19a/0x1d0 [ 34.971719] [<ffffffff8031fca8>] tty_open+0x318/0x580 [ 34.971723] [<ffffffff801376df>] chrdev_open+0xbf/0x1e0 [ 34.971727] [<ffffffff80131526>] do_dentry_open+0x206/0x290 [ 34.971731] [<ffffffff80140391>] do_last+0x2e1/0xed0 [ 34.971735] [<ffffffff80141f03>] path_openat+0xc3/0x4f0 [ 34.971738] [<ffffffff80142cf4>] do_filp_open+0x44/0xb0 [ 34.971741] [<ffffffff801326e3>] do_sys_open+0xf3/0x1e0 [ 34.971744] [<ffffffff8050ad0b>] system_call_fastpath+0x1a/0x1f [ 34.971751] [<00007f4d6b6f2b0d>] 0x7f4d6b6f2b0c [ 34.971753] ---[ end trace ca20eeab42859431 ]--- [ 34.971761] Warning: dev (tty10) tty->count(0) != #fd's(1) in tty_open
As the changes to drivers/xen/console/console.c show that patches.xen/xen3-patch-3.7 does, the Xen console driver has been modified to deal with that situation (as obviously I had seen those warnings too earlier on). However, posting just the fragment above makes this all guesswork anyway, as it remains unclear what "console=" and "xencons=" options you may have on your kernel command line.
I'm NOT sure if this is related to some `shutdown` issues I'm having; trying to figure out what's related, and what's not.
Jan, I'm cc'ing you just to 1st ask -- does this ^^^ discussion/oops belong here in -virtual, or over on the -kernel side?
I very much doubt that kernel side console handling issues matter with reboot, which is done entirely in the hypervisor (even more so that you get a message from the hypervisor stating that it at least started the reboot sequence, i.e. control wouldn't return to the Dom0 kernel after that point). Jan -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
To get this issue, and my other post's restart/shutdown problems, sorted out -- I'm simplifying. Step 1, proper serial port function. I've built a clean box, running Opensuse 12.3/kernel-xen + grub1 + Xen 4.3 + systemd 195, trying to get Serial console output working. It's not. With the following config, I can't get any serial output from Dom0 over cable at all. I can't find a relevant, up-to-date document for the version 'mix' above, and have cribbed from Xen wiki docs & mailing list posts. Something's clearly still wrong. With a goal of being well-informed rather than lucky, I'd appreciate some guidance with getting the consoles (tty0/ttyS0/xvc0) properly configured & working for serial port use on Opensuse/Xen. Here's what I've done for the current instance; I've installed lsb_release -rd Description: openSUSE 12.3 (x86_64) Release: 12.3 uname -a Linux xentest.loc 3.7.10-1.16-xen #1 SMP Fri May 31 20:21:23 UTC 2013 (97c14ba) x86_64 x86_64 x86_64 GNU/Linux rpm -qa | egrep -i "^xen|^systemd-1" xen-tools-4.3.0_06-256.2.x86_64 xen-devel-4.3.0_06-256.2.x86_64 xen-libs-4.3.0_06-256.2.x86_64 xen-4.3.0_06-256.2.x86_64 systemd-195-13.29.1.x86_64 In grub1's /boot/grub/menu.lst, I configure for serial port use as follows ... serial --unit=0 --speed=57600 --word=8 --parity=no --stop=1 terminal --timeout=6 console serial title XEN SERIAL TEST root (hd0,0) kernel /xen.gz vga=gfx-1280x1024x16 com1=57600,8n1,pci console=vga,com1 dom0_mem=1024M,max:1024M dom0_vcpus_pin=true dom0_max_vcpus=4 module /vmlinuz-xen root=/dev/sda1 rootfstype=ext4 noresume vga=0x31a earlyprintk=xen console=tty0 xencons=ttyS0 module /initrd-xen ... Set up the getty listener for ttyS0 cat /etc/systemd/system/serial-getty-custom@.service [Unit] Description=Serial Getty on %I Documentation=man:agetty(8) man:systemd-getty-generator(8) BindsTo=dev-%i.device Conflicts=rescue.service After=dev-%i.device systemd-user-sessions.service plymouth-quit-wait.service Before=getty.target IgnoreOnIsolate=yes [Service] ExecStart=-/sbin/agetty -s %I 57600 vt102 Type=idle Restart=always RestartSec=0 UtmpIdentifier=%I TTYPath=/dev/%I TTYReset=yes TTYVHangup=yes KillMode=process IgnoreSIGPIPE=no KillSignal=SIGHUP ls -al `find /etc/systemd/system | grep serial-getty` lrwxrwxrwx 1 root root 48 Jul 2 10:38 /etc/systemd/system/getty.target.wants/serial-getty-custom@ttyS0.service -> /etc/systemd/system/serial-getty-custom@.service -rw-r--r-- 1 root root 1.1K Jun 13 02:50 /etc/systemd/system/serial-getty-custom@.service checking after boot, ps ax | grep getty 1055 tty4 Ss+ 0:00 /sbin/agetty --noclear tty4 38400 linux 1056 tty5 Ss+ 0:00 /sbin/agetty --noclear tty5 38400 linux 1057 ttyS0 Ss+ 0:00 /sbin/agetty -s ttyS0 57600 vt102 1058 tty3 Ss+ 0:00 /sbin/agetty --noclear tty3 38400 linux 1059 tty2 Ss+ 0:00 /sbin/agetty --noclear tty2 38400 linux 1060 tty6 Ss+ 0:00 /sbin/agetty --noclear tty6 38400 linux 1061 tty1 Ss+ 0:00 /sbin/agetty --noclear tty1 38400 linux and dmesg | egrep -i "tty0|ttyS0|xvc0" [ 0.000000] Command line: root=/dev/sda1 rootfstype=ext4 noresume vga=0x31a earlyprintk=xen console=tty0 xencons=ttyS0 [ 0.000000] Kernel command line: root=/dev/sda1 rootfstype=ext4 noresume vga=0x31a earlyprintk=xen console=tty0 xencons=ttyS0 [ 0.000000] console [tty0] enabled, bootconsole disabled [ 0.914285] Xen virtual console successfully installed as ttyS0 [ 20.851501] systemd[1]: Expecting device dev-ttyS0.device... [ 26.350878] systemd[1]: Found device /dev/ttyS0. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
participants (2)
-
ar16 -
Jan Beulich