http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c1 --- Comment #1 from Chris Ward --- Created attachment 784009 --> http://bugzilla.opensuse.org/attachment.cgi?id=784009&action=edit Screenshot of hang when booting Leap 15.0 in a VM -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c3 --- Comment #3 from Chris Ward --- Created attachment 784140 --> http://bugzilla.opensuse.org/attachment.cgi?id=784140&action=edit /var/log/libvirt/qemu log of attempt to install TW I attempted to install TW from virt-manager. I think this is the log file that you want. The log says libvirt version: 4.6.0, qemu version: 3.0.0 -- You are receiving this mail because: You are on the CC list for the bug.

Created attachment 784140 [details] /var/log/libvirt/qemu log of attempt to install TW

I attempted to install TW from virt-manager. I think this is the log file that you want. Is there any virt-manager error message? I did not see any error in the first log file you provide except some Spice-WARNINGs. From the screenshot, it seems

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c5 --- Comment #5 from Fei Li --- (In reply to Chris Ward from comment #3) the guest's processor has not been booted. Edit the /etc/libvirt/libvirtd.conf as follows and restart the libvirtd.service, and provide the /tmp/libvirtd.log would be much helpful. log_level = 1 log_outputs = "1:file:/tmp/libvirtd.log" A few questions: - do you mean both are failed: no matter install the TW-920 guest and leap15.0 guest? While booting the leap15.0 host on Conroe process is ok? - have you enabled any spectre/meltdown mitigation on the host? BTW, I see you use the former openSUSE-Tumbleweed-DVD-x86_64-Snapshot20180919-Media.iso, would you like to try the latest TW iso and see whether the problem still occur?

The log says libvirt version: 4.6.0, qemu version: 3.0.0

-- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c7 --- Comment #7 from Fei Li --- (In reply to Chris Ward from comment #6)

Created attachment 784256 [details] 'tar -cJf' of libvirt.log log file

...

Is there any virt-manager error message? I did not see any error in the first log file you provide except some Spice-WARNINGs. From the screenshot, it seems the guest's processor has not been booted. No, I couldn't see any error messages either. According to the screenshot, the hang occurred 0.02 seconds after the guest's processor was booted. 'isolinux' ran normally, and the OpenSUSE initial install boot screen was presented normally; I selected 'Installation'. Sorry that I am a little confused by the hang phenomenon. Do you mean the hang occurs during the installation process after you selected the 'Installation', or the 0.02s hang you mentioned above?

...

Edit the /etc/libvirt/libvirtd.conf as follows and restart the libvirtd.service, and provide the /tmp/libvirtd.log would be much helpful. Log file attached. Thanks.

...

A few questions: - do you mean both are failed: no matter install the TW-920 guest and leap15.0 guest? While booting the leap15.0 host on Conroe process is ok? - have you enabled any spectre/meltdown mitigation on the host? TW-920 guest and leap15.0 guest both fail on TW host, and both work on leap15.0 host. The host system was installed as leap 15.0 from DVD, brought up to date with 'zypper up', then moved to TW by changing the software repositories and doing 'zypper dup'. I don't know what that sets up for spectre/meltdown mitigation, but I have not done anything special.

...

openSUSE-Tumbleweed-DVD-x86_64-Snapshot20180919-Media.iso, would you like to try the latest TW iso and see whether the problem still occur? The latest TW snapshot (20180924) has the same problem.

My theory is that there is some change in qemu between the version installed on on leap15.0 and the version installed on TW that is incompatible with the Conroe processor. Is this likely ? Did qemu intend to drop support for Conroe ? Emm, no, I do not think we change any qemu code about Conroe. BTW, could the TW-920 guest and leap15.0 guest be successfully installed just using the following brief qemu command line? /usr/bin/qemu-system-x86_64 -monitor stdio \ -serial none \ -parallel none \ -enable-kvm \ -name "leap15-guest" \ -smp sockets=1,cores=2,threads=2 \ -m 8192 \ -mem-prealloc \ -usb \ -device usb-ehci,id=ehci \ -machine pc,accel=kvm,kernel_irqchip=on,mem-merge=off \ -drive file=/path/leap15.qcow2,format=qcow2,if=none,id=drive-sata0-0-0 \ -device virtio-blk-pci,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 \ -cdrom /path/openSUSE-Tumbleweed-DVD-x86_64-Snapshotxxx-Media.iso \ -boot order=c

-- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c9 --- Comment #9 from Chris Ward --- Created attachment 784323 --> http://bugzilla.opensuse.org/attachment.cgi?id=784323&action=edit Screenshot of hang when installing with qemu-system-x86_64 command I ran your command, modified for '-m 1024' because I don't have 8GB of memory, and got the hang as before. Screenshot is attached, and run log of the qemu-system-x86_64 command is tjcw@linux-ta3i:~> do_install + /usr/bin/qemu-system-x86_64 -monitor stdio -serial none -parallel none -enable-kvm -name tumbleweed-guest -smp sockets=1,cores=2,threads=2 -m 1024 -mem-prealloc -usb -device usb-ehci,id=ehci -machine pc,accel=kvm,kernel_irqchip=on,mem-merge=off -drive file=/home/tjcw/tumbleweed.qcow2,format=qcow2,if=none,id=drive-sata0-0-0 -device virtio-blk-pci,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -cdrom /home/tjcw/Downloads/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20180924-Media.iso -boot order=c QEMU 3.0.0 monitor - type 'help' for more information (qemu) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c11 --- Comment #11 from Chris Ward --- I tried a gdb session; it needed a couple of 'handle' commands to pass SIGUSR1 and SIGTTIN to qemu; but after that it didn't reveal anything, the hang was in the same place and there were no messages. Then I tried downgrading qemu to the Leap 15.0 level . I disabled the TW repo, added the Leap 15.0 repo, and reinstalled qemu through YaST. It automatically downgraded a number of other packages. But on running your command, the VM hung in the same place. So it doesn't look as if qemu is the problem. What should I try next ? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c13 --- Comment #13 from Bruce Rogers --- (In reply to Chris Ward from comment #12)

I replaced "kernel-default" with the corresponding package from Leap 15.0; this also replaces the kernel loadable modules including "kvm.ko". With this done, my TW guest is installing under my TW-with-leap-kernel host with QEMU 3.0.0

So it looks like there is some problem either with the TW kernel or more likely with "kvm.ko" which is breaking things for the Conroe processor.

Do you have a changelog for "kvm.ko" between Leap 15.0 and TW ?

I intended to work on this today, since I've had access to a Conroe for quite some time, but I now find it doesn't work right, so I'm trying to get it working again. Thanks for the analysis on your end. Indeed it does seem to be some change in the kvm kernel module which has broken things on your box. Just so you know, there was a time when kvm was broken on the Conroe, and it is possible it got broken again. I'll look at recent changes done in the TW kernel and see what pops up. Hopefully I can also get some testing going on a Conroe so I can also duplicate it and work on it better on my side. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c15 --- Comment #15 from Chris Ward --- I installed the kernel from Leap 15.1 (4,2,14-lp1515.16-default) and got success. I don't have an archive of TW kernels to try; this machine was running Leap 15.0 until I installed TW 20180920 and got the failure. So I can't say where the problem was introduced more closely than 'somewhere between Leap 15.1 and TW 20180920', I'm afraid. Is there a web site with archives of TW that I could try a few kernels from ? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c17 Bruce Rogers changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |CONFIRMED --- Comment #17 from Bruce Rogers --- With the help of our awesome intern, I have a Conroe working again, and can reproduce. Here's the error were getting: [ 2869.709261] vmwrite error: reg 401e value c0c3d700 (err 12) [ 2869.709269] CPU: 1 PID: 30898 Comm: CPU 0/KVM Not tainted 4.18.8-1-default #1 openSUSE Tumbleweed (unreleased) [ 2869.709270] Hardware name: /DG965MQ, BIOS MQ96510J.86A.1612.2006.1227.1513 12/27/2006 [ 2869.709271] Call Trace: [ 2869.709281] dump_stack+0x85/0xc0 [ 2869.709291] vmx_set_virtual_apic_mode+0x197/0x240 [kvm_intel] [ 2869.709334] kvm_lapic_set_base+0x7b/0x190 [kvm] [ 2869.709353] kvm_set_apic_base+0xac/0xd0 [kvm] [ 2869.709371] kvm_set_msr_common+0x837/0xc00 [kvm] [ 2869.709377] ? vmx_vcpu_run+0x3cf/0xaf0 [kvm_intel] [ 2869.709381] ? vmx_vcpu_run+0x3db/0xaf0 [kvm_intel] [ 2869.709386] ? vmx_vcpu_run+0x3cf/0xaf0 [kvm_intel] [ 2869.709390] ? vmx_vcpu_run+0x3db/0xaf0 [kvm_intel] [ 2869.709394] ? vmx_vcpu_run+0x3cf/0xaf0 [kvm_intel] [ 2869.709399] handle_wrmsr+0x45/0x150 [kvm_intel] [ 2869.709418] vcpu_enter_guest+0x320/0x1490 [kvm] [ 2869.709438] kvm_arch_vcpu_ioctl_run+0xc1/0x510 [kvm] [ 2869.709454] kvm_vcpu_ioctl+0x28a/0x5a0 [kvm] [ 2869.709458] ? update_curr+0x108/0x1f0 [ 2869.709460] ? pick_next_entity+0x83/0x120 [ 2869.709462] ? pick_next_task_fair+0x28d/0x600 [ 2869.709466] do_vfs_ioctl+0x90/0x5e0 [ 2869.709469] ? __fget+0x6e/0xa0 [ 2869.709471] ksys_ioctl+0x60/0x90 [ 2869.709473] __x64_sys_ioctl+0x16/0x20 [ 2869.709476] do_syscall_64+0x60/0x110 [ 2869.709480] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 2869.709483] RIP: 0033:0x7fdcbd634aa7 [ 2869.709484] Code: 00 00 90 48 8b 05 f1 83 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c1 83 2c 00 f7 d8 64 89 01 48 [ 2869.709523] RSP: 002b:00007fdcb226e858 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 2869.709525] RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007fdcbd634aa7 [ 2869.709526] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000016 [ 2869.709527] RBP: 0000000000000000 R08: 000055ff458fc390 R09: 0000000000000001 [ 2869.709529] R10: 0000000000000001 R11: 0000000000000246 R12: 000055ff475fc880 [ 2869.709530] R13: 00007fdcc04fb000 R14: 0000000000000000 R15: 000055ff475fc880 I'll start figuring out what introduced the bug, and what the fix needs to be. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c19 --- Comment #19 from Chris Ward --- (In reply to Bruce Rogers from comment #18)

I'm finding that something in kvm is broken on this machine back to v4.16.0. It seems people don't use this old hardware very much for kvm virtualization!

For me, exploring http://download.opensuse.org/history pays dividends. I find that the 4.18.7-1.5 kernel (in TW 20180917) works, but the 4.18.8-1.3 kernel (in TW 20180919) doesn't work. Hope this ties it down to only a few kernel changes ! -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1109471 http://bugzilla.opensuse.org/show_bug.cgi?id=1109471#c21 --- Comment #21 from Bruce Rogers --- (In reply to Chris Ward from comment #19)

(In reply to Bruce Rogers from comment #18)

...

I'm finding that something in kvm is broken on this machine back to v4.16.0. It seems people don't use this old hardware very much for kvm virtualization!

For me, exploring http://download.opensuse.org/history pays dividends. I find that the 4.18.7-1.5 kernel (in TW 20180917) works, but the 4.18.8-1.3 kernel (in TW 20180919) doesn't work.

Hope this ties it down to only a few kernel changes !

I see a failure with the 4.18.7-1.5 kernel. The install does proceed further, though. I'm going to instrument the running kvm module to see where things go wrong. I had pursued a few other approaches which haven't panned out. As for your commit pointers, I'll look into it. -- You are receiving this mail because: You are on the CC list for the bug.