[Bug 630586] New: When a lot of domU created, some domU stops abnormally without permission.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c0 Summary: When a lot of domU created, some domU stops abnormally without permission. Classification: openSUSE Product: openSUSE 11.3 Version: Final Platform: x86-64 OS/Version: openSUSE 11.3 Status: NEW Severity: Critical Priority: P5 - None Component: Xen AssignedTo: jdouglas@novell.com ReportedBy: k3jun68@gmail.com QAContact: qa@suse.de Found By: --- Blocker: --- User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100723 SUSE/3.6.8-0.1.1 Firefox/3.6.8 21 domU created with xen4.0 + kernel2.6.34. domU is hvm. using memtest86+. One hour passed, and check domU status. ----------------------------------------------------------------------- NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD_WR VBD_RSECT VBD_WSECT SSID Domain-0 -----r 2758 18.1 535552 26.1 no limit n/a 4 0 0 0 0 0 0 0 0 0 0 memtest101 ------ 3227 23.7 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest102 ------ 2187 0.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest103 ------ 3225 23.5 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest104 ------ 3221 22.7 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest105 ------ 27 0.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest106 ------ 3213 23.6 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest107 ------ 638 0.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest108 ------ 1925 0.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest109 ------ 3204 23.8 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest110 ------ 3198 22.6 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest111 ------ 3193 23.3 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest112 ------ 3186 23.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest113 ------ 2145 0.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest114 ------ 3180 23.9 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest115 ------ 3174 23.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest116 -----r 3171 23.7 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest117 ------ 3163 24.0 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest118 -----r 3156 22.4 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest119 ------ 3148 23.1 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest120 ------ 3142 23.5 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 memtest121 -----r 3133 23.3 69600 3.4 69632 3.4 1 1 0 0 1 0 0 0 0 0 0 ----------------------------------------------------------------------- 5 domU stops abnormally without permission. CPU(%) is 0.0. Cannot shutdown Stop-domU. Must destroy. ----------------------------------------------------------------------- linux-pfef:/etc/xen/vm # xm shutdown memtest102 linux-pfef:/etc/xen/vm # xm list Name ID Mem VCPUs State Time(s) Domain-0 0 523 4 r----- 3468.0 memtest102 3 64 1 ------ 2187.8 memtest105 6 64 1 ------ 28.0 memtest107 8 64 1 ------ 638.8 memtest108 9 64 1 ------ 1925.4 memtest113 14 64 1 ------ 2145.2 ----------------------------------------------------------------------- Reproducible: Always Steps to Reproduce: 1.As many as possible to create domU(hvm). 2.1-2 hours wait. 3.determine the status domU. Actual Results: . Expected Results: . Status Information ----------------------------------------------------------------------- linux-pfef:/etc/xen/vm # uname -a Linux linux-pfef 2.6.34.2-2-xen #1 SMP 2010-08-05 00:01:38 +0200 x86_64 x86_64 x86_64 GNU/Linux linux-pfef:/etc/xen/vm # xm info host : linux-pfef release : 2.6.34.2-2-xen version : #1 SMP 2010-08-05 00:01:38 +0200 machine : x86_64 nr_cpus : 4 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 1 cpu_mhz : 2400 hw_caps : bfebfbff:20100800:00000000:00000940:0000e3bd:00000000:00000001:00000000 virt_caps : hvm total_memory : 2003 free_memory : 1144 free_cpus : 0 max_free_memory : 1155 max_para_memory : 1151 max_hvm_memory : 1140 node_to_cpu : node0:0-3 node_to_memory : node0:1144 node_to_dma32_mem : node0:1144 max_node_id : 0 xen_major : 4 xen_minor : 0 xen_extra : .0_21091_05-78. xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : 21091 xen_commandline : cc_compiler : gcc version 4.5.0 20100604 [gcc-4_5-branch revision 160292] (SU cc_compile_by : abuild cc_compile_domain : cc_compile_date : Thu Aug 5 12:20:04 UTC 2010 xend_config_format : 4 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz linux-pfef:/etc/xen/vm # xm list -l (domain (domid 0) (cpu_weight 256) (cpu_cap 0) (pool_name Pool-0) (bootloader ) (vcpus 4) (cpus ((0 1 2 3) (0 1 2 3) (0 1 2 3) (0 1 2 3))) (on_poweroff destroy) (on_crash restart) (uuid 00000000-0000-0000-0000-000000000000) (bootloader_args ) (name Domain-0) (on_reboot restart) (maxmem 16777215) (memory 523) (shadow_memory 0) (features ) (on_xend_start ignore) (on_xend_stop ignore) (cpu_time 3172.40622097) (online_vcpus 4) (image (linux (kernel ) (superpages 0) (nomigrate 0) (tsc_mode 0))) (status 2) (state r-----) ) (domain (domid 3) (cpu_weight 256) (cpu_cap 0) (pool_name Pool-0) (bootloader ) (vcpus 1) (cpus (())) (on_poweroff destroy) (description None) (on_crash destroy) (uuid 1995283c-1031-fada-8b44-a4f76f9e3102) (bootloader_args ) (name memtest102) (on_reboot destroy) (maxmem 64) (memory 64) (shadow_memory 2) (features ) (on_xend_start ignore) (on_xend_stop ignore) (start_time 1281530683.77) (cpu_time 2187.80219251) (online_vcpus 1) (image (hvm (kernel ) (superpages 0) (videoram 4) (hpet 0) (stdvga 1) (loader /usr/lib/xen/boot/hvmloader) (xen_platform_pci 1) (rtc_timeoffset 0) (pci ()) (hap 1) (localtime 0) (timer_mode 1) (pci_msitranslate 1) (oos 1) (apic 0) (display :0) (vpt_align 1) (extid 0) (serial pty) (vncunused 0) (boot d) (pae 1) (viridian 0) (acpi 1) (vnc 0) (nographic 0) (nomigrate 0) (usb 0) (tsc_mode 0) (guest_os_type default) (device_model /usr/lib/xen/bin/qemu-dm) (keymap ja) (pci_power_mgmt 0) (xauthority /root/.Xauthority) (isa 0) (notes (SUSPEND_CANCEL 1)) ) ) (status 2) (state ------) (store_mfn 1044476) (device (vif (bridge br1) (uuid de77c557-4a25-3505-7d86-2e0a580b3bfb) (script /etc/xen/scripts/vif-bridge) (mac 00:16:3e:5e:81:2) (model e1000) (backend 0) ) ) (device (console (protocol vt100) (location 3) (uuid 47589fb8-399e-277c-3af9-456a6c951f94) ) ) (device (vbd (uuid 3a8c4bb9-3792-46a9-91fc-6f8b83c7f7fd) (bootable 1) (dev hda:cdrom) (uname 'file:/home/ISO/memtest86+-4.10.iso') (mode r) (backend 0) (VDI ) ) ) ) ----------------------------------------------------------------------- openSUSE 11.3 milstone 4(kernel 2.6.33) doesn't reproduce it. But system hungup occasionally. Anything cannot be operated. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c kazuma juno <k3jun68@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |Final Summary|When a lot of domU created, |When a lot of domU created, |some domU stops abnormally |some domU stops abnormally |without permission. |without permission. (Xen4.0 | |+ kernel2.6.34) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c Jason Douglas <jdouglas@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jdouglas@novell.com AssignedTo|jdouglas@novell.com |jbeulich@novell.com -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c1 Jan Beulich <jbeulich@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO Found By|--- |Community User InfoProvider| |k3jun68@gmail.com Target Milestone|Final |--- --- Comment #1 from Jan Beulich <jbeulich@novell.com> 2010-08-12 09:18:35 UTC --- Do the DomU-s report anything on their consoles? In any case, we'll need the hypervisor (and possibly Dom0) logs associated with this. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c2 --- Comment #2 from kazuma juno <k3jun68@gmail.com> 2010-08-12 11:00:03 UTC --- Created an attachment (id=382520) --> (http://bugzilla.novell.com/attachment.cgi?id=382520) logfiles -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c3 --- Comment #3 from Jan Beulich <jbeulich@novell.com> 2010-08-12 11:35:40 UTC --- Okay, nothing interesting in Dom0. We'll definitely need the hypervisor log (and ideally you would also provide the log after sending 'q' or running "xm debug-key q" once in this state). -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c4 --- Comment #4 from kazuma juno <k3jun68@gmail.com> 2010-08-12 15:19:26 UTC --- Created an attachment (id=382628) --> (http://bugzilla.novell.com/attachment.cgi?id=382628) xm debug-key q, xm demsg, logfile. Stop-domU is memtest110(ID:10). I do not know how to "xm debug-key q". I was wrong, sorry. Kernel and Xen packages is opensuse-repository. http://download.opensuse.org/repositories/Virtualization/openSUSE_11.3/ http://download.opensuse.org/repositories/Kernel:/openSUSE-11.3/openSUSE_11.... -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=630586 http://bugzilla.novell.com/show_bug.cgi?id=630586#c5 --- Comment #5 from Jan Beulich <jbeulich@novell.com> 2010-08-13 12:57:32 UTC --- Domain 10 (in the case above) is waiting for some event in Xen (it is not dead as I first assumed). Determining which one may be a little difficult, but it would be good if we had more useful logs: - irrelevant domains cleaned up as much as possible before obtaining any of the debugging information - "xm dmesg" issued right *before* running "xm debug-key q" (along with the obviously necessary one after) - "xm debug-key r" and "xm debug-key e" might also be useful (again, "xm dmesg" every time in between, so the log buffer doesn't overflow, unless you would increase the buffer size) Even better would be to attach a serial console of course. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=630586 https://bugzilla.novell.com/show_bug.cgi?id=630586#c6 Jan Beulich <jbeulich@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |CLOSED InfoProvider|k3jun68@gmail.com | Resolution| |NORESPONSE --- Comment #6 from Jan Beulich <jbeulich@novell.com> 2010-11-04 15:34:08 UTC --- No response in over two months. Feel free to re-open if you can provide the requested information. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com