Re: [opensuse-virtual] Xen live migration and xen-kmp issues
Danilo Godec 07/14/11 4:12 PM >>> >Here's the data for a 11.4 guest - kernel 2.6.37.6-0.5-xen #1 SMP, the domain is started with 'vcpus=2' and 'vcpu_avail=1': >... >Call Trace: > [] hypercall_page+0x3aa <-- > [] xen_idle+0x3d > [] cpu_idle+0x55 > [] start_kernel+0x3a5
And that guest is consuming 100% CPU? According to the stack trace it's idle... Anyway, considering a recently analyzed problem, is this (mis)behavior dependent on certain characteristics of the source/destination hosts, or is it happening no matter between which you migrate? Or are all your hosts identical (as is generally recommended)? Jan
On 14.7.2011 16:21, Jan Beulich wrote:
Danilo Godec 07/14/11 4:12 PM >>> Here's the data for a 11.4 guest - kernel 2.6.37.6-0.5-xen #1 SMP, the domain is started with 'vcpus=2' and 'vcpu_avail=1': ... Call Trace: [<ffffffff800033aa>] hypercall_page+0x3aa <-- [<ffffffff8000e4ed>] xen_idle+0x3d [<ffffffff80005255>] cpu_idle+0x55 [<ffffffff8083eb34>] start_kernel+0x3a5
And that guest is consuming 100% CPU? According to the stack trace it's idle...
I'm puzzled - before it would consume 100% CPU. Now it doesn't - even if I try it with the older DomU (11.2). I'll try to replicate the 100% CPU consumption tomorrow and get a trace of that.
Anyway, considering a recently analyzed problem, is this (mis)behavior dependent on certain characteristics of the source/destination hosts, or is it happening no matter between which you migrate? Or are all your hosts identical (as is generally recommended)?
When it comes to software and configuration (apart from the obvious stuff), all my Xen hosts are identical - OpenSuSE 11.4 x86_64. I was running older OpenSuSE Dom0's before (11.2) and never got live migration to work properly - but I think there were different errors and issues. I also setup two additional hosts with updated Xen (11.4 with Xen 4.1.1 from Virtualization repository) but it was the same. Regards, Danilo -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-virtual+help@opensuse.org
On 07/14/2011 05:57 PM, Danilo Godec wrote:
On 14.7.2011 16:21, Jan Beulich wrote:
Danilo Godec 07/14/11 4:12 PM>>> Here's the data for a 11.4 guest - kernel 2.6.37.6-0.5-xen #1 SMP, the domain is started with 'vcpus=2' and 'vcpu_avail=1': ... Call Trace: [<ffffffff800033aa>] hypercall_page+0x3aa<-- [<ffffffff8000e4ed>] xen_idle+0x3d [<ffffffff80005255>] cpu_idle+0x55 [<ffffffff8083eb34>] start_kernel+0x3a5 And that guest is consuming 100% CPU? According to the stack trace it's idle... I'm puzzled - before it would consume 100% CPU. Now it doesn't - even if I try it with the older DomU (11.2).
I'll try to replicate the 100% CPU consumption tomorrow and get a trace of that.
OK, here's the deal: If the migrated DomU only has one active VCPU (with 4 available), it doesn't consume 100% CPU after migration (it still doesn't work)... If the migrated DomU has 2 or more active VCPU's (but less them maximum available), it consumes 100% CPU - xenctx shows this: # /usr/lib64/xen/bin/xenctx -s /root/System.map-migrator114 207 0 rip: ffffffff800033aa hypercall_page+0x3aa flags: 00001246 i z p rsp: ffffffff807cff50 rax: 0000000000000000 rcx: ffffffff800033aa rdx: 0000000000000000 rbx: ffffffff8086e0c0 rsi: 0000000000000000 rdi: 0000000000000001 rbp: 0000000000000000 r8: 0000000000000000 r9: 0000000000000000 r10: 000000000000085b r11: 0000000000000246 r12: ffffffff807ce000 r13: ffffffff8082bcc0 r14: ffffffff807ce000 r15: 0000000000000000 cs: e033 ss: e02b ds: 0000 es: 0000 fs: 0000 @ 00007fc14dfc1700 gs: 0000 @ ffff88001f78d000/0000000000000000 Code (instr addr ffffffff800033aa) cc cc cc cc cc cc cc cc cc cc cc 51 41 53 b8 1d 00 00 00 0f 05 <41> 5b 59 c3 cc cc cc cc cc cc cc Stack: 0720072007200720 00000000ffffffff ffffffff8000e49d ffffffff807ce000 ffffffff8086e0c0 0000000000000000 ffffffff80005255 ffffffff8086e0c0 ffffffff80870c80 ffff88001f8e0d40 ffffffffffffffff 0000000000000000 ffffffff8083eb34 0000000000000000 ffffffff80870c80 0000000000000000 Call Trace: [<ffffffff800033aa>] hypercall_page+0x3aa <-- [<ffffffff8000e49d>] xen_idle+0x3d [<ffffffff80005255>] cpu_idle+0x55 [<ffffffff8083eb34>] start_kernel+0x3a5 # /usr/lib64/xen/bin/xenctx -s /root/System.map-migrator114 207 1 rip: ffffffff800939e0 stop_machine_cpu_stop+0x60 flags: 00001297 i s nz a p c rsp: ffff88001ed81de0 rax: 0000000000000000 rcx: 0000000000000000 rdx: 0000000000000003 rbx: ffff88001d515e70 rsi: 0000000000000002 rdi: ffff88001d515e70 rbp: 0000000000000003 r8: ffff88001ed80000 r9: 0000000000000001 r10: 0000000000000001 r11: 00000000ffffffff r12: ffff88001d515d00 r13: ffffffff80093980 r14: ffff88001d515e70 r15: ffff88001ed7e440 cs: e033 ss: e02b ds: 0000 es: 0000 fs: 0000 @ 00007fc84f2be700 gs: 0000 @ ffff88001f79f000/0000000000000000 Code (instr addr ffffffff800939e0) 4b 20 8b 73 10 83 c1 01 89 73 24 89 4b 20 83 fa 04 74 32 f3 90 <8b> 6b 20 39 d5 74 f2 83 fd 02 75 Stack: 0000000000000000 040204028003b740 ffff88001f7a7700 ffff88001ed7e440 ffff88001d515df0 ffffffff80093bd5 ffff88001f7a7710 ffff88001ed81fd8 0000000000009dc0 0000000000009dc0 ffff88001ed81fd8 0000000000009dc0 0000000000009dc0 ffff88001ed7e440 ffff88001ecde040 ffff88001ed7e440 Call Trace: [<ffffffff800939e0>] stop_machine_cpu_stop+0x60 <-- [<ffffffff80093bd5>] cpu_stopper_thread+0xf5 [<ffffffff8002a1b9>] __wake_up_common+0x59 [<ffffffff80093ae0>] cpu_stopper_thread [<ffffffff800640c6>] kthread+0x96 [<ffffffff80007e24>] kernel_thread_helper+0x4 [<ffffffff80064030>] kthread [<ffffffff80007e20>] kernel_thread_helper Hope this helps. Danilo -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-virtual+help@opensuse.org
participants (2)
-
Danilo Godec -
Jan Beulich