[opensuse-kernel] 3.10rc2 crash
Hi, I see this in openqa tests and rc2 seems to be fresh enough not to go through bugzilla queue with it. It doesn't seem to be fatal even though it says panic, but I thought you may get something out of it. Greetings, Stephan [ 1701.024795] swapper/0[0]: segfault at ffffffff81031d62 ip ffffffff81031d62 sp ffffffff81c01ed0 error 15 [ 1701.033143] BUG: unable to handle kernel NULL pointer dereference at 0000000000000340 [ 1701.034008] IP: [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] PGD 3c1d1067 PUD 3cfcf067 PMD 0 [ 1701.034008] Oops: 0000 [#1] PREEMPT SMP [ 1701.034008] Modules linked in: btrfs raid6_pq zlib_deflate xor ufs qnx4 hfsplus hfs minix vfat msdos fat jfs xfs libcrc32c reiserfs dm_mod nls_utf8 fuse st snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd soundcore snd_page_alloc lp ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_raw ipt_REJECT xt_tcpudp xt_pkttype xt_LOG xt_limit iptable_raw xt_CT iptable_filter ip6table_mangle nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 ip_tables xt_conntrack nf_conntrack ip6table_filter ip6_tables x_tables af_packet bnep bluetooth rfkill sr_mod cdrom ata_generic ppdev joydev microcode ata_piix pcspkr virtio_net floppy parport_pc parport button i2c_piix4 sg autofs4 virtio_blk virtio_pci virtio_ring virtio cirrus ttm drm_kms_helper drm sysimgblt sysfillrect syscopyarea processor thermal_sys scsi_dh_hp_sw scsi_dh_emc scsi_dh_rdac scsi_dh_alua scsi_dh [last unloaded: snd_page_alloc] [ 1701.034008] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 3.10.0-rc2-1.gd28ac96-desktop #1 [ 1701.034008] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1701.034008] task: ffffffff81c12440 ti: ffffffff81c00000 task.ti: ffffffff81c00000 [ 1701.034008] RIP: 0010:[<ffffffff811d757f>] [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] RSP: 0018:ffffffff81c01c58 EFLAGS: 00010292 [ 1701.034008] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 1701.034008] RDX: ffffffff81c01f58 RSI: ffff88003a0c79f0 RDI: 000000000000000b [ 1701.034008] RBP: ffffffff81c01e70 R08: 000000000000c038 R09: 0000000000030001 [ 1701.034008] R10: 0000000000000dba R11: 0000000000000001 R12: ffffffff81c01e70 [ 1701.034008] R13: ffffffff81c12440 R14: 000000000000000a R15: ffffffff81c141a0 [ 1701.034008] FS: 0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000 [ 1701.034008] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1701.034008] CR2: 0000000000000340 CR3: 000000003ce57000 CR4: 00000000000006f0 [ 1701.034008] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1701.034008] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1701.034008] Stack: [ 1701.034008] 0000000000000046 0000000000000046 ffffffff81048c25 0000000000000004 [ 1701.034008] 00000000ffffffff 0000000000000000 ffffffff81048a0f 0000000000000031 [ 1701.034008] 0000000000000246 ffffffff81eacbe0 0000000000000031 0000000000000080 [ 1701.034008] Call Trace: [ 1701.034008] [<ffffffff8105bcc9>] get_signal_to_deliver+0x2a9/0x630 [ 1701.034008] [<ffffffff810023d3>] do_signal+0x63/0x8c0 [ 1701.034008] [<ffffffff81002cb8>] do_notify_resume+0x88/0xb0 [ 1701.034008] [<ffffffff815d78fc>] retint_signal+0x48/0x8c [ 1701.034008] Code: a8 00 00 00 00 00 00 00 48 c7 84 24 98 00 00 00 00 00 00 00 48 8b 80 d0 02 00 00 48 89 94 24 90 00 00 00 48 89 84 24 a0 00 00 00 <48> 8b 83 40 03 00 00 48 89 84 24 a8 00 00 00 e8 ed 46 ef ff 48 [ 1701.034008] RIP [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] RSP <ffffffff81c01c58> [ 1701.034008] CR2: 0000000000000340 [ 1701.115310] ---[ end trace e20cdda21ceccb17 ]--- [ 1701.115707] Kernel panic - not syncing: Attempted to kill the idle task! [ 1701.116261] drm_kms_helper: panic occurred, switching back to text console -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Mon 03-06-13 12:12:56, Stephan Kulow wrote:
Hi,
I see this in openqa tests and rc2 seems to be fresh enough not to go through bugzilla queue with it. It doesn't seem to be fatal even though it says panic, but I thought you may get something out of it.
Ohh boy, crashing while taking a crashdump is a bit ironic, isn't it. Where can I find vmlinux for that kernel? Could you make it awailable or point me at the rpm package, please?
Greetings, Stephan
[ 1701.024795] swapper/0[0]: segfault at ffffffff81031d62 ip ffffffff81031d62 sp ffffffff81c01ed0 error 15 [ 1701.033143] BUG: unable to handle kernel NULL pointer dereference at 0000000000000340 [ 1701.034008] IP: [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] PGD 3c1d1067 PUD 3cfcf067 PMD 0 [ 1701.034008] Oops: 0000 [#1] PREEMPT SMP [ 1701.034008] Modules linked in: btrfs raid6_pq zlib_deflate xor ufs qnx4 hfsplus hfs minix vfat msdos fat jfs xfs libcrc32c reiserfs dm_mod nls_utf8 fuse st snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd soundcore snd_page_alloc lp ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_raw ipt_REJECT xt_tcpudp xt_pkttype xt_LOG xt_limit iptable_raw xt_CT iptable_filter ip6table_mangle nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 ip_tables xt_conntrack nf_conntrack ip6table_filter ip6_tables x_tables af_packet bnep bluetooth rfkill sr_mod cdrom ata_generic ppdev joydev microcode ata_piix pcspkr virtio_net floppy parport_pc parport button i2c_piix4 sg autofs4 virtio_blk virtio_pci virtio_ring virtio cirrus ttm drm_kms_helper drm sysimgblt sysfillrect syscopyarea processor thermal_sys scsi_dh_hp_sw scsi_dh_emc scsi_dh_rdac scsi_dh_alua scsi_dh [last unloaded: snd_page_alloc] [ 1701.034008] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 3.10.0-rc2-1.gd28ac96-desktop #1 [ 1701.034008] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1701.034008] task: ffffffff81c12440 ti: ffffffff81c00000 task.ti: ffffffff81c00000 [ 1701.034008] RIP: 0010:[<ffffffff811d757f>] [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] RSP: 0018:ffffffff81c01c58 EFLAGS: 00010292 [ 1701.034008] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 1701.034008] RDX: ffffffff81c01f58 RSI: ffff88003a0c79f0 RDI: 000000000000000b [ 1701.034008] RBP: ffffffff81c01e70 R08: 000000000000c038 R09: 0000000000030001 [ 1701.034008] R10: 0000000000000dba R11: 0000000000000001 R12: ffffffff81c01e70 [ 1701.034008] R13: ffffffff81c12440 R14: 000000000000000a R15: ffffffff81c141a0 [ 1701.034008] FS: 0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000 [ 1701.034008] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1701.034008] CR2: 0000000000000340 CR3: 000000003ce57000 CR4: 00000000000006f0 [ 1701.034008] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1701.034008] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1701.034008] Stack: [ 1701.034008] 0000000000000046 0000000000000046 ffffffff81048c25 0000000000000004 [ 1701.034008] 00000000ffffffff 0000000000000000 ffffffff81048a0f 0000000000000031 [ 1701.034008] 0000000000000246 ffffffff81eacbe0 0000000000000031 0000000000000080 [ 1701.034008] Call Trace: [ 1701.034008] [<ffffffff8105bcc9>] get_signal_to_deliver+0x2a9/0x630 [ 1701.034008] [<ffffffff810023d3>] do_signal+0x63/0x8c0 [ 1701.034008] [<ffffffff81002cb8>] do_notify_resume+0x88/0xb0 [ 1701.034008] [<ffffffff815d78fc>] retint_signal+0x48/0x8c [ 1701.034008] Code: a8 00 00 00 00 00 00 00 48 c7 84 24 98 00 00 00 00 00 00 00 48 8b 80 d0 02 00 00 48 89 94 24 90 00 00 00 48 89 84 24 a0 00 00 00 <48> 8b 83 40 03 00 00 48 89 84 24 a8 00 00 00 e8 ed 46 ef ff 48 [ 1701.034008] RIP [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] RSP <ffffffff81c01c58> [ 1701.034008] CR2: 0000000000000340 [ 1701.115310] ---[ end trace e20cdda21ceccb17 ]--- [ 1701.115707] Kernel panic - not syncing: Attempted to kill the idle task! [ 1701.116261] drm_kms_helper: panic occurred, switching back to text console -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
-- Michal Hocko SUSE Labs -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 03.06.2013 21:05, Michal Hocko wrote:
On Mon 03-06-13 12:12:56, Stephan Kulow wrote:
Hi,
I see this in openqa tests and rc2 seems to be fresh enough not to go through bugzilla queue with it. It doesn't seem to be fatal even though it says panic, but I thought you may get something out of it.
Ohh boy, crashing while taking a crashdump is a bit ironic, isn't it.
Where can I find vmlinux for that kernel? Could you make it awailable or point me at the rpm package, please?
Hi, This is kernel-desktop-3.10.rc2-1.1.gd28ac96.x86_64, which is the rpm you can find at http://download.opensuse.org/factory/repo/oss/suse/x86_64/ Greetings, Stephan -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Tue 04-06-13 09:53:47, Stephan Kulow wrote:
On 03.06.2013 21:05, Michal Hocko wrote:
On Mon 03-06-13 12:12:56, Stephan Kulow wrote:
Hi,
I see this in openqa tests and rc2 seems to be fresh enough not to go through bugzilla queue with it. It doesn't seem to be fatal even though it says panic, but I thought you may get something out of it.
Ohh boy, crashing while taking a crashdump is a bit ironic, isn't it.
Where can I find vmlinux for that kernel? Could you make it awailable or point me at the rpm package, please?
Hi,
This is kernel-desktop-3.10.rc2-1.1.gd28ac96.x86_64, which is the rpm you can find at http://download.opensuse.org/factory/repo/oss/suse/x86_64/
OK, that looks a bit saner.
[ 1701.034008] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 3.10.0-rc2-1.gd28ac96-desktop #1
Btw. what was the previous warning that tainted the kernel?
[ 1701.034008] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1701.034008] task: ffffffff81c12440 ti: ffffffff81c00000 task.ti: ffffffff81c00000 [ 1701.034008] RIP: 0010:[<ffffffff811d757f>] [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] RSP: 0018:ffffffff81c01c58 EFLAGS: 00010292 [ 1701.034008] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 1701.034008] RDX: ffffffff81c01f58 RSI: ffff88003a0c79f0 RDI: 000000000000000b [ 1701.034008] RBP: ffffffff81c01e70 R08: 000000000000c038 R09: 0000000000030001 [ 1701.034008] R10: 0000000000000dba R11: 0000000000000001 R12: ffffffff81c01e70 [ 1701.034008] R13: ffffffff81c12440 R14: 000000000000000a R15: ffffffff81c141a0 [ 1701.034008] FS: 0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000 [ 1701.034008] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1701.034008] CR2: 0000000000000340 CR3: 000000003ce57000 CR4: 00000000000006f0 [ 1701.034008] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1701.034008] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1701.034008] Stack: [ 1701.034008] 0000000000000046 0000000000000046 ffffffff81048c25 0000000000000004 [ 1701.034008] 00000000ffffffff 0000000000000000 ffffffff81048a0f 0000000000000031 [ 1701.034008] 0000000000000246 ffffffff81eacbe0 0000000000000031 0000000000000080 [ 1701.034008] Call Trace: [ 1701.034008] [<ffffffff8105bcc9>] get_signal_to_deliver+0x2a9/0x630 [ 1701.034008] [<ffffffff810023d3>] do_signal+0x63/0x8c0 [ 1701.034008] [<ffffffff81002cb8>] do_notify_resume+0x88/0xb0 [ 1701.034008] [<ffffffff815d78fc>] retint_signal+0x48/0x8c [ 1701.034008] Code: a8 00 00 00 00 00 00 00 48 c7 84 24 98 00 00 00 00 00 00 00 48 8b 80 d0 02 00 00 48 89 94 24 90 00 00 00 48 89 84 24 a0 00 00 00 <48> 8b 83 40 03 00 00 48 89 84 24 a8 00 00 00 e8 ed 46 ef ff 48 [ 1701.034008] RIP [<ffffffff811d757f>] do_coredump+0x7f/0xe90 [ 1701.034008] RSP <ffffffff81c01c58> [ 1701.034008] CR2: 0000000000000340 [ 1701.115310] ---[ end trace e20cdda21ceccb17 ]--- [ 1701.115707] Kernel panic - not syncing: Attempted to kill the idle task! [ 1701.116261] drm_kms_helper: panic occurred, switching back to text console
ffffffff811d7500 <do_coredump>: ffffffff811d7500: 41 57 push %r15 ffffffff811d7502: 65 48 8b 04 25 40 b9 mov %gs:0xb940,%rax ffffffff811d7509: 00 00 ffffffff811d750b: 41 56 push %r14 ffffffff811d750d: 41 55 push %r13 ffffffff811d750f: 41 54 push %r12 ffffffff811d7511: 49 89 fc mov %rdi,%r12 ffffffff811d7514: 55 push %rbp ffffffff811d7515: 53 push %rbx ffffffff811d7516: 48 81 ec 18 01 00 00 sub $0x118,%rsp ffffffff811d751d: 48 8b 98 e8 02 00 00 mov 0x2e8(%rax),%rbx # looks like current current->mm [...] ffffffff811d757f: 48 8b 83 40 03 00 00 mov 0x340(%rbx),%rax <<< BANG RBX is NULL as we can see from the dumped registers. This looks like we are trying to dereference mm->flags here. It is even more interesting to see that the killed task looks like idle thread (pid 0) which doesn't have any mm associated with it. How could we end up receiving any signal which results in a core dump? SEGV can be ruled out as there is no address space and the thread doesn't dereference any memory. Is this reproducible? -- Michal Hocko SUSE Labs -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/03/2013 12:12 PM, Stephan Kulow wrote:
Hi,
I see this in openqa tests and rc2 seems to be fresh enough not to go through bugzilla queue with it. It doesn't seem to be fatal even though it says panic, but I thought you may get something out of it.
Greetings, Stephan
[ 1701.024795] swapper/0[0]: segfault at ffffffff81031d62 ip ffffffff81031d62 sp ffffffff81c01ed0 error 15 [ 1701.033143] BUG: unable to handle kernel NULL pointer dereference at 0000000000000340 [ 1701.034008] IP: [<ffffffff811d757f>] do_coredump+0x7f/0xe90
Do we try to dump a core of the idle ("swapper") task? Well... Is there anything relevant before the swapper segfault line? - -- js suse labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRrddkAAoJEL0lsQQGtHBJ8BQP/RrgXWvoUQhpWyehl10fsCzd KCUXAMYtrHpEYLkguj6UqIET69sww0NvVFiVcc4ycE3mzmk1vaJp5JWYIw7/VD0p Qeuq1/oDubMQ2mI3XUO8DQLov89u731g+LY+E8rE+4aPsJDaXiosqCqZNUCZiMfY ocAaBa5XX1LXIVf/ClVMBpx6paVRwmLlrWAYEdoxCDzUqaseOra1KkADuekI7xWE fFsuOQoqs7gjyr4P8U7tycH4MK/SDC19kdOF04yX+7EfMCyAYzyedKopSmGBEbk6 eBAzlAXa23V0RzT+/LfoZz20Hu0hBfDS5Tx5c5DkKnC+AyMon219+4j2sFfBXmjN sZXZAZb1FvUV0RQJTmpBoKdEyzqJxj49qPSp81Ql7jJtjytrD4M0MUG+bHTsaUSN 0JXJ9ki5orKXq4YTn+e2TYlDVn+TjWBirvc5rJI8LJVtMAlAK9F9cF4HWZ7hp8TS r8MX7+JoEuT8JOw35xo2MA+Kf3Qzrldb55ZhTwJ1UNqlw+PjdQIbdMu3djC8x6Rb 1nTArrbueDkmBo7Pv9ShI5G4cH2YlvdSL/GxIXI6UjafKUqyrf6F0UCpH8zYxtQA wzdaoMxhBGSLzBXbsHSKTWidDrBTQND6NzmALY3lEAryES4DJBT8qh37cF4Miok/ kewmBT+ZsVZAlxhMLNnp =hBsR -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04.06.2013 14:02, Jiri Slaby wrote:
On 06/03/2013 12:12 PM, Stephan Kulow wrote:
Hi,
I see this in openqa tests and rc2 seems to be fresh enough not to go through bugzilla queue with it. It doesn't seem to be fatal even though it says panic, but I thought you may get something out of it.
Greetings, Stephan
[ 1701.024795] swapper/0[0]: segfault at ffffffff81031d62 ip ffffffff81031d62 sp ffffffff81c01ed0 error 15 [ 1701.033143] BUG: unable to handle kernel NULL pointer dereference at 0000000000000340 [ 1701.034008] IP: [<ffffffff811d757f>] do_coredump+0x7f/0xe90
Do we try to dump a core of the idle ("swapper") task? Well...
Is there anything relevant before the swapper segfault line?
Unfortunately openqa only adds the end of dmesg to the log file, so I can't say Greetings, Stephan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iD8DBQFRrdoYwFSBhlBjoJYRAoKwAKC6975HfEyu6oau8sOqzJPHHIyPnwCfTcqu FwTaoGrlgIhdPb495JjHE+8= =FiVt -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/04/2013 02:14 PM, Stephan Kulow wrote:
On 04.06.2013 14:02, Jiri Slaby wrote:
On 06/03/2013 12:12 PM, Stephan Kulow wrote:
Hi,
I see this in openqa tests and rc2 seems to be fresh enough not to go through bugzilla queue with it. It doesn't seem to be fatal even though it says panic, but I thought you may get something out of it.
Greetings, Stephan
[ 1701.024795] swapper/0[0]: segfault at ffffffff81031d62 ip ffffffff81031d62 sp ffffffff81c01ed0 error 15 [ 1701.033143] BUG: unable to handle kernel NULL pointer dereference at 0000000000000340 [ 1701.034008] IP: [<ffffffff811d757f>] do_coredump+0x7f/0xe90
Do we try to dump a core of the idle ("swapper") task? Well...
Is there anything relevant before the swapper segfault line?
Unfortunately openqa only adds the end of dmesg to the log file, so I can't say
Hmm. It's weird -- ffffffff81031d62 is a retq instruction. You have perhaps a corrupted stack :/. - -- js suse labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRrdtKAAoJEL0lsQQGtHBJT7UP/iB8urL5Q7+GRqWmBw8aQyW3 6EbhEs/Av7vO7f/bGbmbiB9M3uTuGTA/5KGTVLCr537qcbL9mkKit6kbaT7nsJwN RxdqQGX9FPmHBZ0YTzwUEcXy3J9JS0Q0uYMHcD/kC7s+EqayrOkljrJZJrfl6+0h s5JlJhiDw6P3N1ea1D2JD6sY2VYZgkItkp9wetv/X/BIzCJPvhm+jG/NaYtx7K4L 5teWrIbMkr322mIYHu4rm80ZgX3vUDELW7t5TlHvGetpG6Qt3fLWl6A/S+ry1Y8A IA5dMFZFe+5I8GEbhqD7m/bE6zBj1wc4oOTCZrbVM8JIfAngXKfFDscLj/4ic1KM PtTp6PsbUce+hnPav+5DQv6BwONQFCsyThsRhTgF3YKhw5E8yr7e+Io4ukfyRkuA C76nEPBaD3fhH0r8Vh2BOk6bKi0gORI4SzdHbUtgfLYOXZBN/T9CALOu1stp7P9F HFJtQvK88ZRH2E9mn6Z+j8wY+Ad15AA9OQcBesGoTIFXgBgoPt6ZMWVlJdKsZ7r+ UOUJE2K7ULpx7+yzV38xe2bOhNXVvCz2/nv6/OMfe9i8ajuf/4vnLixEs5NOeyp0 in7KoUHc6anTRbrtBkjcaL91ynXRkhyOyclC/1WGzGF3LI64P9m+IN56VVqguAmX Yps2qsbAZnnPNHlrGMhb =URMz -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
participants (3)
-
Jiri Slaby -
Michal Hocko -
Stephan Kulow