[Bug 1164503] New: [Build20200219] general protection fault when installer triggers VM reboot
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 Bug ID: 1164503 Summary: [Build20200219] general protection fault when installer triggers VM reboot Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other URL: https://openqa.opensuse.org/tests/1180733/modules/remo te_target/steps/4 OS: Other Status: NEW Severity: Major Priority: P5 - None Component: Kernel Assignee: kernel-maintainers@forge.provo.novell.com Reporter: riafarov@suse.com QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- Created attachment 830830 --> http://bugzilla.opensuse.org/attachment.cgi?id=830830&action=edit YaST logs ## Observation We tried to enable VNC installation for TW, but noticed that there is an issue and system doesn't get rebooted in the end of installation (even if we wait more than 10 minutes). After closer look, we saw [ 788.547801] general protection fault: 0000 [#1] SMP PTI [ 788.552206] CPU: 0 PID: 3758 Comm: y2start Tainted: G W 5.5.4-1-default #1 openSUSE Tumbleweed (unreleased) Please, find full dmesg in separate comment. Also, we were not able to reproduce the issue manually and it happens only on the openqa.opensuse.org instance of openQA. Might be related to the qemu version. Attaching yast logs as they contain dmesg too and other relevant logs. openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-remote_vnc_target@rwx788/os-autoinst-distri-opensuse#yast@64bit_cirrus fails in [remote_target](https://openqa.opensuse.org/tests/1180733/modules/remote_target/steps/4) ## Test suite description The target of the vnc remote installation triggered by the controller ## Reproducible Fails since (at least) Build [rwx788/os-autoinst-distri-opensuse#yast](https://openqa.opensuse.org/tests/1180733) (current job) ## Expected result Last good: (unknown) (or more recent) ## Further details Always latest result in this scenario: [latest](https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=...) -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 http://bugzilla.opensuse.org/show_bug.cgi?id=1164503#c1 --- Comment #1 from Rodion Iafarov <riafarov@suse.com> --- [ 788.547096] ------------[ cut here ]------------ [ 788.547529] RIP: 0010:btrfs_release_global_block_rsv+0x70/0xc0 [btrfs] [ 788.547530] Code: 48 83 bb d8 01 00 00 00 75 54 48 83 bb e0 01 00 00 00 75 56 48 83 bb 18 02 00 00 00 75 58 48 83 bb 10 02 00 00 00 75 02 5b c3 <0f> 0b 5b c3 0f 0b 48 83 bb 70 01 00 00 00 74 b2 0f 0b 48 83 bb a0 [ 788.547531] RSP: 0018:ffffb286004efdb8 EFLAGS: 00010206 [ 788.547532] RAX: 0000000000a68000 RBX: ffff9a7c9ba52000 RCX: 0000000000000001 [ 788.547532] RDX: 0000000000000001 RSI: ffff9a7c9d56f000 RDI: ffff9a7c9d56f000 [ 788.547533] RBP: ffff9a7c9ba52000 R08: 0000000000000000 R09: ffffffffc0657b00 [ 788.547533] R10: ffff9a7c9e933000 R11: 0000000000a68000 R12: ffff9a7c9ba53470 [ 788.547534] R13: ffff9a7c9ba52090 R14: dead000000000100 R15: dead000000000100 [ 788.547535] FS: 00007f9ba9d7e740(0000) GS:ffff9a7c97800000(0000) knlGS:0000000000000000 [ 788.547536] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 788.547536] CR2: 00005614370113b0 CR3: 000000005ba68000 CR4: 00000000000006f0 [ 788.547539] Call Trace: [ 788.547554] btrfs_free_block_groups+0x212/0x2a0 [btrfs] [ 788.547573] close_ctree+0x270/0x2ff [btrfs] [ 788.547575] generic_shutdown_super+0x6c/0x100 [ 788.547576] kill_anon_super+0x14/0x30 [ 788.547600] btrfs_kill_super+0x12/0xa0 [btrfs] [ 788.547602] deactivate_locked_super+0x36/0x70 [ 788.547604] cleanup_mnt+0x104/0x160 [ 788.547605] task_work_run+0xa1/0xc0 [ 788.547607] exit_to_usermode_loop+0x10c/0x130 [ 788.547621] do_syscall_64+0x1fa/0x240 [ 788.547622] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 788.547623] RIP: 0033:0x7f9baa3dbd1b [ 788.547624] Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 d3 4c f9 ff 8b 7c 24 0c 41 89 c0 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 11 4d f9 ff 8b 44 [ 788.547625] RSP: 002b:00007ffd98765d30 EFLAGS: 00000293 ORIG_RAX: 0000000000000003 [ 788.547626] RAX: 0000000000000000 RBX: 0000561438970ad0 RCX: 00007f9baa3dbd1b [ 788.547627] RDX: 000056143756bac0 RSI: 0000000000000001 RDI: 000000000000002b [ 788.547627] RBP: 00005614389709a8 R08: 0000000000000000 R09: 0000000000000011 [ 788.547628] R10: 0000561431d7b370 R11: 0000000000000293 R12: 00000000000003b0 [ 788.547628] R13: 0000000000000496 R14: 00007f9baa4afe28 R15: 0000561432c28df0 [ 788.547630] ---[ end trace 3c6d2f1515b5a643 ]--- [ 788.547647] ------------[ cut here ]------------ [ 788.547664] WARNING: CPU: 0 PID: 3758 at fs/btrfs/block-group.c:3181 btrfs_free_block_groups+0x22a/0x2a0 [btrfs] [ 788.547664] Modules linked in: fuse nls_utf8 isofs usb_storage parport_pc parport btrfs xor raid6_pq libcrc32c dm_multipath dm_mod 8021q garp mrp stp llc arc4 libarc4 fan thermal nfs lockd grace fscache nls_iso8859_1 nls_cp437 af_packet sg st iscsi_ibft iscsi_boot_sysfs sunrpc hid_generic usbhid sr_mod cdrom cirrus drm_kms_helper ehci_pci ehci_hcd drm ata_generic usbcore ata_piix virtio_net joydev pcspkr serio_raw net_failover floppy failover virtio_blk virtio_scsi i2c_piix4 syscopyarea sysfillrect sysimgblt fb_sys_fops qemu_fw_cfg button scsi_dh_rdac scsi_dh_emc scsi_dh_alua edd squashfs loop [last unloaded: ppa] [ 788.547678] CPU: 0 PID: 3758 Comm: y2start Tainted: G W 5.5.4-1-default #1 openSUSE Tumbleweed (unreleased) [ 788.547678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-0-ga698c89-rebuilt.opensuse.org 04/01/2014 [ 788.547694] RIP: 0010:btrfs_free_block_groups+0x22a/0x2a0 [btrfs] [ 788.547695] Code: 14 00 00 49 be 00 01 00 00 00 00 ad de e8 4e 9c ff ff 48 8b 85 70 14 00 00 49 bd 22 01 00 00 00 00 ad de 49 39 c4 75 3c eb 5f <0f> 0b 31 c9 31 d2 4c 89 fe 48 89 ef e8 65 86 ff ff 48 8b 43 08 48 [ 788.547695] RSP: 0018:ffffb286004efdc8 EFLAGS: 00010206 [ 788.547696] RAX: ffff9a7c9d56f088 RBX: ffff9a7c9d56f088 RCX: 00000000000b7f02 [ 788.547697] RDX: 00000000000b7f01 RSI: ffff9a7c97834040 RDI: 0000000000034040 [ 788.547697] RBP: ffff9a7c9ba52000 R08: 0000000080080005 R09: ffffffffa936e101 [ 788.547698] R10: ffff9a7c9e072c00 R11: ffff9a7c4fd15018 R12: ffff9a7c9ba53470 [ 788.547699] R13: dead000000000122 R14: dead000000000100 R15: ffff9a7c9d56f000 [ 788.547700] FS: 00007f9ba9d7e740(0000) GS:ffff9a7c97800000(0000) knlGS:0000000000000000 [ 788.547700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 788.547701] CR2: 00005614370113b0 CR3: 000000005ba68000 CR4: 00000000000006f0 [ 788.547703] Call Trace: [ 788.547722] close_ctree+0x270/0x2ff [btrfs] [ 788.547724] generic_shutdown_super+0x6c/0x100 [ 788.547726] kill_anon_super+0x14/0x30 [ 788.547736] btrfs_kill_super+0x12/0xa0 [btrfs] [ 788.547738] deactivate_locked_super+0x36/0x70 [ 788.547739] cleanup_mnt+0x104/0x160 [ 788.547741] task_work_run+0xa1/0xc0 [ 788.547743] exit_to_usermode_loop+0x10c/0x130 [ 788.547744] do_syscall_64+0x1fa/0x240 [ 788.547746] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 788.547747] RIP: 0033:0x7f9baa3dbd1b [ 788.547748] Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 d3 4c f9 ff 8b 7c 24 0c 41 89 c0 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 11 4d f9 ff 8b 44 [ 788.547748] RSP: 002b:00007ffd98765d30 EFLAGS: 00000293 ORIG_RAX: 0000000000000003 [ 788.547752] RAX: 0000000000000000 RBX: 0000561438970ad0 RCX: 00007f9baa3dbd1b [ 788.547753] RDX: 000056143756bac0 RSI: 0000000000000001 RDI: 000000000000002b [ 788.547753] RBP: 00005614389709a8 R08: 0000000000000000 R09: 0000000000000011 [ 788.547754] R10: 0000561431d7b370 R11: 0000000000000293 R12: 00000000000003b0 [ 788.547754] R13: 0000000000000496 R14: 00007f9baa4afe28 R15: 0000561432c28df0 [ 788.547756] ---[ end trace 3c6d2f1515b5a644 ]--- [ 788.547762] BTRFS info (device vda2): space_info 4 has 144916480 free, is not full [ 788.547763] BTRFS info (device vda2): space_info total=268435456, used=123174912, pinned=0, reserved=16384, may_use=262144, readonly=65536 [ 788.547764] BTRFS info (device vda2): global_block_rsv: size 0 reserved 0 [ 788.547765] BTRFS info (device vda2): trans_block_rsv: size 0 reserved 0 [ 788.547766] BTRFS info (device vda2): chunk_block_rsv: size 0 reserved 0 [ 788.547767] BTRFS info (device vda2): delayed_block_rsv: size 0 reserved 0 [ 788.547767] BTRFS info (device vda2): delayed_refs_rsv: size 524288 reserved 262144 [ 788.547801] general protection fault: 0000 [#1] SMP PTI [ 788.552206] CPU: 0 PID: 3758 Comm: y2start Tainted: G W 5.5.4-1-default #1 openSUSE Tumbleweed (unreleased) [ 788.553581] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-0-ga698c89-rebuilt.opensuse.org 04/01/2014 [ 788.554970] RIP: 0010:__queue_work+0x1f/0x3e0 [ 788.555510] Code: ff 0f 0b e9 7a ff ff ff 0f 1f 00 0f 1f 44 00 00 41 57 49 89 d7 41 56 41 89 fe 41 55 41 89 fd 41 54 49 89 f4 55 53 48 83 ec 10 <f6> 86 02 01 00 00 01 0f 85 e7 02 00 00 48 bd eb 83 b5 80 46 86 c8 [ 788.557706] RSP: 0018:ffffb286004efa98 EFLAGS: 00010082 [ 788.558324] RAX: 0000000080000000 RBX: 0000000000000286 RCX: ffff9a7c9ee25190 [ 788.559161] RDX: ffff9a7c9aecd880 RSI: 8c330b1ed38612fd RDI: 0000000000000200 [ 788.560001] RBP: 0000000000000200 R08: ffff9a7c97834100 R09: ffff9a7c9a50b2e0 [ 788.560860] R10: 00000000000ee1ce R11: 0000000000008000 R12: 8c330b1ed38612fd [ 788.561705] R13: 0000000000000200 R14: 0000000000000200 R15: ffff9a7c9aecd880 [ 788.562542] FS: 00007f9ba9d7e740(0000) GS:ffff9a7c97800000(0000) knlGS:0000000000000000 [ 788.563488] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 788.564155] CR2: 00005614370113b0 CR3: 000000005ba68000 CR4: 00000000000006f0 [ 788.565031] Call Trace: [ 788.565354] queue_work_on+0x85/0x90 [ 788.565813] btrfs_wq_submit_bio+0xa2/0xc0 [btrfs] [ 788.566394] btree_submit_bio_hook+0x53/0xc0 [btrfs] [ 788.567020] ? btree_csum_one_bio+0x230/0x230 [btrfs] [ 788.567629] submit_one_bio+0x31/0x50 [btrfs] [ 788.568163] btree_write_cache_pages+0x356/0x370 [btrfs] [ 788.568809] ? irq_work_queue+0xb/0x30 [ 788.569250] ? wake_up_klogd+0x30/0x40 [ 788.569693] ? vprintk_emit+0x103/0x260 [ 788.570151] do_writepages+0x43/0xd0 [ 788.570600] __writeback_single_inode+0x3d/0x350 [ 788.571164] writeback_single_inode+0xad/0x120 [ 788.571692] write_inode_now+0x86/0xc0 [ 788.572139] iput+0x172/0x200 [ 788.572522] close_ctree+0x27c/0x2ff [btrfs] [ 788.573032] generic_shutdown_super+0x6c/0x100 [ 788.573558] kill_anon_super+0x14/0x30 [ 788.574016] btrfs_kill_super+0x12/0xa0 [btrfs] [ 788.574561] deactivate_locked_super+0x36/0x70 [ 788.575096] cleanup_mnt+0x104/0x160 [ 788.575522] task_work_run+0xa1/0xc0 [ 788.575975] exit_to_usermode_loop+0x10c/0x130 [ 788.576509] do_syscall_64+0x1fa/0x240 [ 788.576963] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 788.577571] RIP: 0033:0x7f9baa3dbd1b [ 788.577998] Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 d3 4c f9 ff 8b 7c 24 0c 41 89 c0 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 11 4d f9 ff 8b 44 [ 788.580272] RSP: 002b:00007ffd98765d30 EFLAGS: 00000293 ORIG_RAX: 0000000000000003 [ 788.581183] RAX: 0000000000000000 RBX: 0000561438970ad0 RCX: 00007f9baa3dbd1b [ 788.582063] RDX: 000056143756bac0 RSI: 0000000000000001 RDI: 000000000000002b [ 788.582937] RBP: 00005614389709a8 R08: 0000000000000000 R09: 0000000000000011 [ 788.583778] R10: 0000561431d7b370 R11: 0000000000000293 R12: 00000000000003b0 [ 788.584635] R13: 0000000000000496 R14: 00007f9baa4afe28 R15: 0000561432c28df0 [ 788.585471] Modules linked in: fuse nls_utf8 isofs usb_storage parport_pc parport btrfs xor raid6_pq libcrc32c dm_multipath dm_mod 8021q garp mrp stp llc arc4 libarc4 fan thermal nfs lockd grace fscache nls_iso8859_1 nls_cp437 af_packet sg st iscsi_ibft iscsi_boot_sysfs sunrpc hid_generic usbhid sr_mod cdrom cirrus drm_kms_helper ehci_pci ehci_hcd drm ata_generic usbcore ata_piix virtio_net joydev pcspkr serio_raw net_failover floppy failover virtio_blk virtio_scsi i2c_piix4 syscopyarea sysfillrect sysimgblt fb_sys_fops qemu_fw_cfg button scsi_dh_rdac scsi_dh_emc scsi_dh_alua edd squashfs loop [last unloaded: ppa] [ 788.591977] ---[ end trace 3c6d2f1515b5a645 ]--- [ 788.592534] RIP: 0010:__queue_work+0x1f/0x3e0 [ 788.593050] Code: ff 0f 0b e9 7a ff ff ff 0f 1f 00 0f 1f 44 00 00 41 57 49 89 d7 41 56 41 89 fe 41 55 41 89 fd 41 54 49 89 f4 55 53 48 83 ec 10 <f6> 86 02 01 00 00 01 0f 85 e7 02 00 00 48 bd eb 83 b5 80 46 86 c8 [ 788.595242] RSP: 0018:ffffb286004efa98 EFLAGS: 00010082 [ 788.595852] RAX: 0000000080000000 RBX: 0000000000000286 RCX: ffff9a7c9ee25190 [ 788.596722] RDX: ffff9a7c9aecd880 RSI: 8c330b1ed38612fd RDI: 0000000000000200 [ 788.597555] RBP: 0000000000000200 R08: ffff9a7c97834100 R09: ffff9a7c9a50b2e0 [ 788.598410] R10: 00000000000ee1ce R11: 0000000000008000 R12: 8c330b1ed38612fd [ 788.599246] R13: 0000000000000200 R14: 0000000000000200 R15: ffff9a7c9aecd880 [ 788.600081] FS: 00007f9ba9d7e740(0000) GS:ffff9a7c97800000(0000) knlGS:0000000000000000 [ 788.601043] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 788.601729] CR2: 00005614370113b0 CR3: 000000005ba68000 CR4: 00000000000006f0 [ 788.547138] WARNING: CPU: 0 PID: 3758 at fs/btrfs/block-group.c:132 btrfs_put_block_group+0x42/0x50 [btrfs] [ 788.547138] Modules linked in: fuse nls_utf8 isofs usb_storage parport_pc parport btrfs xor raid6_pq libcrc32c dm_multipath dm_mod 8021q garp mrp stp llc arc4 libarc4 fan thermal nfs lockd grace fscache nls_iso8859_1 nls_cp437 af_packet sg st iscsi_ibft iscsi_boot_sysfs sunrpc hid_generic usbhid sr_mod cdrom cirrus drm_kms_helper ehci_pci ehci_hcd drm ata_generic usbcore ata_piix virtio_net joydev pcspkr serio_raw net_failover floppy failover virtio_blk virtio_scsi i2c_piix4 syscopyarea sysfillrect sysimgblt fb_sys_fops qemu_fw_cfg button scsi_dh_rdac scsi_dh_emc scsi_dh_alua edd squashfs loop [last unloaded: ppa] [ 788.547165] CPU: 0 PID: 3758 Comm: y2start Not tainted 5.5.4-1-default #1 openSUSE Tumbleweed (unreleased) [ 788.547165] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-0-ga698c89-rebuilt.opensuse.org 04/01/2014 [ 788.547187] RIP: 0010:btrfs_put_block_group+0x42/0x50 [btrfs] [ 788.547189] Code: 2d 48 83 7d 30 00 75 22 48 8b 85 d0 01 00 00 48 85 c0 75 1e 48 8b bd c0 00 00 00 e8 f8 ca 64 e8 48 89 ef 5d e9 ef ca 64 e8 c3 <0f> 0b eb da 0f 0b eb cf 0f 0b eb de 66 90 0f 1f 44 00 00 31 d2 e9 [ 788.547190] RSP: 0018:ffffb286004efdb8 EFLAGS: 00010206 [ 788.547191] RAX: 0000000000000001 RBX: ffff9a7c9e933ac8 RCX: 0000000080380004 [ 788.547192] RDX: ffff9a7c9e933b08 RSI: 0000000000000001 RDI: ffff9a7c9e933a00 [ 788.547192] RBP: ffff9a7c9e933a00 R08: 0000000000000000 R09: ffffffffc0611400 [ 788.547193] R10: ffff9a7c96829510 R11: 0000000000000001 R12: ffff9a7c9ba52080 [ 788.547193] R13: ffff9a7c9ba52090 R14: ffff9a7c9e933a00 R15: dead000000000100 [ 788.547195] FS: 00007f9ba9d7e740(0000) GS:ffff9a7c97800000(0000) knlGS:0000000000000000 [ 788.547196] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 788.547196] CR2: 00005614370113b0 CR3: 000000005ba68000 CR4: 00000000000006f0 [ 788.547199] Call Trace: [ 788.547230] btrfs_free_block_groups+0x158/0x2a0 [btrfs] [ 788.547250] close_ctree+0x270/0x2ff [btrfs] [ 788.547258] generic_shutdown_super+0x6c/0x100 [ 788.547271] kill_anon_super+0x14/0x30 [ 788.547282] btrfs_kill_super+0x12/0xa0 [btrfs] [ 788.547288] deactivate_locked_super+0x36/0x70 [ 788.547291] cleanup_mnt+0x104/0x160 [ 788.547294] task_work_run+0xa1/0xc0 [ 788.547299] exit_to_usermode_loop+0x10c/0x130 [ 788.547304] do_syscall_64+0x1fa/0x240 [ 788.547311] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 788.547321] RIP: 0033:0x7f9baa3dbd1b [ 788.547323] Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 d3 4c f9 ff 8b 7c 24 0c 41 89 c0 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 11 4d f9 ff 8b 44 [ 788.547324] RSP: 002b:00007ffd98765d30 EFLAGS: 00000293 ORIG_RAX: 0000000000000003 [ 788.547325] RAX: 0000000000000000 RBX: 0000561438970ad0 RCX: 00007f9baa3dbd1b [ 788.547326] RDX: 000056143756bac0 RSI: 0000000000000001 RDI: 000000000000002b [ 788.547327] RBP: 00005614389709a8 R08: 0000000000000000 R09: 0000000000000011 [ 788.547327] R10: 0000561431d7b370 R11: 0000000000000293 R12: 00000000000003b0 [ 788.547328] R13: 0000000000000496 R14: 00007f9baa4afe28 R15: 0000561432c28df0 [ 788.547330] ---[ end trace 3c6d2f1515b5a641 ]--- [ 788.547342] ------------[ cut here ]------------ [ 788.547361] WARNING: CPU: 0 PID: 3758 at fs/btrfs/block-rsv.c:354 btrfs_release_global_block_rsv+0xbc/0xc0 [btrfs] [ 788.547361] Modules linked in: fuse nls_utf8 isofs usb_storage parport_pc parport btrfs xor raid6_pq libcrc32c dm_multipath dm_mod 8021q garp mrp stp llc arc4 libarc4 fan thermal nfs lockd grace fscache nls_iso8859_1 nls_cp437 af_packet sg st iscsi_ibft iscsi_boot_sysfs sunrpc hid_generic usbhid sr_mod cdrom cirrus drm_kms_helper ehci_pci ehci_hcd drm ata_generic usbcore ata_piix virtio_net joydev pcspkr serio_raw net_failover floppy failover virtio_blk virtio_scsi i2c_piix4 syscopyarea sysfillrect sysimgblt fb_sys_fops qemu_fw_cfg button scsi_dh_rdac scsi_dh_emc scsi_dh_alua edd squashfs loop [last unloaded: ppa] [ 788.547375] CPU: 0 PID: 3758 Comm: y2start Tainted: G W 5.5.4-1-default #1 openSUSE Tumbleweed (unreleased) [ 788.547376] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-0-ga698c89-rebuilt.opensuse.org 04/01/2014 [ 788.547393] RIP: 0010:btrfs_release_global_block_rsv+0xbc/0xc0 [btrfs] [ 788.547394] Code: 01 00 00 00 74 ae 0f 0b 48 83 bb d8 01 00 00 00 74 ac 0f 0b 48 83 bb e0 01 00 00 00 74 aa 0f 0b 48 83 bb 18 02 00 00 00 74 a8 <0f> 0b eb a4 0f 1f 44 00 00 41 56 41 55 49 89 f5 41 54 55 53 48 8b [ 788.547395] RSP: 0018:ffffb286004efdb8 EFLAGS: 00010206 [ 788.547396] RAX: 0000000000a68000 RBX: ffff9a7c9ba52000 RCX: 0000000000000001 [ 788.547396] RDX: 0000000000000001 RSI: ffff9a7c9d56f000 RDI: ffff9a7c9d56f000 [ 788.547397] RBP: ffff9a7c9ba52000 R08: 0000000000000000 R09: ffffffffc0657b00 [ 788.547400] R10: ffff9a7c9e933000 R11: 0000000000a68000 R12: ffff9a7c9ba53470 [ 788.547400] R13: ffff9a7c9ba52090 R14: dead000000000100 R15: dead000000000100 [ 788.547402] FS: 00007f9ba9d7e740(0000) GS:ffff9a7c97800000(0000) knlGS:0000000000000000 [ 788.547402] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 788.547403] CR2: 00005614370113b0 CR3: 000000005ba68000 CR4: 00000000000006f0 [ 788.547405] Call Trace: [ 788.547421] btrfs_free_block_groups+0x212/0x2a0 [btrfs] [ 788.547440] close_ctree+0x270/0x2ff [btrfs] [ 788.547442] generic_shutdown_super+0x6c/0x100 [ 788.547444] kill_anon_super+0x14/0x30 [ 788.547454] btrfs_kill_super+0x12/0xa0 [btrfs] [ 788.547456] deactivate_locked_super+0x36/0x70 [ 788.547457] cleanup_mnt+0x104/0x160 [ 788.547459] task_work_run+0xa1/0xc0 [ 788.547461] exit_to_usermode_loop+0x10c/0x130 [ 788.547462] do_syscall_64+0x1fa/0x240 [ 788.547464] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 788.547465] RIP: 0033:0x7f9baa3dbd1b [ 788.547466] Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 d3 4c f9 ff 8b 7c 24 0c 41 89 c0 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 11 4d f9 ff 8b 44 [ 788.547467] RSP: 002b:00007ffd98765d30 EFLAGS: 00000293 ORIG_RAX: 0000000000000003 [ 788.547468] RAX: 0000000000000000 RBX: 0000561438970ad0 RCX: 00007f9baa3dbd1b [ 788.547469] RDX: 000056143756bac0 RSI: 0000000000000001 RDI: 000000000000002b [ 788.547469] RBP: 00005614389709a8 R08: 0000000000000000 R09: 0000000000000011 [ 788.547470] R10: 0000561431d7b370 R11: 0000000000000293 R12: 00000000000003b0 [ 788.547470] R13: 0000000000000496 R14: 00007f9baa4afe28 R15: 0000561432c28df0 [ 788.547472] ---[ end trace 3c6d2f1515b5a642 ]--- [ 788.547481] ------------[ cut here ]------------ [ 788.547498] WARNING: CPU: 0 PID: 3758 at fs/btrfs/block-rsv.c:355 btrfs_release_global_block_rsv+0x70/0xc0 [btrfs] [ 788.547499] Modules linked in: fuse nls_utf8 isofs usb_storage parport_pc parport btrfs xor raid6_pq libcrc32c dm_multipath dm_mod 8021q garp mrp stp llc arc4 libarc4 fan thermal nfs lockd grace fscache nls_iso8859_1 nls_cp437 af_packet sg st iscsi_ibft iscsi_boot_sysfs sunrpc hid_generic usbhid sr_mod cdrom cirrus drm_kms_helper ehci_pci ehci_hcd drm ata_generic usbcore ata_piix virtio_net joydev pcspkr serio_raw net_failover floppy failover virtio_blk virtio_scsi i2c_piix4 syscopyarea sysfillrect sysimgblt fb_sys_fops qemu_fw_cfg button scsi_dh_rdac scsi_dh_emc scsi_dh_alua edd squashfs loop [last unloaded: ppa] [ 788.547512] CPU: 0 PID: 3758 Comm: y2start Tainted: G W 5.5.4-1-default #1 openSUSE Tumbleweed (unreleased) [ 788.547513] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-0-ga698c89-rebuilt.opensuse.org 04/01/2014 [ 924.081196] sysrq: Show Blocked State [ 924.081699] task PC stack pid father -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 Oliver Kurz <okurz@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |okurz@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 http://bugzilla.opensuse.org/show_bug.cgi?id=1164503#c2 --- Comment #2 from Filipe Manana <fdmanana@suse.com> --- So is this failing all the time or just often? And how often? I'm seeing one race here where we get dirty pages in the btree inode after stopping all work queues, that will definitely trigger the crash and the warnings before it. But hitting this race should be really hard. Thanks. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 http://bugzilla.opensuse.org/show_bug.cgi?id=1164503#c4 --- Comment #4 from Rodion Iafarov <riafarov@suse.com> --- (In reply to Filipe Manana from comment #2)
So is this failing all the time or just often? And how often?
I'm seeing one race here where we get dirty pages in the btree inode after stopping all work queues, that will definitely trigger the crash and the warnings before it. But hitting this race should be really hard.
Thanks.
Hi Filipe! Actually, seems that in openQA it's failing every time (but it's not to get interactive shell on that VM). Manually I was not able to reproduce this at all after many tries. So in case you have some idea what we can get from the run to proceed with investigation - let us know. Thanks! -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 http://bugzilla.opensuse.org/show_bug.cgi?id=1164503#c6 --- Comment #6 from Filipe Manana <fdmanana@suse.com> --- Hum, So the race I was looking at that can cause a crash during unmount is different from the one you are reporting. I've sent a fix for it upstream (not yet on Linus tree, only linux-next): https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?... The one being reported implies there are still dirty pages of the btrfs' btree inode at the very end of the unmount path, which is not supposed to happen. The only way I can see to help debug this is by creating a debug patch, for the kernel, with some assertions (BUG_ON or WARN_ON) and printks at a few suspected places under some special conditions. However, how to get such a patch into a kernel into tumbleweed? Any idea? Thanks. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 http://bugzilla.opensuse.org/show_bug.cgi?id=1164503#c8 --- Comment #8 from Filipe Manana <fdmanana@suse.com> --- (In reply to Rodion Iafarov from comment #7)
(In reply to Filipe Manana from comment #6)
Hum,
So the race I was looking at that can cause a crash during unmount is different from the one you are reporting. I've sent a fix for it upstream (not yet on Linus tree, only linux-next):
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/ ?h=next-20200316&id=48693eda22cf640fa6f7b0ce87941fe771262f69
The one being reported implies there are still dirty pages of the btrfs' btree inode at the very end of the unmount path, which is not supposed to happen.
The only way I can see to help debug this is by creating a debug patch, for the kernel, with some assertions (BUG_ON or WARN_ON) and printks at a few suspected places under some special conditions.
However, how to get such a patch into a kernel into tumbleweed? Any idea?
Thanks.
Sorry for the late response, we had issues logging into bugzilla. We could have used mksusecd tool to update kernel in the image, like it's described in this blog entry and run openQA test using that image, as I wasn't able to reproduce it manually.
No worries, I got delayed too.
Of course, adding some printk might fix the issue as is expensive operation, but we could try.
Indeed. However my printks only are run if something unexpected happens. I've experienced once one other crash during unmount, where we end up the delayed iputs list not empty - that can be a problem if an inode has a link count of 0, since the iput will create a transaction and dirty some pages of the btree inode. You are getting a crash because the btree inode still has dirty pages, it may be because the delayed iputs list is not empty (either you have CONFIG_BTRFS_ASSERT disabled or an iput was added to the list after we asserted it was empty during the unmount. Other thing I suspected besides the delayed iput list not being empty is that after we commit the last transaction there's still some job running on one of the workqueues that creates a transaction and dirties some pages in the btree inode. I've added a BUG_ON() to detect that for which the resulting trace will let me known which type of worker/workqueue is doing that. Debug patch attached. Thanks! -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1164503 http://bugzilla.opensuse.org/show_bug.cgi?id=1164503#c9 --- Comment #9 from Filipe Manana <fdmanana@suse.com> --- Created attachment 833516 --> http://bugzilla.opensuse.org/attachment.cgi?id=833516&action=edit Debug patch -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com