http://bugzilla.suse.com/show_bug.cgi?id=1037847 Bug ID: 1037847 Summary: btrfs goes read only with fs/btrfs/extent-tree.c:2956 btrfs_run_delayed_refs+0x282/0x2e0 BTRFS: Transaction aborted (error -28) Classification: openSUSE Product: openSUSE Distribution Version: Leap 42.2 Hardware: x86-64 OS: All Status: NEW Severity: Critical Priority: P5 - None Component: Kernel Assignee: kernel-maintainers@forge.provo.novell.com Reporter: s.priebe@profihost.ag QA Contact: qa-bugs@suse.de CC: dchang@suse.com, donald.buchholz@intel.com, tiwai@suse.com Depends on: 1034635 Found By: --- Blocker: --- Hello, while running a git based kernel build from openSUSE-42.2 commit id 328b08be8a2f8a1d09c27e5f8038f9c7446669c3. I'm no longer able to write more than 1 to 2 hours of data to my disk. # dmesg: ------------[ cut here ]------------ WARNING: CPU: 5 PID: 6589 at fs/btrfs/extent-tree.c:2956 btrfs_run_delayed_refs+0x282/0x2e0 [btrfs]() BTRFS: Transaction aborted (error -28) Modules linked in: netconsole xt_multiport ipt_REJECT nf_reject_ipv4 xt_set iptable_filter ip_tables x_tables ip_set_hash_net ip_set nfnetlink 8021q garp bonding ipmi_devintf x86_pkg_temp_thermal coretemp ast kvm_intel ttm kvm drm_kms_helper irqbypass drm crc32_pclmul ghash_clmulni_intel sb_edac fb_sys_fops syscopyarea sysfillrect sysimgblt edac_core i40e(O) vxlan ip6_udp_tunnel udp_tunnel shpchp wmi ipmi_si ipmi_msghandler button loop btrfs dm_mod raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid0 multipath linear usbhid raid1 md_mod sg sd_mod xhci_pci ehci_pci xhci_hcd ehci_hcd i2c_i801 ahci igb usbcore i2c_algo_bit usb_common libahci i2c_core ptp pps_core aacraid CPU: 5 PID: 6589 Comm: btrfs-transacti Tainted: G W O 4.4.65+132-ph #1 Hardware name: Supermicro Super Server/X10SRi-F, BIOS 2.0 12/17/2015 0000000000000000 ffff880a7847bbb8 ffffffff813c23f3 ffff880a7847bc00 ffffffffc037114d ffff880a7847bbf0 ffffffff81083746 ffff880ff32f23c0 ffff880cac6c2000 ffff880fb4ee6060 ffffffffffffffff ffff880cac6c2000 Call Trace: [<ffffffff81008b45>] try_stack_unwind+0x175/0x190 [<ffffffff81007d24>] dump_trace+0x64/0x380 [<ffffffff81008bad>] show_trace_log_lvl+0x4d/0x60 [<ffffffff81008148>] show_stack_log_lvl+0x108/0x170 [<ffffffff81008c05>] show_stack+0x25/0x70 [<ffffffff813c23f3>] dump_stack+0x63/0x90 [<ffffffff81083746>] warn_slowpath_common+0x86/0xc0 [<ffffffff810837cc>] warn_slowpath_fmt+0x4c/0x50 [<ffffffffc02d6322>] btrfs_run_delayed_refs+0x282/0x2e0 [btrfs] [<ffffffffc02ebdd8>] create_pending_snapshot.isra.23+0x5b8/0x1010 [btrfs] [<ffffffffc02ec8ba>] create_pending_snapshots+0x8a/0xb0 [btrfs] [<ffffffffc02ecf11>] btrfs_commit_transaction.part.24+0x371/0xa30 [btrfs] [<ffffffffc02ed60a>] btrfs_commit_transaction+0x3a/0x70 [btrfs] [<ffffffffc02e78df>] transaction_kthread+0x1ff/0x260 [btrfs] [<ffffffff810a0f5b>] kthread+0xeb/0x110 [<ffffffff816bdf0f>] ret_from_fork+0x3f/0x70 DWARF2 unwinder stuck at ret_from_fork+0x3f/0x70 Leftover inexact backtrace: [<ffffffff810a0e70>] ? kthread_park+0x60/0x60 ---[ end trace 457d5c0a7792677e ]--- BTRFS: error (device sdc1) in btrfs_run_delayed_refs:2956: errno=-28 No space left BTRFS info (device sdc1): forced readonly BTRFS: error (device sdc1) in create_pending_snapshot:1603: errno=-28 No space left BTRFS warning (device sdc1): Skipping commit of aborted transaction. BTRFS: error (device sdc1) in cleanup_transaction:1852: errno=-28 No space left BTRFS error (device sdc1): pending csums is 383705088 I i umount the device i get a new trace: # umount /backup # dmesg -t BTRFS error (device sdc1): cleaner transaction attach returned -30 ------------[ cut here ]------------ WARNING: CPU: 2 PID: 12105 at fs/btrfs/extent-tree.c:133 btrfs_put_block_group+0x60/0x70 [btrfs]() Modules linked in: netconsole xt_multiport ipt_REJECT nf_reject_ipv4 xt_set iptable_filter ip_tables x_tables ip_set_hash_net ip_set nfnetlink 8021q garp bonding ipmi_devintf x86_pkg_temp_thermal coretemp ast kvm_intel ttm kvm drm_kms_helper irqbypass drm crc32_pclmul ghash_clmulni_intel sb_edac fb_sys_fops syscopyarea sysfillrect sysimgblt edac_core i40e(O) vxlan ip6_udp_tunnel udp_tunnel shpchp wmi ipmi_si ipmi_msghandler button loop btrfs dm_mod raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid0 multipath linear usbhid raid1 md_mod sg sd_mod xhci_pci ehci_pci xhci_hcd ehci_hcd i2c_i801 ahci igb usbcore i2c_algo_bit usb_common libahci i2c_core ptp pps_core aacraid CPU: 2 PID: 12105 Comm: umount Tainted: G W O 4.4.65+132-ph #1 Hardware name: Supermicro Super Server/X10SRi-F, BIOS 2.0 12/17/2015 0000000000000000 ffff880753a13ce0 ffffffff813c23f3 0000000000000000 ffffffffc037114d ffff880753a13d18 ffffffff81083746 ffff881030d25000 ffff881030d25000 ffff88103c3c4088 ffff88103c3c4000 ffff88103c3c4098 Call Trace: [<ffffffff81008b45>] try_stack_unwind+0x175/0x190 [<ffffffff81007d24>] dump_trace+0x64/0x380 [<ffffffff81008bad>] show_trace_log_lvl+0x4d/0x60 [<ffffffff81008148>] show_stack_log_lvl+0x108/0x170 [<ffffffff81008c05>] show_stack+0x25/0x70 [<ffffffff813c23f3>] dump_stack+0x63/0x90 [<ffffffff81083746>] warn_slowpath_common+0x86/0xc0 [<ffffffff8108383a>] warn_slowpath_null+0x1a/0x20 [<ffffffffc02cd670>] btrfs_put_block_group+0x60/0x70 [btrfs] [<ffffffffc02d5358>] btrfs_free_block_groups+0xf8/0x430 [btrfs] [<ffffffffc02e7a8d>] close_ctree+0x14d/0x320 [btrfs] [<ffffffffc02b6429>] btrfs_put_super+0x19/0x20 [btrfs] [<ffffffff811d2aff>] generic_shutdown_super+0x6f/0x100 [<ffffffff811d2ba2>] kill_anon_super+0x12/0x20 [<ffffffffc02b71b6>] btrfs_kill_super+0x16/0x90 [btrfs] [<ffffffff811d24d3>] deactivate_locked_super+0x43/0x70 [<ffffffff811d255c>] deactivate_super+0x5c/0x60 [<ffffffff811ee23f>] cleanup_mnt+0x3f/0x90 [<ffffffff811ee2d2>] __cleanup_mnt+0x12/0x20 [<ffffffff8109f51e>] task_work_run+0x7e/0xa0 [<ffffffff81002383>] exit_to_usermode_loop+0xf3/0x100 [<ffffffff81002bb4>] syscall_return_slowpath+0xd4/0x130 [<ffffffff816bdd0c>] int_ret_from_sys_call+0x25/0x8f DWARF2 unwinder stuck at int_ret_from_sys_call+0x25/0x8f Leftover inexact backtrace: ---[ end trace 457d5c0a7792677f ]--- ------------[ cut here ]------------ WARNING: CPU: 2 PID: 12105 at fs/btrfs/extent-tree.c:9818 btrfs_free_block_groups+0x291/0x430 [btrfs]() Modules linked in: netconsole xt_multiport ipt_REJECT nf_reject_ipv4 xt_set iptable_filter ip_tables x_tables ip_set_hash_net ip_set nfnetlink 8021q garp bonding ipmi_devintf x86_pkg_temp_thermal coretemp ast kvm_intel ttm kvm drm_kms_helper irqbypass drm crc32_pclmul ghash_clmulni_intel sb_edac fb_sys_fops syscopyarea sysfillrect sysimgblt edac_core i40e(O) vxlan ip6_udp_tunnel udp_tunnel shpchp wmi ipmi_si ipmi_msghandler button loop btrfs dm_mod raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid0 multipath linear usbhid raid1 md_mod sg sd_mod xhci_pci ehci_pci xhci_hcd ehci_hcd i2c_i801 ahci igb usbcore i2c_algo_bit usb_common libahci i2c_core ptp pps_core aacraid CPU: 2 PID: 12105 Comm: umount Tainted: G W O 4.4.65+132-ph #1 Hardware name: Supermicro Super Server/X10SRi-F, BIOS 2.0 12/17/2015 0000000000000000 ffff880753a13cf8 ffffffff813c23f3 0000000000000000 ffffffffc037114d ffff880753a13d30 ffffffff81083746 ffff880f0feb2200 ffff880f0feb2288 ffff88103c3c4088 ffff88103c3c4000 ffff88103c3c4098 Call Trace: [<ffffffff81008b45>] try_stack_unwind+0x175/0x190 [<ffffffff81007d24>] dump_trace+0x64/0x380 [<ffffffff81008bad>] show_trace_log_lvl+0x4d/0x60 [<ffffffff81008148>] show_stack_log_lvl+0x108/0x170 [<ffffffff81008c05>] show_stack+0x25/0x70 [<ffffffff813c23f3>] dump_stack+0x63/0x90 [<ffffffff81083746>] warn_slowpath_common+0x86/0xc0 [<ffffffff8108383a>] warn_slowpath_null+0x1a/0x20 [<ffffffffc02d54f1>] btrfs_free_block_groups+0x291/0x430 [btrfs] [<ffffffffc02e7a8d>] close_ctree+0x14d/0x320 [btrfs] [<ffffffffc02b6429>] btrfs_put_super+0x19/0x20 [btrfs] [<ffffffff811d2aff>] generic_shutdown_super+0x6f/0x100 [<ffffffff811d2ba2>] kill_anon_super+0x12/0x20 [<ffffffffc02b71b6>] btrfs_kill_super+0x16/0x90 [btrfs] [<ffffffff811d24d3>] deactivate_locked_super+0x43/0x70 [<ffffffff811d255c>] deactivate_super+0x5c/0x60 [<ffffffff811ee23f>] cleanup_mnt+0x3f/0x90 [<ffffffff811ee2d2>] __cleanup_mnt+0x12/0x20 [<ffffffff8109f51e>] task_work_run+0x7e/0xa0 [<ffffffff81002383>] exit_to_usermode_loop+0xf3/0x100 [<ffffffff81002bb4>] syscall_return_slowpath+0xd4/0x130 [<ffffffff816bdd0c>] int_ret_from_sys_call+0x25/0x8f DWARF2 unwinder stuck at int_ret_from_sys_call+0x25/0x8f Leftover inexact backtrace: ---[ end trace 457d5c0a77926780 ]--- BTRFS info (device sdc1): space_info 1 has 144388956160 free, is not full BTRFS info (device sdc1): space_info total=21839908700160, used=21695519477760, pinned=0, reserved=4096, may_use=0, readonly=262144 # btrfs filesystem df /vmbackup/ Data, RAID0: total=19.86TiB, used=19.72TiB System, RAID0: total=64.00MiB, used=1.23MiB Metadata, RAID0: total=76.00GiB, used=72.62GiB GlobalReserve, single: total=512.00MiB, used=6.16MiB # btrfs filesystem usage /vmbackup/ Overall: Device size: 58.20TiB Device allocated: 19.94TiB Device unallocated: 38.26TiB Device missing: 0.00B Used: 19.79TiB Free (estimated): 38.41TiB (min: 38.41TiB) Data ratio: 1.00 Metadata ratio: 1.00 Global reserve: 512.00MiB (used: 213.95MiB) Data,RAID0: Size:19.86TiB, Used:19.72TiB /dev/sdc1 4.97TiB /dev/sdd1 4.97TiB /dev/sde1 4.97TiB /dev/sdf1 4.97TiB Metadata,RAID0: Size:76.00GiB, Used:72.61GiB /dev/sdc1 19.00GiB /dev/sdd1 19.00GiB /dev/sde1 19.00GiB /dev/sdf1 19.00GiB System,RAID0: Size:64.00MiB, Used:1.23MiB /dev/sdc1 16.00MiB /dev/sdd1 16.00MiB /dev/sde1 16.00MiB /dev/sdf1 16.00MiB Unallocated: /dev/sdc1 9.57TiB /dev/sdd1 9.57TiB /dev/sde1 9.57TiB /dev/sdf1 9.57TiB I already tried to clear the space cache but it happens again after some time. Greets, Stefan -- You are receiving this mail because: You are on the CC list for the bug.