[Bug 679898] New: kernel OOPS while writing to ext4 loop
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c0 Summary: kernel OOPS while writing to ext4 loop Classification: openSUSE Product: openSUSE 11.5 Version: Factory Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Kernel AssignedTo: jack@novell.com ReportedBy: coolo@novell.com QAContact: qa@suse.de Found By: --- Blocker: --- While I experimented with clicfs and ext3 loop mounts into it, I got a OOPS - very likely as the container reached its size limit: Note this is factory - 2.6.38-rc8 [ 2852.318574] EXT4-fs (loop0): delayed block allocation failed for inode 12 at logical offset 186368 with max blocks 2048 with error -5 [ 2852.318579] EXT4-fs (loop0): This should not happen!! Data will be lost [ 2852.318580] [ 2852.320164] EXT4-fs (loop0): delayed block allocation failed for inode 12 at logical offset 186368 with max blocks 2048 with error -5 [ 2852.320168] EXT4-fs (loop0): This should not happen!! Data will be lost [ 2852.320169] [ 2852.320209] ------------[ cut here ]------------ [ 2852.320246] kernel BUG at /usr/src/packages/BUILD/kernel-desktop-2.6.38/linux-2.6.37/fs/ext4/inode.c:2189! [ 2852.320301] invalid opcode: 0000 [#1] PREEMPT SMP [ 2852.320340] last sysfs file: /sys/devices/LNXSYSTM:00/device:00/PNP0A08:00/device:01/PNP0C09:00/PNP0C0A:00/power_supply/BAT0/energy_full [ 2852.320409] CPU 0 [ 2852.320424] Modules linked in: loop fuse tun rfcomm sco bnep l2cap binfmt_misc af_packet cpufreq_conservative cpufreq_userspace cpufreq_powersave microcode acpi_cpufreq mperf arc4 ecb iwlagn snd_hda_codec_conexant iwlcore snd_hda_intel mac80211 snd_hda_codec snd_hwdep cfg80211 snd_pcm kvm_intel kvm e1000e iTCO_wdt snd_timer thinkpad_acpi dm_mod btusb snd_page_alloc pcspkr bluetooth iTCO_vendor_support sg rfkill i2c_i801 snd soundcore battery wmi ac autofs4 ext4 jbd2 crc16 i915 drm_kms_helper drm i2c_algo_bit button video fan processor thermal thermal_sys [ 2852.320871] [ 2852.320883] Pid: 4414, comm: flush-7:0 Not tainted 2.6.38-rc7-2-desktop #1 LENOVO 7470BN2/7470BN2 [ 2852.320947] RIP: 0010:[<ffffffffa014a572>] [<ffffffffa014a572>] ext4_da_block_invalidatepages+0x122/0x130 [ext4] [ 2852.321007] RSP: 0018:ffff88002f487890 EFLAGS: 00010246 [ 2852.321007] RAX: 0020000000000024 RBX: 000000000002dfff RCX: 0000000000000000 [ 2852.321007] RDX: 000000000000000e RSI: 0000000000000001 RDI: ffff88002f756740 [ 2852.321007] RBP: ffff88002f4878a0 R08: 0000000000000001 R09: ffffea00012e61f8 [ 2852.321007] R10: 0000000000000001 R11: 0000000000000001 R12: ffff88007355e618 [ 2852.321007] R13: ffffea00017e8060 R14: ffff88002f487980 R15: ffff88002f487a88 [ 2852.321007] FS: 0000000000000000(0000) GS:ffff880079200000(0000) knlGS:0000000000000000 [ 2852.321007] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2852.321007] CR2: 00007f77ca9d24b0 CR3: 0000000001a03000 CR4: 00000000000406f0 [ 2852.321007] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2852.321007] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2852.321007] Process flush-7:0 (pid: 4414, threadinfo ffff88002f486000, task ffff88002f756740) [ 2852.321007] Stack: [ 2852.321007] 0000000000000020 0000000e2f4878f0 000000000000000e ffffea0000be1a68 [ 2852.321007] ffffea00017e8060 ffffea00017e80d0 ffffea000176d400 ffffea000176d470 [ 2852.321007] ffffea000079bb20 ffffea000079bb90 ffffea00003d1a60 ffffea00003d1ad0 [ 2852.321007] Call Trace: [ 2852.321007] [<ffffffffa0151050>] mpage_da_map_and_submit+0x270/0x460 [ext4] [ 2852.321007] [<ffffffffa0151412>] __mpage_da_writepage+0xe2/0x180 [ext4] [ 2852.321007] [<ffffffffa015162f>] write_cache_pages_da+0x17f/0x290 [ext4] [ 2852.321007] [<ffffffffa0151acd>] ext4_da_writepages+0x38d/0x700 [ext4] [ 2852.321007] [<ffffffff8117cf65>] writeback_single_inode+0x95/0x230 [ 2852.321007] [<ffffffff8117d325>] generic_writeback_sb_inodes+0xd5/0x170 [ 2852.321007] [<ffffffff8117d4b3>] __writeback_inodes_sb+0xf3/0x100 [ 2852.321007] [<ffffffff8117e16d>] wb_writeback+0x15d/0x3f0 [ 2852.321007] [<ffffffff8117eb22>] wb_do_writeback+0xc2/0x280 [ 2852.321007] [<ffffffff8117ed94>] bdi_writeback_thread+0xb4/0x320 [ 2852.321007] [<ffffffff8107bbc6>] kthread+0x96/0xa0 [ 2852.321007] [<ffffffff81003d04>] kernel_thread_helper+0x4/0x10 [ 2852.321007] Code: 49 83 c5 01 85 f6 74 08 48 89 ef e8 09 be fb e0 4c 39 eb 0f 83 38 ff ff ff 48 81 c4 98 00 00 00 5b 5d 41 5c 41 5d 41 5e 41 5f c3 <0f> 0b 0f 0b 66 2e 0f 1f 84 00 00 00 00 00 41 56 41 55 41 54 4d [ 2852.321007] RIP [<ffffffffa014a572>] ext4_da_block_invalidatepages+0x122/0x130 [ext4] [ 2852.321007] RSP <ffff88002f487890> [ 2852.338189] ---[ end trace ef6f03d937c1abd8 ]--- -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c1 --- Comment #1 from Jan Kara <jack@novell.com> 2011-03-15 23:44:34 UTC --- Hmm, it's interesting that the returned error is EIO (not ENOSPC). Aren't there other errors before this in the log? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c2 --- Comment #2 from Stephan Kulow <coolo@novell.com> 2011-03-16 10:02:41 CET --- EIO instead of ENOSPC might be a clicfs/fuse limitation and yes, there are additional logs: Mar 15 20:21:22 xanthippe kernel: [ 2852.190109] Buffer I/O error on device loop0, logical block 206942 Mar 15 20:21:22 xanthippe kernel: [ 2852.190112] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190119] Buffer I/O error on device loop0, logical block 206943 Mar 15 20:21:22 xanthippe kernel: [ 2852.190121] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190127] Buffer I/O error on device loop0, logical block 206944 Mar 15 20:21:22 xanthippe kernel: [ 2852.190129] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190135] Buffer I/O error on device loop0, logical block 206945 Mar 15 20:21:22 xanthippe kernel: [ 2852.190137] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190143] Buffer I/O error on device loop0, logical block 206946 Mar 15 20:21:22 xanthippe kernel: [ 2852.190145] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190151] Buffer I/O error on device loop0, logical block 206947 Mar 15 20:21:22 xanthippe kernel: [ 2852.190153] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190158] Buffer I/O error on device loop0, logical block 206948 Mar 15 20:21:22 xanthippe kernel: [ 2852.190160] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190167] Buffer I/O error on device loop0, logical block 206949 Mar 15 20:21:22 xanthippe kernel: [ 2852.190168] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190174] Buffer I/O error on device loop0, logical block 206950 Mar 15 20:21:22 xanthippe kernel: [ 2852.190176] lost page write due to I/O error on loop0 Mar 15 20:21:22 xanthippe kernel: [ 2852.190183] Buffer I/O error on device loop0, logical block 206951 Mar 15 20:21:22 xanthippe kernel: [ 2852.190185] lost page write due to I/O error on loop0 I really did not nothing fancy, I mounted an ext3 on clicfs/loop and dd if=/dev/zero into the mount point. My target to crash was really clicfs not the kernel :) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c3 --- Comment #3 from Jan Kara <jack@novell.com> 2011-03-16 15:04:02 UTC --- Ok, thanks. So that explains the "delayed allocation failed" messages. Now I just have to find out what is wrong in the error recovery path. BTW: You keep speaking about ext3 but the filesystem is ext4. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c Jan Kara <jack@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c4 --- Comment #4 from Stephan Kulow <coolo@novell.com> 2011-03-16 16:32:25 CET --- yeah, but the filename is traddionally fsdata.ext3 :) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c5 --- Comment #5 from Jan Kara <jack@novell.com> 2011-03-16 20:57:16 UTC --- I've managed to reproduce the problem and the following patches fix the problem for me. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c6 --- Comment #6 from Jan Kara <jack@novell.com> 2011-03-16 20:58:12 UTC --- Created an attachment (id=419804) --> (http://bugzilla.novell.com/attachment.cgi?id=419804) [PATCH 1/2] Patch fixing BUG_ON in ext4_da_invalidatepages() -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c7 --- Comment #7 from Jan Kara <jack@novell.com> 2011-03-16 20:58:51 UTC --- Created an attachment (id=419806) --> (http://bugzilla.novell.com/attachment.cgi?id=419806) [PATCH 2/2] Patch fixing deadlocks in presence of IO errors -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c8 --- Comment #8 from Jan Kara <jack@novell.com> 2011-03-16 21:03:17 UTC --- I've submitted these patches upstream and if they get accepted I can push them to openSUSE 11.4 kernel. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c9 Jan Kara <jack@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED --- Comment #9 from Jan Kara <jack@novell.com> 2011-04-19 17:00:58 UTC --- OK, actually both issues have already been worked on upstream in parallel and had existing patches (e0fd9b90, c7f5938adce6727b9d17785f289c1146bd88d678). They are upstream now so I've used these commits and pushed them to openSUSE 11.4. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c10 --- Comment #10 from Bernhard Wiedemann <bwiedemann@novell.com> 2011-04-25 17:00:28 CEST --- This is an autogenerated message for OBS integration: This bug (679898) was mentioned in https://build.opensuse.org/request/show/68313 https://build.opensuse.org/request/show/68316 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c11 --- Comment #11 from Bernhard Wiedemann <bwiedemann@novell.com> 2011-04-28 13:56:29 CEST --- This is an autogenerated message for OBS integration: This bug (679898) was mentioned in https://build.opensuse.org/request/show/68369 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c12 Swamp Workflow Management <swamp@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status Whiteboard| |maint:released:11.4:40457 --- Comment #12 from Swamp Workflow Management <swamp@suse.com> 2011-04-29 09:38:22 UTC --- Update released for: kernel-debug, kernel-debug-base, kernel-debug-base-debuginfo, kernel-debug-debuginfo, kernel-debug-debugsource, kernel-debug-devel, kernel-debug-devel-debuginfo, kernel-default, kernel-default-base, kernel-default-base-debuginfo, kernel-default-debuginfo, kernel-default-debugsource, kernel-default-devel, kernel-default-devel-debuginfo, kernel-desktop, kernel-desktop-base, kernel-desktop-base-debuginfo, kernel-desktop-debuginfo, kernel-desktop-debugsource, kernel-desktop-devel, kernel-desktop-devel-debuginfo, kernel-devel, kernel-docs, kernel-ec2, kernel-ec2-base, kernel-ec2-base-debuginfo, kernel-ec2-debuginfo, kernel-ec2-debugsource, kernel-ec2-devel, kernel-ec2-devel-debuginfo, kernel-ec2-extra, kernel-ec2-extra-debuginfo, kernel-pae, kernel-pae-base, kernel-pae-base-debuginfo, kernel-pae-debuginfo, kernel-pae-debugsource, kernel-pae-devel, kernel-pae-devel-debuginfo, kernel-source, kernel-source-vanilla, kernel-syms, kernel-trace, kernel-trace-base, kernel-trace-base-debuginfo, kernel-trace-debuginfo, kernel-trace-debugsource, kernel-trace-devel, kernel-trace-devel-debuginfo, kernel-vanilla, kernel-vanilla-base, kernel-vanilla-base-debuginfo, kernel-vanilla-debuginfo, kernel-vanilla-debugsource, kernel-vanilla-devel, kernel-vanilla-devel-debuginfo, kernel-vmi, kernel-vmi-base, kernel-vmi-base-debuginfo, kernel-vmi-debuginfo, kernel-vmi-debugsource, kernel-vmi-devel, kernel-vmi-devel-debuginfo, kernel-xen, kernel-xen-base, kernel-xen-base-debuginfo, kernel-xen-debuginfo, kernel-xen-debugsource, kernel-xen-devel, kernel-xen-devel-debuginfo, preload-kmp-default, preload-kmp-desktop Products: openSUSE 11.4 (debug, i586, x86_64) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=679898 https://bugzilla.novell.com/show_bug.cgi?id=679898#c13 Swamp Workflow Management <swamp@suse.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status Whiteboard|maint:released:11.4:40457 |maint:released:11.4:40457 | |maint:released:11.3:42359 --- Comment #13 from Swamp Workflow Management <swamp@suse.de> 2011-08-02 07:59:43 UTC --- Update released for: kernel-debug, kernel-debug-base, kernel-debug-base-debuginfo, kernel-debug-debuginfo, kernel-debug-debugsource, kernel-debug-devel, kernel-debug-devel-debuginfo, kernel-default, kernel-default-base, kernel-default-base-debuginfo, kernel-default-debuginfo, kernel-default-debugsource, kernel-default-devel, kernel-default-devel-debuginfo, kernel-desktop, kernel-desktop-base, kernel-desktop-base-debuginfo, kernel-desktop-debuginfo, kernel-desktop-debugsource, kernel-desktop-devel, kernel-desktop-devel-debuginfo, kernel-devel, kernel-ec2, kernel-ec2-base, kernel-ec2-base-debuginfo, kernel-ec2-debuginfo, kernel-ec2-debugsource, kernel-ec2-devel, kernel-ec2-devel-debuginfo, kernel-ec2-extra, kernel-ec2-extra-debuginfo, kernel-pae, kernel-pae-base, kernel-pae-base-debuginfo, kernel-pae-debuginfo, kernel-pae-debugsource, kernel-pae-devel, kernel-pae-devel-debuginfo, kernel-source, kernel-source-vanilla, kernel-syms, kernel-trace, kernel-trace-base, kernel-trace-base-debuginfo, kernel-trace-debuginfo, kernel-trace-debugsource, kernel-trace-devel, kernel-trace-devel-debuginfo, kernel-vanilla, kernel-vanilla-base, kernel-vanilla-base-debuginfo, kernel-vanilla-debuginfo, kernel-vanilla-debugsource, kernel-vanilla-devel, kernel-vanilla-devel-debuginfo, kernel-vmi, kernel-vmi-base, kernel-vmi-base-debuginfo, kernel-vmi-debuginfo, kernel-vmi-debugsource, kernel-vmi-devel, kernel-vmi-devel-debuginfo, kernel-xen, kernel-xen-base, kernel-xen-base-debuginfo, kernel-xen-debuginfo, kernel-xen-debugsource, kernel-xen-devel, kernel-xen-devel-debuginfo, preload-kmp-default, preload-kmp-desktop Products: openSUSE 11.3 (debug, i586, x86_64) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com