http://bugzilla.suse.com/show_bug.cgi?id=1131931
Bug ID: 1131931 Summary: OCFS2: Defragmentation error: No space left on device Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: High Availability Assignee: ha-bugs@suse.de Reporter: weikai.wang@suse.com QA Contact: qa-bugs@suse.de Found By: --- Blocker: ---
Created attachment 802412 --> http://bugzilla.suse.com/attachment.cgi?id=802412&action=edit this script is used to generate disk framentation
when I used the defragfs.ocfs2 to clean up disk fragmentation that generated by the attached shell script.
you could execute the scrip by command "./defragfs_test.sh -d DEVICE -m MOUNT_POINT -n CLUSTER_NAME -s pcmk -o /tmp"
You should use the device on your cluster replace DEVICE, use the same way replace MOUNT_POINT and CLUSTER_NAME.
Then use the command defragfs.ocfs2 MOUNT_POINT. And if no accident, you will see some message like this: “ defragfs.ocfs2 1.8.5 [1/201]/mnt/ocfs2/tmp_file:Success [ERROR]Move extent failed:"/mnt/ocfs2/test_from_dd1" - No space left on device [2/201]/mnt/ocfs2/test_from_dd1:Failed [ERROR]Move extent failed:"/mnt/ocfs2/test_from_dd2" - No space left on device [3/201]/mnt/ocfs2/test_from_dd2:Failed [ERROR]Move extent failed:"/mnt/ocfs2/test_from_dd3" - No space left on device ”
but if you execute defragfs.ocfs2 MOUNT_POINT again, the error message will not show again. and all the defragmentation will pass.
please make sure your device's capacity is more tha 1G.
Some times this error can make mount, unmount, and mkfs crushed.
https://bugzilla.suse.com/show_bug.cgi?id=1131931
heming zhao heming.zhao@suse.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Priority|P4 - Low |P3 - Medium CC| |heming.zhao@suse.com, | |zzhou@suse.com Assignee|ghe@suse.com |heming.zhao@suse.com
https://bugzilla.suse.com/show_bug.cgi?id=1131931 https://bugzilla.suse.com/show_bug.cgi?id=1131931#c1
--- Comment #1 from heming zhao heming.zhao@suse.com --- There are some comments for the c#0 attached test script.
1. It's better to comment out line 264 "numblks=10485760". If not, we can't use whole disk size.
2. I use kvm-qemu for test: 4 vcpu, 5G mem, 10GB raw file under virtio mode.
3. it's better to add "exit 0" after line 302. the code area for "Test option '-T mail'" is not related with this bug.
4. the attachment script will stop before running defragfs.ocfs2. For triggering the issue, there are two method: - run the defragfs by manual. - comment line 298 "exit 100"
5. how to run test script - ./defragfs_test.sh -d /dev/vdf -m /mnt -n ocfstst -s pcmk -o /tmp - ./defragfs_test.sh -d /dev/vdf -m /mnt -n ocfstst -s pcmk -o /tmp -b 1024 -c 4096 - ./defragfs_test.sh -d /dev/vdf -m /mnt -n ocfstst -s pcmk -o /tmp -b 1024 -c 8192
https://bugzilla.suse.com/show_bug.cgi?id=1131931 https://bugzilla.suse.com/show_bug.cgi?id=1131931#c2
--- Comment #2 from heming zhao heming.zhao@suse.com --- ocfs2-tools related code: do_file_defrag + struct ocfs2_move_extents me = { | .me_start = 0, | .me_len = buf->st_size, | .me_flags = OCFS2_MOVE_EXT_FL_AUTO_DEFRAG | }; | + fd = open64(file, O_RDWR, 0400); | + ioctl(fd, OCFS2_IOC_MOVE_EXT, &me) //return errno: ENOSPC //errno ENOSPC response with "No space left on device".
kernel related code: ocfs2_defrag_extent() { ... ... ret = __ocfs2_claim_clusters(handle, context->data_ac, 1, *len, &new_phys_cpos, &new_len) ... ... if (new_len != *len) { if (!partial) { context->range->me_flags &= ~OCFS2_MOVE_EXT_FL_COMPLETE; ret = -ENOSPC; need_free = 1; goto out_commit; } } ... ... }
------------ This bug has a workround (without remount). when defragfs reports -ENNOSPC, use dd to create some new files to consume la-window. After ocfs2 switch to another la-window, re-run defrag may succeed.
``` tw-xfs1:~ # defragfs.ocfs2 /mnt/test_from_dd1 defragfs.ocfs2 1.8.7 [ERROR]Move extent failed:"/mnt/test_from_dd1" - No space left on device [ERROR]"/mnt/test_from_dd1":No space left on device tw-xfs1:~ #
tw-xfs1:~ # dd if=/dev/zero of=/mnt/zhm.1 bs=4k count=200 200+0 records in 200+0 records out 819200 bytes (819 kB, 800 KiB) copied, 0.0166738 s, 49.1 MB/s
tw-xfs1:~ # dd if=/dev/zero of=/mnt/zhm.2 bs=1M count=20 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.118353 s, 177 MB/s
tw-xfs1:~ # defragfs.ocfs2 /mnt/test_from_dd1 defragfs.ocfs2 1.8.7 /mnt/test_from_dd1: Succeeded ```
-
bugzilla_noreply＠novell.com -
bugzilla_noreply＠suse.com