https://bugzilla.suse.com/show_bug.cgi?id=1199970 Jan Kara changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kernel-performance-bugs@sus | |e.de Assignee|kernel-bugs@opensuse.org |jack@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1199970 https://bugzilla.suse.com/show_bug.cgi?id=1199970#c2 --- Comment #2 from Jan Kara --- And marvin7 reaim-io-disk-ext4 regression: good-27b38686 bad-077d0c2c Hmean disk-1 3456.22 ( 4.15%) 3267.97 ( -1.53%) Hmean disk-25 219298.25 ( -1.17%) 198938.99 * -10.34%* Hmean disk-49 436201.78 ( -0.30%) 378865.98 * -13.40%* Hmean disk-73 600000.00 ( 0.82%) 504608.30 * -15.21%* Hmean disk-97 734848.49 * -4.29%* 621794.87 * -19.02%* Hmean disk-121 828767.12 ( -0.46%) 666055.05 * -20.00%* Hmean disk-145 877016.13 ( -2.82%) 744863.01 * -17.47%* Hmean disk-169 944134.08 ( -0.00%) 803486.53 * -14.90%* Hmean disk-193 900466.56 * -4.82%* 851470.59 * -10.00%* -- You are receiving this mail because: You are on the CC list for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1199970 https://bugzilla.suse.com/show_bug.cgi?id=1199970#c4 --- Comment #4 from Jan Kara --- OK, some updates after rather long time. The regression seems to be mostly triggered by reaim rapidly creating, fsyncing, and deleting small files. I was able to reproduce the regression using a simpler stress-unlink benchmark: stress-unlink -s -c 10000 -f 22528 16 0 /mnt Results with this benchmark are: AVG STDDEV mb_optimize_scan=0: 28.285800 0.156846 mb_optimize_scan=1: 30.896600 0.323324 -- You are receiving this mail because: You are on the CC list for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1199970 https://bugzilla.suse.com/show_bug.cgi?id=1199970#c6 --- Comment #6 from Jan Kara --- For explanation, this invocation of the benchmark spawns 16 processes, each process does create 22k file, fsync, unlink in a loop 10000 times. -- You are receiving this mail because: You are on the CC list for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1199970 https://bugzilla.suse.com/show_bug.cgi?id=1199970#c8 Jan Kara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |IN_PROGRESS --- Comment #8 from Jan Kara --- OK, I was looking more into why allocations get more spread among groups with mb_optimize_scan=1. Let me summarize here my current findings so that I don't forget while I'm on vacation next week :). The things get somewhat obscured by group preallocations because small allocations (below 64k which is our case) get allocated from those. Group preallocations are per-CPU and they get initialized like: mb_optimize_scan=0: 49 81 113 97 17 33 113 49 81 33 97 113 81 1 17 33 33 81 1 113 97 17 113 113 33 33 97 81 49 81 17 49 mb_optimize_scan=1: 127 126 126 125 126 127 125 126 127 124 123 124 122 122 121 120 119 118 117 116 115 116 114 113 111 110 109 108 107 106 105 104 104 So we can see groups from which "group preallocations" get allocated drift with mb_optimize_scan=1 while they keep jumping among same groups with mb_optimize_scan=0. This is likely because with mb_optimize_scan=0 we always start searching for free space in the goal group which is determined by the inode and inode's group is determined by the parent directory. So in that case we always start the search in the same group. With mb_optimize_scan=1 we always call ext4_mb_choose_next_group_cr0() to determine the first group to search. The drifting seems to be caused by each free space update (e.g. mb_mark_used()) calls mb_set_largest_free_order() which deletes group from bb_largest_free_order_node list, recomputes the order, and inserts it at the tail of the list again. Anyway this seems a bit suboptimal because inode - data block locality is desirable. Also given the algorithm of group selection all allocations of the same size will be contending on the same group. After I return from vacation, I'll take this upstream for discussion, whether the current behavior does not need tweaking. -- You are receiving this mail because: You are on the CC list for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1199970 https://bugzilla.suse.com/show_bug.cgi?id=1199970#c10 Jan Kara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|IN_PROGRESS |RESOLVED Resolution|--- |FIXED --- Comment #10 from Jan Kara --- Forgot to update this one. Patches to fix these problems were already merged upstream in September: 4fca50d440cc ("ext4: make mballoc try target group first even with mb_optimize_scan") 1940265ede66 ("ext4: avoid unnecessary spreading of allocations among groups") 613c5a85898d ("ext4: make directory inode spreading reflect flexbg size") a9f2a2931d0e ("ext4: use locality group preallocation for small closed files") 83e80a6e3543 ("ext4: use buckets for cr 1 block scan instead of rbtree") Closing the bug. -- You are receiving this mail because: You are on the CC list for the bug.