-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2013-10-27 at 12:02 +0100, Carlos E. R. wrote:
Here I repeat the test on real hardware. I have used a script with variations as I find out things; when the difference is significative, I repeated the test. I'll post the script below. I did this on my same main machine, but on a spare 1TB HD. I reuse the same test partition, creating a different filesystem in the script prior to testing it.
...
I'll put on another email the results of testing a larger partition with fewer files.
For this part of the test, I enlarged the partition to 120 GiB. The script was the same, but the goal now was to time writing of files of different sizes on the different types of filesystem. Same directory (intentional). On files bigger than 1 MB, the limiting factor is raw disk write speed. Smaller, and the limit is CPU and code (ie, the disk was not at max w. speed). Partition of 120 GiB 10000 files of 10 MB (not 10 MiB). Disk busy at up to 155MB/s - hardware is the limit. time (real/user/sys) | listing time | rm files time | used free % | (real/user/sys) | (real/user/sys) | space space reiserfs 12m39.628s/0m4.775s/3m10.519s | 0m0.309s/0m0.032s/0m0.005s | 0m13.438s/0m0.032s/0m0.005s | 94G 27G 78% ext4 12m15.974s/0m3.833s/1m43.232s | 0m0.077s/0m0.043s/0m0.009s | 0m6.394s/0m0.004s/0m1.490s | 94G 19G 84% ext3 15m20.931s/0m3.756s/2m40.206s | 0m0.156s/0m0.045s/0m0.007s | 0m9.677s/0m0.005s/0m3.006s | 94G 19G 84% xfs 11m11.731s/0m4.737s/1m26.311s | 0m0.155s/0m0.024s/0m0.005s | 0m9.480s/0m0.011s/0m1.584s | 94G 27G 78% btrfs 10m37.071s/0m4.837s/1m11.542s | 0m0.376s/0m0.027s/0m0.004s | 0m9.755s/0m0.004s/0m2.691s | 92G 27G 78% 10000 files of 1 MB (not 1 MiB). time (real/user/sys) | listing time | rm files time | used free % | (real/user/sys) | (real/user/sys) | space space reiserfs 1m 6.632s/0m2.175s/0m20.469s | 0m0.171s/0m0.026s/0m0.003s | 0m9.939s/0m0.004s/0m2.638s | 9.4G 111G 8% ext4 1m 5.183s/0m1.362s/0m11.854s | 0m0.200s/0m0.043s/0m0.007s | 0m7.921s/0m0.010s/0m1.376s | 9.6G 103G 9% ext3 1m31.627s/0m2.897s/0m19.668s | 0m0.132s/0m0.043s/0m0.007s | 0m7.735s/0m0.006s/0m1.600s | 9.6G 103G 9% xfs 0m57.785s/0m2.869s/0m9.561s | 0m0.150s/0m0.025s/0m0.004s | 0m6.516s/0m0.009s/0m1.428s | 9.4G 111G 8% btrfs 0m56.036s/0m3.342s/0m10.203s | 0m0.163s/0m0.025s/0m0.006s | 0m1.913s/0m0.008s/0m1.853s | 8.1G 110G 7% 10000 files of 100 KB . time (real/user/sys) | listing time | rm files time | used free % | (real/user/sys) | (real/user/sys) | space space reiserfs 0m12.491s/0m3.102s/0m8.400s | 0m0.031s/0m0.027s/0m0.005s | 0m0.537s/0m0.006s/0m0.528s | 1011M 120G 1% No more than 20 MB/s ext4 0m11.733s/0m0.786s/0m1.725s | 0m0.050s/0m0.045s/0m0.005s | 0m0.361s/0m0.005s/0m0.350s | 1.2G 111G 2% No more than 20 MB/s ext3 0m28.568s/0m3.134s/0m8.338s | 0m0.051s/0m0.046s/0m0.005s | 0m0.470s/0m0.004s/0m0.459s | 1.2G 111G 2% No more than 50 MB/s xfs 0m11.061s/0m1.032s/0m2.495s | 0m0.030s/0m0.026s/0m0.005s | 0m0.891s/0m0.004s/0m0.441s | 1.1G 119G 1% negligible or delayed btrfs 0m11.006s/0m0.846s/0m1.735s | 0m0.030s/0m0.025s/0m0.005s | 0m0.600s/0m0.006s/0m0.583s | 392M 118G 1% negligible or delayed 10000 files of 10 KB . time (real/user/sys) | listing time | rm files time | used free % | (real/user/sys) | (real/user/sys) | space space reiserfs 0m10.244s/0m0.667s/0m1.273s | 0m0.029s/0m0.026s/0m0.003s | 0m0.619s/0m0.008s/0m0.485s | 151M 120G 1% negligible + peak of 60 ext4 0m10.137s/0m0.766s/0m1.400s | 0m0.050s/0m0.044s/0m0.007s | 0m0.152s/0m0.004s/0m0.144s | 345M 112G 1% ext3 0m11.788s/0m0.676s/0m1.361s | 0m0.049s/0m0.043s/0m0.007s | 0m0.181s/0m0.001s/0m0.175s | 306M 112G 1% xfs 0m10.149s/0m0.559s/0m1.237s | 0m0.028s/0m0.024s/0m0.005s | 0m0.261s/0m0.007s/0m0.253s | 348M 120G 1% btrfs 0m10.143s/0m0.732s/0m1.217s | 0m0.030s/0m0.023s/0m0.007s | 0m0.362s/0m0.010s/0m0.348s | 68M 118G 1% 10000 files of 1 KB time (real/user/sys) | listing time | rm files time | used free % | (real/user/sys) | (real/user/sys) | space space reiserfs 0m10.112s/0m0.653s/0m1.317s | 0m0.030s/0m0.026s/0m0.004s | 0m0.294s/0m0.001s/0m0.290s | 20M 118G 1% ext4 0m10.135s/0m0.812s/0m1.471s | 0m0.049s/0m0.047s/0m0.002s | 0m0.125s/0m0.003s/0m0.119s | 266M 112G 1% ext3 0m10.326s/0m0.769s/0m1.536s | 0m0.052s/0m0.043s/0m0.009s | 0m0.154s/0m0.001s/0m0.150s | 228M 112G 1% xfs 0m10.209s/0m0.634s/0m1.279s | 0m0.029s/0m0.027s/0m0.002s | 0m0.240s/0m0.008s/0m0.231s | 270M 120G 1% btrfs 0m10.155s/0m0.761s/0m1.447s | 0m0.030s/0m0.022s/0m0.008s | 0m0.289s/0m0.009s/0m0.277s | 20M 118G 1% 10000 files of 100 B time (real/user/sys) | listing time | rm files time | used free % | (real/user/sys) | (real/user/sys) | space space reiserfs 0m10.171s/0m0.795s/0m1.445s | 0m0.030s/0m0.027s/0m0.003s | 0m0.282s/0m0.002s/0m0.277s | 9.9M 118G 1% ext4 0m10.087s/0m0.779s/0m1.387s | 0m0.050s/0m0.045s/0m0.006s | 0m0.126s/0m0.004s/0m0.121s | 266M 112G 1% ext3 0m10.266s/0m0.732s/0m1.415s | 0m0.050s/0m0.040s/0m0.010s | 0m0.156s/0m0.005s/0m0.148s | 228M 112G 1 xfs 0m10.193s/0m0.680s/0m1.332s | 0m0.028s/0m0.023s/0m0.005s | 0m0.235s/0m0.003s/0m0.231s | 270M 120G 1% btrfs 0m10.195s/0m0.689s/0m1.394s | 0m0.030s/0m0.026s/0m0.004s | 0m0.280s/0m0.005s/0m0.273s | 9.6M 118G 1% Here I thought that it would be interesting to add a "sync" in the script and find out the differences. The results were curious: the operation was faster, sometimes significantly so, if we stopped to do a sync. I altered the script to insert a sync every thousand writes, and a few more (timed). It does a double test: with and without sync. Notice that without sync the times should be about the same as the previous round... but that was not so. Curious! 10000 files of 10 MB (not 10 MiB). time (real/user/sys) reiserfs 10m51.732s/0m4.660s/1m12.536s :-? why so fast now? ext4 12m26.604s/0m3.790s/0m3.790s One initial test, surprisingly fast. Verify script and repeat test. mkfs | cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m 2.965s | 13m1.503s/0m4.594s/3m10.574s | 14m2.930s/0m4.761s/3m35.965s ext4 0m 1.853s | 12m28.622s/0m3.748s/1m44.049s | 13m5.449s/0m4.612s/2m10.158s ext3 0m18.785s | 15m30.909s/0m3.751s/2m41.437s | 14m44.870s/0m4.696s/2m49.492s xfs 0m 0.533s | 11m30.054s/0m4.661s/1m27.444s | 31m47.276s/0m4.671s/1m53.780s btrfs 0m 0.018s | 10m54.451s/0m4.551s/1m12.906s | 11m6.555s/0m6.311s/1m37.479s | 0m9.513s # Created batch mode script, can run full night no intervention. Here I modified the script to the final form. I got tired of starting the script many times, so I decided to make a script that would run unattended the entire remaining tests. On this section is when I had crashes. Flow: 10000 file copy operations, with a sync every 1000. Time that. 10000 file copy operations, timed. one sync operation at the end, timed. Repeat for all filesystem types (reformatting the same partition), then repeat for a different file size. (log_tests.log) 10000 files of 10 MB (not 10 MiB). mkfs | cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m 2.897s | 13m2.899s/0m4.682s/3m10.229s | 0m0.104s/0m5.103s/3m35.783s | 0m9.404s ext4 0m 1.652s | 12m32.215s/0m6.298s/1m48.388s | 13m 8.495s/0m3.760s/2m3.915s | 0m9.744s ext3 0m18.623s | 15m36.742s/0m3.735s/2m40.927s | 15m15.403s/0m3.535s/2m45.379s | 0m2.636s xfs 0m 0.653s | 11m24.977s/0m4.969s/1m28.766s | 30m59.347s/0m3.964s/1m53.809s | 0m11.319s btrfs 0m 0.076s | 11m13.964s/0m5.135s/1m15.799s | 11m11.298s/0m5.208s/1m38.627s | 0m1.867s Notice the longer time XFS takes on the final sync. XFS caches metadata internally, I understand. Also notice the much longer time the operation takes _without_ periodic syncs, about 3 times more! The only explanation I can think is that the cache filled. (log_tests.2.log) 10000 files of 1 MB (not 1 MiB). mkfs | cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m 3.405s | 1m37.456s/0m1.862s/0m20.887s | 1m17.111s/0m3.297s/0m3.297s | 0m4.956s ext4 0m 1.457s | 1m32.267s/0m3.553s/0m15.194s | 1m0.911s/0m3.537s/0m18.777s | 0m8.439s ext3 0m18.491s | 1m39.369s/0m2.648s/0m19.839s | 1m36.755s/0m2.471s/0m20.403s| 0m8.131s xfs 0m 0.549s | 1m22.179s/0m1.300s/0m11.512s | 1m7.481s/0m1.344s/0m13.137s | 0m12.275s btrfs 0m 0.062s | 1m21.574s/0m1.874s/0m11.104s | 1m7.150s/0m2.163s/0m11.553s | 0m6.719s 10000 files of 100 KB. cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m20.739s/0m2.924s/0m8.877s | 0m13.576s/0m2.891s/0m8.807s | 0m2.334s ext4 0m20.953s/0m2.696s/0m7.068s | 0m14.385s/0m2.986s/0m8.717s | 1m57.328s * ext3 0m43.545s/0m3.001s/0m8.815s | 0m16.560s/0m2.942s/0m8.786s | 0m13.453s xfs 0m21.382s/0m2.912s/0m7.603s | 0m17.859s/0m3.003s/0m8.767s | 0m4.363s btrfs 0m21.254s/0m1.532s/0m4.542s | 0m11.828s/0m2.920s/0m8.103s | 0m6.200s Notice here the 2 minute delay that ext4 took to sync after the test cycle without syncs inserted. Possibly because it had cached and delayed a lot of operations. On the whole, with syncs inserted it is slower, which is expected. But not so with XFS, btrfs and reiserfs, which is, well, surprising. 10000 files of 10 KB. cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m17.250s/0m0.545s/0m1.560s | 0m11.078s/0m0.571s/0m1.691s | 0m0.526s ext4 0m17.346s/0m0.515s/0m1.625s | 0m10.825s/0m0.634s/0m1.841s | 1m53.964s * ext3 0m17.282s/0m0.537s/0m1.688s | 0m10.546s/0m0.624s/0m1.711s | 0m1.020s xfs 0m14.766s/0m0.552s/0m1.635s | 0m11.943s/0m1.096s/0m3.143s | 0m0.517s btrfs 0m15.313s/0m0.582s/0m1.841s | 0m10.771s/0m0.717s/0m2.027s | 0m1.658s kernel crash happened around here. I think that btrfs was umountable. Corruption happened around here. umount of btrfs must have failed, because: reiserfs_create: could not open /dev/disk/by-id/wwn-0x5000c500613c92d5-part6: Device or resource busy 2013-10-25T23:14:22.755216+02:00 mounted ext4 Crash happened after 23:21:24, while running 100B btrfs test. There are syslog entries at 23:42:12.756218 about creating btrfs. Probably next trial round failed. Repeat test from the 10 KB cycle. (log_tests.3.log) 10000 files of 10 KB. cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m14.895s/0m0.705s/0m1.268s | 0m11.043s/0m1.652s/0m3.153s | 0m0.319s ext4 0m16.641s/0m1.357s/0m1.528s | 0m10.640s/0m1.333s/0m2.775s | 0m0.116s ext3 0m14.608s/0m0.580s/0m1.362s | 0m11.303s/0m0.572s/0m1.373s | 0m0.554s xfs 0m12.927s/0m0.584s/0m1.421s | 0m11.263s/0m0.581s/0m1.622s | 0m0.670s btrfs 0m13.590s/0m0.622s/0m1.492s | 0m10.470s/0m0.617s/0m1.738s | 0m1.538s Now, notice the lack of the 2 minute delay on the extra sync, for the ext4 test. I have no explanation for this; perhaps as I had to reboot, the system cache was fully emptied. Maybe, to do these test properly, the system would have to be rebooted on each test :-? 10000 files of 1 KB (not 10 MiB). cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m19.629s/0m0.559s/0m1.420s | 0m10.434s/0m0.565s/0m1.457s | 0m0.248s ext4 0m16.324s/0m0.444s/0m1.455s | 0m10.400s/0m0.554s/0m1.562s | 0m0.143s ext3 0m15.161s/0m0.483s/0m1.379s | 0m10.079s/0m0.482s/0m1.228s | 0m0.322s xfs 0m13.030s/0m0.575s/0m1.535s | 0m11.279s/0m0.499s/0m1.565s | 0m0.616s btrfs 0m13.153s/0m0.587s/0m1.686s | 0m10.371s/0m0.606s/0m1.578s | 0m2.363s 10000 files of 100 B (not 10 MiB). cp with sync every 1000 files | cp without sync | extra sync reiserfs 0m17.863s/0m0.555s/0m1.460s | 0m10.433s/0m0.571s/0m1.486s | 0m1.230s and kernel crash in the ext4 test. Repeat. (log_tests.4.log) reiserfs 0m20.510s/0m0.694s/0m1.386s | 0m10.512s/0m0.951s/0m1.420s | 0m0.991s ext4 0m25.058s/0m1.327s/0m1.491s | 0m10.539s/0m0.909s/0m1.565s | 0m0.125s ext3 0m15.586s/0m0.420s/0m1.497s | 0m10.158s/0m0.440s/0m1.471s | 0m0.529s xfs 0m13.545s/0m0.555s/0m1.457s | 0m11.247s/0m0.512s/0m1.534s | 0m0.600s btrfs 0m12.838s/0m0.557s/0m1.463s | 0m10.240s/0m0.519s/0m1.398s | 0m0.455s The only conclusion I draw, for my own consumption, is that I will not use btrfs, nor in 12.3, nor in 13.1. I managed to reliable crash it several times... And if I managed to crash it with a simple, home grown stress test, what other traps may be lurking in it yet to be found? Another conclusion (that makes 2) is that reiserfs is no longer the fastest filesystem; and that despite being more than 10 years old, with only minimal maintenance, it can hold its own against newer or better maintained code. This tells a lot in its favour. With more love, it might surpass them all. Pity. For the curious: http://en.wikipedia.org/wiki/Reiserfs - -- Cheers, Carlos E. R. (from 12.3 x86_64 "Dartmouth" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlJs/EcACgkQtTMYHG2NR9Wd1QCdG09nkuXIIvAR9peZvUJy5FP7 6xMAnA4el22KA2Xx1r3TcAx3ByeV1wK3 =EJvt -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org