[opensuse-factory] A look on RPM compression in openSUSE
Compression performance plots are often done with something like the Silesia corpus. Linux distributions have rather different proportions of file types, I think. They have a lot of machine code, and even more data files, and probably not so much text and images. Since our data set is also 2.8 orders of magnitude bigger, rerunning a compression shootout will give more detail. So I did just that. http://paste.opensuse.org/15790105 http://inai.de/files/openSUSE-compression.ods (My measurements included just *.x86_64.rpm + *.noarch.rpm.) The takeaway from that is: * xz outperforms zstd in the regions that xz caters to. But overall, xz forms the far end of the "law of diminishing returns". * Moving openSUSE from xz-5 to xz-2 saves 50% of time for an investment of just 3.2 GB of space. Or, moving to zstd-7, saving 85% for ~6.1 GB. Other observations: * There are steps in compressor behavior, and that penalizes a lot of levels, leaving only a few sensible ones: zstd-2-3-7-12, xz-1-2-3-4-5-9 (disregarding memory use, which is another factor). * Some of our packages are too fat. kicad-packages takes longer to compress than the entire remaining distro at zstd-19 with 16x-parallelism. In other words, a sufficiently parallelized system may have to wait just for that one to complete. (* "Trend lines" in LibreOffice Calc are quite useless sometimes, as it does not appear to calculate a constant offset portion for exp/pow fittings.) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (10)
-
Bernhard M. Wiedemann
-
cagsm
-
Carlos E. R.
-
Dave Plater
-
H.Merijn Brand
-
Jan Engelhardt
-
Joachim Wagner
-
Neal Gompa
-
Stefan Brüns
-
Stephan Kulow