On 6/7/19 4:58 PM, Jan Engelhardt wrote:
These is this month's compression shootout.
News:
* Fedora announced intent to switch rpms to Zstd compression; the reported timings on the proposal paper https://fedoraproject.org/wiki/Changes/Switch_RPMs_to_zstd_compression suggest to me that the measurement was not done with due statistical care with regard to CPU thermal throttling characteristics.
* Earlier this year Ubuntu switched kernel and initramfs to lz4 https://lists.ubuntu.com/archives/ubuntu-devel/2018-March/040258.html
* In both of these instances, decompression time is the hot new topic. (Something the previous openSUSE report, https://lists.opensuse.org/opensuse-factory/2019-05/msg00344.html did not analyze)
Change in methods:
* I cut the testset to just x86_64 rpms (dropping *.noarch.rpm) to cut time. The proportion of executables over data should be higher now.
* Ran xz only from -1..-5; we all know its progression characteristics (-6..-9) from older tests.
* Added brotli, all levels
New results:
* http://paste.opensuse.org/view/97377621 - compression * http://paste.opensuse.org/view/80494192 - decompression * http://inai.de/files/openSUSE-compression-201906.ods - details
Interpretation:
* The ups and downs in the decompression times are read as jitter; they often complete in less than a minute.
* For a particular algorithm, decompression speed is unaffected by chosen compression level. This means the choice of level is only influenced by the desired compression-time characteristics.
* zstd decompression is decidedly faster than xz (~4.3x). Fedora is making sensible choices.
* lz4 decompression is 15% slower than zstd. Ubuntu is making suboptimal choices again.
* Brotli: only marginally better in the low-to-medium-level compression than zstd. Equally weak in decomp like lz4. That's probably the reasons it was not worth supporting in rpm.
* Fedora's plan to replace xz-2 with zstd-18 will at least double their compression times.
* Replacing openSUSE's xz-5 with zstd-18 gives the decompression benefit, no improvement in compression time and a slight space increase.
* xz-5 to zstd-10 would also give a 7x compression time saving with a slight more space increase. (23.6->27.9G)
So with that, I propose changing openSUSE to zstd-10 that will give people time back they're waiting for.. for the computer. xkcd.com/303 .
The problem is that even though we have a suitable version of rpm since Dec 2017, the maintainer did not enable zstd so the "compatibility time" accumulated since was unfortunately wasted.
Hi. Thank you for working on that, I was suggesting the same here: https://github.com/openSUSE/rpm-config-SUSE/issues/11 Note that one possible blocker is missing support for zstd in deltarpm. Martin -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org