On Dienstag, 9. Oktober 2018 13:52:26 CEST Jan Engelhardt wrote:
On Tuesday 2018-10-09 12:52, Dave Plater wrote:
On 08/10/2018 14:51, Jan Engelhardt wrote:
* Some of our packages are too fat. kicad-packages takes longer to compress
than the entire remaining distro at zstd-19 with 16x-parallelism. In other words, a sufficiently parallelized system may have to wait just for that one to complete.
Interesting, apart from the main kicad package which is a binary/python mix the other packages are all noarch text of which kicad-packages3D is enormous in size. My assumption from this would be the need for a different compressor for noarch packages.
noarch does not mean anything with regard to compression, just that it does not (better: is believed to not have) any arch-specific content.
In my test, xz -6 outperform zstd -19 both compression and speed wise. But as compression is all about statistics, I have found a neat cheap trick to improve the compression efficiency for kicad-packages3D considerably: The packages (3d models) come in pairs of 2 files, one VRML and one STEP (ISO-10301 part 214) file. Both are ASCII formats, but with different keywords, syntax and the like. Just reordering the cpio contents to have all VRML files first and STEP files after reduces the files size from 291 MByte to 277 MByte (xz -6). Compression time is 40 minutes in both cases (i7-4500U, single thread). For comparison, zstd -4 needs 40 minutes too, but ends up with ~650 MByte. I think this may work for a number of other cases: - doc packages, mixing HTML and PNG, JPEG, ... - python packages, mixing sources and .pyc/.pyo I think this could even be incorporated in RPM itself, feed it the file list, order by filename suffix. Kind regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen home: +49 241 53809034 mobile: +49 151 50412019