Mailinglist Archive: opensuse-factory (381 mails)

< Previous Next >
Re: [opensuse-factory] A look on RPM compression in openSUSE
  • From: Joachim Wagner <jwagner@xxxxxxxxxxxxxxxx>
  • Date: Mon, 08 Oct 2018 15:10:32 +0100
  • Message-id: <3516826.fVTDfrAMEb@e5430>
* Some of our packages are too fat. kicad-packages takes longer to compress
than the entire remaining distro at zstd-19 with 16x-parallelism.

This data will therefore dominate the stats you created. Other packages may
behave differently.

If the goal is to minimise network traffic, the sizes should be weighted by
popularity (number of downloads).

One could also question whether picking a single compression method for all
packages is the way forward. A simple suggestion would be to select the
compression method for each package independently. Even better will be to
allow an arbitrary number of subsection in a package that can each pick its
own decompressor (more subsection than decompressors can make sense as one may
want to use different compression settings). Each subsection can contain
multiple files, scripts and/or metadata. For an example of a file format that
can mix different compression methods see PDF.

In other
words, a sufficiently parallelized system
may have to wait just for that one to complete.

Simple countermeasures are to
* split large packages into smaller ones (current kicad-packages would then
become a meta-package/pattern that pulls in the other packages)
* order compression tasks so that largest packages come first
* not parallelise too much: use stats from previous run to estimate optimal
number of workers

To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-factory+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups