Mailinglist Archive: opensuse-factory (498 mails)

< Previous Next >
Re: [opensuse-factory] Slow RPM generating on libreoffice

On Tuesday 2016-05-17 15:54, Tomas Chvatal wrote:

I am having quite problems with slowness that happens when building
libreoffice [1]. This time the build was exceptionally slow but aside
from that I noticed 1/3rd of the time was spent generating the rpms.

Thats really really bad.
End is [37099s] OTHER/rpmlint.log
And build ends on [24352s] ~/rpmbuild/BUILD/libreoffice-5.1.3.2

At 24352 (405 minutes), you have reached the end of %install - at
least, the part you are seeing in the .spec file.

Remember that there is practically no parallelism in anything outside
of the `make -j` commands you do in the .spec. After 24352,
/usr/lib/rpm/find-* and /usr/lib/rpm/brp* run until 34650. This
includes the debuginfo extractor (paralellizable since it operates
on individual files) and the compressor (chunkable as shown by
implementations like pixz).

Then you got the post-build-checks/rpmlint/build-compare run.
So we note down:

* 0..559 (1.5%)
initialization and start (single threaded)
* 559..24352 (64.1%)
make -j4 build (hopefully); until the visible end of %install
* 24352..34650 (27.7%)
/usr/lib/rpm/brp* and /usr/lib/rpm/find-* which are
implicitly appended to %install. Compressor runs.
* 30574..34650 (10.9%) compressing the rpm data stream
* 34670..35367 (1.8%) post-build-checks
* 35367..35844 (1.2%) rpmlint
* 35844..37041 (3.2%) build-compare
* 37041..37099 (0.1%) transmission and pickup by scheduler

Blame points:
- brp/find implemented in sh
- reeking of O(n^p),p>=2 behavior, e.g. brp-compress.
- find-debuginfo operating serially on independent(methinks) files,
could be turned into make -jX like
- compressor being inherently invoked serial *and* single-threaded,
could be turned into parallel chunked processing à la pixz.

That calls for a hackweek.
--
To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-factory+owner@xxxxxxxxxxxx

< Previous Next >
References