Mailinglist Archive: opensuse-factory (1134 mails)

< Previous Next >
Re: [opensuse-factory] Build system solutions (was Re: Calling for a new openSUSE development model)
On Thu, Jun 14, 2012 at 09:48:30PM +0200, Adrian Schröter wrote:
Am Donnerstag, 14. Juni 2012, 12:34:44 schrieb Greg KH:
On Thu, Jun 14, 2012 at 10:46:32AM +0200, Stephan Kulow wrote:
3. As working more strictly will require more time, I would like to
either ditch release schedules all together or release only once a
year and then rebase Tumbleweed - as already discussed in the RC1

I like this idea, for a variety of reasons :)

But, before that can ever happen, we need two big things to be solved:

1) Build system speed
I speak as someone who builds stuff a lot, probably second only to
Factory. The build system speed sucks. Badly. Now I know we are

yes, we need to become faster. We have multiple ideas atm how to approach
it, we just need some time for it ...

Nice, any pointers to where these ideas are?

running it in virtual machines, shared with other processes doing
the same thing, but when I can do a 'make allyesconfig' kernel
build on my local "old" system in less than 2 minutes, and it
takes over an hour in the build system, something is wrong.
Also note that we are routinely doing kernel builds like this in
less than a minute on "modern" hardware, and even faster coupled
with real storage devices (i.e. flash PCIe cards.)
We need to figure out a way to upgrade what we have, and provide a
very fast storage system (FusionIO or Micron anyone?, spinning
rust really doesn't cut it anymore for any production system that
has lots of i/o like we do). Good thing is, this problem can just
be solved by throwing money at it, it's not all that tough to do
at all.

We will order some SSD's for one build host and evaluate it.
Right now we are not sure how big the effect will be in real life.

Also when we have capability based job assignment, we can have a few
systems which build in ram only. They should be able to process the small
packages (eg. perl modules) fast.

That would speed some things up, but it's the longer-running builds
(kde, gnome, libreoffice, kernel, glibc) that really need the

2) Build system dependencies
Related to the above, we need a "smarter" way to determine when
packages need to be rebuilt. As an example, for a few days I
based Tumbleweed on openSUSE:12.1:Update. Big mistake. A bugfix
for a library like libxml2 causes everything in Tumbleweed to be
rebuilt for no real reason. If the .so symbols don't change, the
packages that depend on that library should not need to be
rebuilt. That is how Gentoo solves this problem, I'm sure that
Debian also has something like this to solve the problem as well.
But if we are ever going to think that we can move to a real
"rolling" release model, this is going to have to be solved in the
build system scheduler somehow.
And yes, sometimes you want to really rebuild all dependant
packages (i.e. python/perl updates, but not gcc or glibc unless
it's necessary, and it usually isn't.) We need a way to be able
to mark a checkin as needing to cause this rebuild or not if we
are going to be able to do this properly, or for a way for the
build system to determine this on its own.

you can configure you repo already for building only on source changes.

How? And what do you mean "only on source changes", the tarball or
version number change?

But real life has shown that marking packages for incompatible changes
is not working good enough. And suddenly you hunt problems for a long
time which in the end a rebuild is fixing.

Yeah, it's a tough problem. Gentoo has a tool that does this for you.
Anyone know what Debian does? They must have solved this otherwise
their build system would be constantly running behind.


greg k-h
To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-factory+owner@xxxxxxxxxxxx

< Previous Next >
This Thread