Mailinglist Archive: opensuse-factory (1134 mails)

< Previous Next >
Re: [opensuse-factory] Build system solutions (was Re: Calling for a new openSUSE development model)
  • From: Adrian Schröter <adrian@xxxxxxx>
  • Date: Thu, 14 Jun 2012 21:48:30 +0200
  • Message-id: <3178209.TvBqj0h8Cu@scherben>
Am Donnerstag, 14. Juni 2012, 12:34:44 schrieb Greg KH:
On Thu, Jun 14, 2012 at 10:46:32AM +0200, Stephan Kulow wrote:
3. As working more strictly will require more time, I would like to
either ditch release schedules all together or release only once a
year and then rebase Tumbleweed - as already discussed in the RC1
thread.

I like this idea, for a variety of reasons :)

But, before that can ever happen, we need two big things to be solved:

1) Build system speed
I speak as someone who builds stuff a lot, probably second only to
Factory. The build system speed sucks. Badly. Now I know we are

yes, we need to become faster. We have multiple ideas atm how to approach
it, we just need some time for it ...

running it in virtual machines, shared with other processes doing
the same thing, but when I can do a 'make allyesconfig' kernel
build on my local "old" system in less than 2 minutes, and it
takes over an hour in the build system, something is wrong.
Also note that we are routinely doing kernel builds like this in
less than a minute on "modern" hardware, and even faster coupled
with real storage devices (i.e. flash PCIe cards.)
We need to figure out a way to upgrade what we have, and provide a
very fast storage system (FusionIO or Micron anyone?, spinning
rust really doesn't cut it anymore for any production system that
has lots of i/o like we do). Good thing is, this problem can just
be solved by throwing money at it, it's not all that tough to do
at all.

We will order some SSD's for one build host and evaluate it.
Right now we are not sure how big the effect will be in real life.

Also when we have capability based job assignment, we can have a few
systems which build in ram only. They should be able to process the small
packages (eg. perl modules) fast.

2) Build system dependencies
Related to the above, we need a "smarter" way to determine when
packages need to be rebuilt. As an example, for a few days I
based Tumbleweed on openSUSE:12.1:Update. Big mistake. A bugfix
for a library like libxml2 causes everything in Tumbleweed to be
rebuilt for no real reason. If the .so symbols don't change, the
packages that depend on that library should not need to be
rebuilt. That is how Gentoo solves this problem, I'm sure that
Debian also has something like this to solve the problem as well.
But if we are ever going to think that we can move to a real
"rolling" release model, this is going to have to be solved in the
build system scheduler somehow.
And yes, sometimes you want to really rebuild all dependant
packages (i.e. python/perl updates, but not gcc or glibc unless
it's necessary, and it usually isn't.) We need a way to be able
to mark a checkin as needing to cause this rebuild or not if we
are going to be able to do this properly, or for a way for the
build system to determine this on its own.

you can configure you repo already for building only on source changes.

But real life has shown that marking packages for incompatible changes
is not working good enough. And suddenly you hunt problems for a long
time which in the end a rebuild is fixing.

Good thing is, solve #1, and #2 becomes less of an issue.

Solve #2 only and we have hope that someday our build system can catch
up with the pending builds.

But solve neither of them, then we can never properly move to any type
of incremental release system like people are wanting to do so, it will
just take too long, and people will get upset that they are downloading
new copies of LibreOffice every other day.

Well, every other week if #1 doesn't get solved :)

thanks,

greg k-h

--
Adrian Schroeter
SUSE Linux Products GmbH
email: adrian@xxxxxxx

--
To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-factory+owner@xxxxxxxxxxxx

< Previous Next >
This Thread
Follow Ups
References