Mailinglist Archive: opensuse-buildservice (166 mails)

< Previous Next >
[opensuse-buildservice] OBS change: new build repo handling

Hi OBS users,

after testing out the new build repository handling on the stage
area for a few weeks, we've now enabled it for the home projects
as well. Here is what the change is about:


OBS maintains an internal repo used for building ("the :full tree").
The "useforbuild" flag can be used to configure if built packages
are copied into the tree or not.

The problem:
------------

The old code checked the "useforbuild" flag only after a package
was successfully built. In that case, the old packages in the full
tree were replaced with the new results, otherwise the full tree
was not changed. This has a couple of drawbacks:

- if two packages built the same binary, the last built wins. Thus,
the state of the full tree depends on the timeline of the builds.
This somewhat contradicts the build service goal of having
reproducible builds.
- if the project maintainer changes the useforbuild flag, nothing
happens as the flag is only looked at the next time a build is done.
This means that it's very hard to repair the effects of a bad
useforbuild setting.

Our solution:
-------------

The new code does away with the dependency on the build timeline.
This means that at any point in time the build repository can be
reconstructed from the build results. This means:

- there's now a fixed rule if the packages build the same binary:
the package that comes lexicographical first wins. E.g. if package
"A" builds "foo.rpm" and package "B" also builds "foo.rpm", the
build repo will contain foo.rpm from "A".
- changing "useforbuild" will have an immediate effect on the build
repository.

Note that the last point makes "useforbuild" semantically different
from the "publish" flag. This is a little drawback, but we argue that
the "publish" flag is often used to "stage" new releases, which is
a bit different from "useforbuild".


There's another change that was needed because of the new useforbuild
handling: as we need to be able to reconstruct the build repo from
the package build results, we also needed to store the build results
imported from other architectures (i.e. the -32bit packages) in a
different place. Before the change they went directly into the build
repo, now we need to store them somewhere else. So we've decided to
store them in the packages next to the other build results.

E.g. if package foo builds both "foo-1.1.i586.rpm" and
"foo-32bit-1.1.x86_64.rpm" on the i586 scheduler and "foo-1.1.x86_64.rpm"
on the x86_64 scheduler, we'll now store:

i586/foo
foo-1.1.src.rpm
foo-1.1.i586.rpm
foo-32bit-1.1.x86_64.rpm

x86_64/foo
foo-1.1.src.rpm
foo-1.1.x86_64.rpm
::import::i586::foo-32bit-1.1.x86_64.rpm

So don't be surprised if you see those strange "::import" files popping
up in the build results.

(Having the imported binary packages available with the normal package
build results also fixes a long standing limitation with kiwi product
builds, but I don't want to bore you with the details.)

Cheers,
Michael.

--
Michael Schroeder mls@xxxxxxx
SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-buildservice+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups