Hi OBS users,

after testing out the new build repository handling on the stage area for a few weeks, we've now enabled it for the home projects as well. Here is what the change is about:

OBS maintains an internal repo used for building ("the :full tree"). The "useforbuild" flag can be used to configure if built packages are copied into the tree or not.

The problem: ------------

The old code checked the "useforbuild" flag only after a package was successfully built. In that case, the old packages in the full tree were replaced with the new results, otherwise the full tree was not changed. This has a couple of drawbacks:

- if two packages built the same binary, the last built wins. Thus, the state of the full tree depends on the timeline of the builds. This somewhat contradicts the build service goal of having reproducible builds. - if the project maintainer changes the useforbuild flag, nothing happens as the flag is only looked at the next time a build is done. This means that it's very hard to repair the effects of a bad useforbuild setting.

Our solution: -------------

The new code does away with the dependency on the build timeline. This means that at any point in time the build repository can be reconstructed from the build results. This means:

- there's now a fixed rule if the packages build the same binary: the package that comes lexicographical first wins. E.g. if package "A" builds "foo.rpm" and package "B" also builds "foo.rpm", the build repo will contain foo.rpm from "A". - changing "useforbuild" will have an immediate effect on the build repository.

Note that the last point makes "useforbuild" semantically different from the "publish" flag. This is a little drawback, but we argue that the "publish" flag is often used to "stage" new releases, which is a bit different from "useforbuild".

There's another change that was needed because of the new useforbuild handling: as we need to be able to reconstruct the build repo from the package build results, we also needed to store the build results imported from other architectures (i.e. the -32bit packages) in a different place. Before the change they went directly into the build repo, now we need to store them somewhere else. So we've decided to store them in the packages next to the other build results.

E.g. if package foo builds both "foo-1.1.i586.rpm" and "foo-32bit-1.1.x86_64.rpm" on the i586 scheduler and "foo-1.1.x86_64.rpm" on the x86_64 scheduler, we'll now store:

i586/foo foo-1.1.src.rpm foo-1.1.i586.rpm foo-32bit-1.1.x86_64.rpm

x86_64/foo foo-1.1.src.rpm foo-1.1.x86_64.rpm ::import::i586::foo-32bit-1.1.x86_64.rpm

So don't be surprised if you see those strange "::import" files popping up in the build results.

(Having the imported binary packages available with the normal package build results also fixes a long standing limitation with kiwi product builds, but I don't want to bore you with the details.)

Cheers, Michael.