Mailinglist Archive: opensuse-packaging (106 mails)

< Previous Next >
Re: [opensuse-packaging] reproducible builds
  • From: Ludwig Nussel <ludwig.nussel@xxxxxxx>
  • Date: Wed, 16 Dec 2015 11:18:32 +0100
  • Message-id: <>
Michael Matz wrote:
On Fri, 11 Dec 2015, Adam Spiers wrote:

Is anyone working on (or thinking of working on) making our build
process reproducible?

We have that since about forever as far as easily possible. The hard part
is changing packages to not depend on things like build time (e.g.
encoding build date/time into strings into executables). That's not
something you can do generally in a build system, but must be changed in
each and every individual package.

It's probably worth to explain the difference between our reproducible
builds and this new interpretation. SUSE and openSUSE distributions
have always had reproducible builds, for something like 20 years now.
Reproducible in the sense that a packager never builds binaries on his
own system in some magic way and then uploads binaries.

We always build sources server side (nowadays OBS, previously
autobuild). How the build environment has to look like is defined via
BuildRequires in the spec file and settings in the project config on
server side. Moreover, we don't allow packagers to directly build
packages in the distribution's project. There's always a review step
(four eyes principle). Some distributions don't have that and only have
reviews when a package is accepted for the first time.

OBS always re-creates the build environment from scratch for each
package and automatically uses other packages in the same project to set
up that build environment. Ie there's no magic base build system, the
distribution bootstraps itself. Not only on request or mass rebuilds but
fully automatic. So even packages that haven't been submitted for
years are rebuilt with current compilers and libraries. Additionally
every binary rpm produced by obs contains a back reference to the
used sources (in the disturl).

IOW our process and infrastructure guarantees that our packages can
reproducibly be built from source. Everyone can reproduce that by
firing up their own build service and linking to OBS. In that sense
_our build process is reproducible_ and has always been. The terrifying
news here is that other distributions still have to do homework to
even get there.

However, that project wants to achieve an
addition goal and that is to have bit identical rpm results of two
build runs. We trust our process and infrastructure so it wasn't
important to us so far.

The mechanism we have in place to detect unchanged binary rpms
(build-compare) only partially gets us there as it solves a
different problem. build-compare is basically a heuristic to avoid
triggering rebuilds of other packages and publishing of packages
with only trivial changes (like time stamps). It does that by
"normalizing" some content and throwing away results with trivial
changes. It does not necessarily prevent trivial changes.
Some of the measurements to please build-compare are also useful for
bit identical results though, like eliminating time stamps in build
results in the first place.

So if anyone wants to pursue that idea,
fixing the issues with build-compare is a good first step.

Also, it needs to be defined what a bit identical rpm result
actually is. Only the rpm payload isn't sufficient as the header
includes e.g. scripts. The full header can't be compared as it
contains time stamps, build host names, signatures etc. So again a
script similar to what build-compare does would be needed to do the
actual comparison, md5sums of rpm files won't suffice.

Another project related to that might be to verify source rpms can
be rebuilt. So far our assumption is that a build service is
available. So everything is centered around build service packages
containers rather than srpms.


(o_ Ludwig Nussel
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard,
Graham Norton, HRB 21284 (AG Nürnberg)
To unsubscribe, e-mail: opensuse-packaging+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-packaging+owner@xxxxxxxxxxxx

< Previous Next >