Am Donnerstag, 18. Februar 2016 schrieb Bernhard M. Wiedemann:
On 2016-02-17 22:07, Christian Boltz wrote:
Am Mittwoch, 17. Februar 2016 schrieb Bernhard M. Wiedemann:
now I had some time to look into how reproducible our builds are and had a VM (with 4 cores) build all Leap 42.1 packages named a*
I just checked the Debian result for some of them on https://tests.reproducible-builds.org/reproducible.html and interestingly, several of the packages you listed above are reproducible on Debian: acct, aegisub, aespipe, anthy, apel, asl, aspell, autogen, avogadro, avrdude.
I know the reproducible builds team at Debian is actively pushing patches to make the build reproducible, but I'm not sure if they were able to fix that many packages already (10 out of 36 - actually I should probably say 10 out of ~25 because some of the packages you listed don't exist in Debian, at least not with this name).
I think, they are very active and use latest upstream versions,
BTW: when it comes to Debian, I wouldn't always assume that Leap packages are older ;-)
possibly with extra patches,
Yes, they are actively providing patches to make the builds reproducible (which will first be in Debian of course), but they also send those patches upstream, so we should get them sooner or later.
while I tested with Leap, which is an older codebase, but Factory was too fast-moving for me to test with atm.
That's understandable - checking a fixed target makes things much easier.
Leap is probably a good starting point - mass-rebuild all packages, get the list of packages that differ - and then check only those packages in Factory. A list of some hundred packages is easier to handle on a moving target than some thousand packages ;-)
(In the long term, I'd love to have Factory reproducible - but we need to start somewhere.)
Some diffs could also be caused by the fact that we do not always rebuild all dependent packages on changes. That could be avoided by doing it like the debian guys - rebuilding twice in two very different contexts.
They always rebuild with exactly the same versions of the BuildRequires both times (but with different timezone, locale, whatever).
Can you upload the differences for the packages you tested somewhere so that interested people don't need to rebuild everything just to get the diff?
I was curious and tried with aspell.
It seems the only difference is in the rpmtags - (none) vs. obs://build.opensuse.org/openSUSE:Leap:42.1/standard/dd3d6bcef5a4f7f e0ae04047dd14a214-aspell
Is this really a relevant difference, or should it be whitelisted in build-compare?
that is what I did with https://github.com/bmwiedemann/reproducibleopensuse/blob/master/build-%3E compare.diff but not yet in a nice upstreamable way.
Ah, I didn't look at that patch ;-)
for aspell (and only this 1 of 36) I found, that is is reproducible when building with osc build --vm-type=kvm but when using the default chroot, it shows
comparing filelist @@ -1,8 +1,8 @@ /usr/bin/aspell 0 (none) 100755 root root 0 4294967295 /usr/bin/aspell-import 0 (none) 100755 root root 0 4294967295 -/usr/bin/precat 0 (none) 100755 root root 0 4294967295 -/usr/bin/preunzip 0 (none) 120777 root root 0 4294967295 precat -/usr/bin/prezip 0 (none) 120777 root root 0 4294967295 precat +/usr/bin/precat 0 (none) 120777 root root 0 4294967295 prezip +/usr/bin/preunzip 0 (none) 120777 root root 0 4294967295 prezip +/usr/bin/prezip 0 (none) 100755 root root 0 4294967295 /usr/bin/prezip-bin 0 (none) 100755 root root 0 4294967295 /usr/bin/run-with-aspell 0 (none) 100755 root root 0 4294967295 /usr/bin/word-list-compress 0 (none) 100755 root root 0 4294967295 comparing file checksum creating rename script RPM file checksum differs. Extracting packages @@ -1 +0,0 @@ -precat symlink target for /usr/bin/prezip differs
probably from the brp-symlink doing things differently in that context.
I used osc build, but didn't see those differences. Maybe the relevant part is that I'm building in /dev/shm
The takeaway from the filelist diff is that (probably) brp-symlink needs a "sort" somewhere to get a stable order (including LC_ALL=C to ensure sort behaves the same all the time). I wouldn't be surprised if this little change fixes filelist diffs in lots of packages.
I also tried aespipe, and get a real difference. Maybe someone should compare the openSUSE package with the Debian package to find out what they do different to make the package reproducible ;-)
e.g. https://www.zq1.de/~bernhard/linux/reproducibleopensuse/compare/alpine -c ompare.out shows an embedded ASCII date in 2 binaries.
Indeed, embedding the build date is the cause for differences in several of the packages you tested. Not only in compiled binaries - I even found the build date in *.png files in awesome-compare.out ;-)
Embedded build dates aren't too hard to fix - either completely leave it out (easiest solution) or use a fixed date that is still helpful. That fixed date can be something like the date of the last *.changes entry, which would IMHO be more useful than the information when OBS finally had some time to build the package. (We probably need to discuss which date to choose - I'd vote for the last *.changes entry.)
BTW: axis-compare.out shows a big difference in the filelist - it seems like your rebuild is missing half of the files in /usr/share/javadoc/. Also, /usr/share/javadoc/axis/javax/xml/namespace/class-use/QName.html in your rebuild seems to be completely empty.
I have no idea what caused this, but it's clearly more than some "random" change during the rebuild. I'm especially surprised that those missing files didn't cause the build to fail - which I'd call a bug.