enabling reproducible builds in Factory and ALP
Hi *, to support https://en.opensuse.org/openSUSE:Reproducible_Builds I plan to get the following project configuration for Factory and ALP enabled: Macros: %source_date_epoch_from_changelog Y %clamp_mtime_to_source_date_epoch Y %use_source_date_epoch_as_buildtime Y %_buildhost reproducible :Macros I'm still waiting to get https://github.com/openSUSE/product-builder/pull/26 merged to prevent a bug with mirrors that will otherwise happen. Afterwards a time of the build is still in the HTTP header Last-Modified when requesting the rpm file. HTTP mtime is a few seconds earlier than end time of the build (seen in OBS web UI only as start + duration). I didn't double check, but it should also be recorded in the Tumbleweed archive Bernhard keeps, which can with a bit of work be looked up by content hash or name including version of the rpm file if its not on the normal mirrors anymore. Both the build time and host can also still be seen in the build log. Comments welcome. -- Best regards, Jan Zerebecki
On Wed, Jul 12, 2023 at 3:50 PM Jan Zerebecki <jan.suse@zerebecki.de> wrote:
Hi *,
to support https://en.opensuse.org/openSUSE:Reproducible_Builds
I plan to get the following project configuration for Factory and ALP enabled:
Macros: %source_date_epoch_from_changelog Y %clamp_mtime_to_source_date_epoch Y %use_source_date_epoch_as_buildtime Y %_buildhost reproducible :Macros
Is there a good reason to set %use_source_date_epcoh_as_buildtime and %_buildhost, given that we can easily ignore those fields when doing RPM comparisons to identify payload reproducibility? Not to mention, it reduces debuggability and tracing when trying to figure out broken builds caused by environmental factors. The rest of the parameters make sense to me for reproducibility, but not those two. -- 真実はいつも一つ！/ Always, there's only one truth!
On 12/07/2023 21.50, Jan Zerebecki wrote:
As of 2024-03-11 this is in a slightly modified way enabled by default in Factory. Thank you, to the people who helped with this. This was first tried nearly 5 years ago, see https://bugzilla.suse.com/show_bug.cgi?id=1148824 . -- Best regards, Jan Zerebecki
Will this help in getting build-compare enabled in the Fedora repos? (bsc#1206734) Last time i tried to test this, it was broken because of the changing build-id files.
On Fri, Mar 15, 2024 at 1:31 AM Bruno Pitrus <brunopitrus@hotmail.com> wrote:
Will this help in getting build-compare enabled in the Fedora repos? (bsc#1206734) Last time i tried to test this, it was broken because of the changing build-id files.
Not unless build-compare is set up to filter out certain RPM header tags that are intentionally non-reproducible (build host, build time, rpm signature, and a few others). I am not aware of randomly changing build-id files on anything except Go software, and that's been fixed: https://pagure.io/go-rpm-macros/c/1980932bf3a21890a9571effaa23fbe034fd388d With my Fedora hat on, I can say there has been a reproducible builds effort that's been active for years, but the difference is that we are not following Debian's exact definition of it, because it is not fully applicable for Fedora (or indeed most RPM distributions). More details are on the effort website: https://docs.fedoraproject.org/en-US/reproducible-builds/ -- 真実はいつも一つ！/ Always, there's only one truth!
On Fri, Mar 15, 2024 at 04:11:00AM -0700, Neal Gompa wrote:
On Fri, Mar 15, 2024 at 1:31 AM Bruno Pitrus <brunopitrus@hotmail.com> wrote:
Will this help in getting build-compare enabled in the Fedora repos? (bsc#1206734) Last time i tried to test this, it was broken because of the changing build-id files.
Not unless build-compare is set up to filter out certain RPM header tags that are intentionally non-reproducible (build host, build time, rpm signature, and a few others).
I am not aware of randomly changing build-id files on anything except Go software, and that's been fixed: https://pagure.io/go-rpm-macros/c/1980932bf3a21890a9571effaa23fbe034fd388d
With my Fedora hat on, I can say there has been a reproducible builds effort that's been active for years, but the difference is that we are not following Debian's exact definition of it, because it is not fully applicable for Fedora (or indeed most RPM distributions).
More details are on the effort website: https://docs.fedoraproject.org/en-US/reproducible-builds/
I let Jan fill in, but he and others worked hard on handling this to make this even happen at RPM level: FWIW this was changed: - build host is now fixed (the host is now always named "reproducible") - build time is now fixed (to the changes modification time) The idea is to have the exact same RPMs. Ciao, Marcus
On 15/03/2024 12.11, Neal Gompa wrote:
On Fri, Mar 15, 2024 at 1:31 AM Bruno Pitrus <brunopitrus@hotmail.com> wrote:
Will this help in getting build-compare enabled in the Fedora repos? (bsc#1206734) Last time i tried to test this, it was broken because of the changing build-id files.
Bruno, either way trying again might be worthwhile. Please comment on the bug if you do. Reproducible builds in general should be helpful, but my guess is, that this specific change might not be necessary for build-compare to work. This change is only for Factory and anything that inherits from it. While Tumbleweed is built this way, things that inherit from Tumbleweed will not have it enabled by default yet, but you can do that by using the settings listed on https://en.opensuse.org/openSUSE:Reproducible_Builds . Enabling it for a project that inherits from Fedora is more complicated as the script to pass some environment variables from OBS to rpm isn't included there: https://github.com/openSUSE/post-build-checks/pull/62 But as before, I guess that is not necessary for your use case.
Not unless build-compare is set up to filter out certain RPM header tags that are intentionally non-reproducible (build host, build time, rpm signature, and a few others).
build-compare is to do just that: To filter out known types of changes that are considered the same in packages that are not fully bit-wise reproducible.
I am not aware of randomly changing build-id files on anything except Go software, and that's been fixed: https://pagure.io/go-rpm-macros/c/1980932bf3a21890a9571effaa23fbe034fd388d
While this macro uses SOURCE_DATE_EPOCH, it will still work without it, and thus without the change I announced.
With my Fedora hat on, I can say there has been a reproducible builds effort that's been active for years, but the difference is that we are not following Debian's exact definition of it, because it is not fully applicable for Fedora (or indeed most RPM distributions).
More details are on the effort website: https://docs.fedoraproject.org/en-US/reproducible-builds/
From my reading your link contradicts what you are saying. It says it uses the exact same definition, bit-by-bit. That it is applicable to Fedora. It recognizes that even getting part of the way there is useful. Which Debian did also, as it took a long time for packages to be build reproducible by default in Debian. It links to the rpm upstream discussions where I argue for improving support for this in rpm. Now in Factory we have bit-by-bit reproducibility except for the signature. (Though there is still a regression in rpmsign when deleting the signature adds zeros https://github.com/rpm-software-management/rpm/issues/2965 which needs to be fixed to be able to do the bit wise compare of two builds.) Where I disagree with the text at the link is that I think in the future, step by step we should get to full bit-by-bit reproducible rpms. But making packages reproducible in all the other ways should not wait on that.
真実はいつも一つ！/ Always, there's only one truth!
Modern science as written about by Karl Popper (see keywords falsifiability, critical rationalism), shares the idea that only one truth exists. With the caveat that humans have no way to detect the one truth, but they can detect what is not true. So humans can learn from what they recognized as not true. Science requires reproducibility. -- Best regards, Jan Zerebecki
participants (4)
-
Bruno Pitrus -
Jan Zerebecki -
Marcus Meissner -
Neal Gompa