[opensuse-go] Stripping of Go binaries
Hi! The Packaging Go page in the Wiki [1] recommends to use the %{go_nostrip} macro to prevent issues with Go tooling. However, it is unclear to me whether this recommendation should also be followed for application binaries. After all, the page mostly focuses on the, now deprecated, development packages. For these I fully see the purpose of unstripped binaries. However, for normal applications it seems to me stripping would make sense, as it reduces binary size and it is unlikely people will run tooling on those. So what's the best practice regarding stripping application binaries? Kind Regards, Matthias [1]: https://en.opensuse.org/openSUSE:Packaging_Go -- Dr. Matthias Bach www.marix.org „Der einzige Weg, die Grenzen des Möglichen zu finden, ist ein klein wenig über diese hinaus in das Unmögliche vorzustoßen.“ - Arthur C. Clarke
Matthias Bach writes:
The Packaging Go page in the Wiki [1] recommends to use the %{go_nostrip} macro to prevent issues with Go tooling. However, it is unclear to me whether this recommendation should also be followed for application binaries. After all, the page mostly focuses on the, now deprecated, development packages. For these I fully see the purpose of unstripped binaries. However, for normal applications it seems to me stripping would make sense, as it reduces binary size and it is unlikely people will run tooling on those.
So what's the best practice regarding stripping application binaries?
Hello Matthias, Many users do strip binaries, and it is generally considered safe: https://dominik.honnef.co/posts/2016/10/go-and-strip/ (2016) For openSUSE/SUE packaging purposes, I think it might be best to fully understand the oldest versions of Go that will be required/supported, align with any requirements of reproducible builds, and plan for stripped binaries as one item in the process of improving Go packaging and documentation. An upstream proposal may address the size of Go binaries in a future release: go#26074 proposal: cmd/link: by default, do not write out DWARF https://github.com/golang/go/issues/26074 go#6853 all: binaries too big and growing (2013-2018) https://github.com/golang/go/issues/6853 go#23934 all: unstripped binary size growth between Go 1.9 and 1.10 https://github.com/golang/go/issues/23934 Thanks, Jeff -- Jeff Kowalczyk Software Engineer, Go Developer Experience SUSE Linux http://suse.com -- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
On 2018-11-08, Jeff Kowalczyk
The Packaging Go page in the Wiki [1] recommends to use the %{go_nostrip} macro to prevent issues with Go tooling. However, it is unclear to me whether this recommendation should also be followed for application binaries. After all, the page mostly focuses on the, now deprecated, development packages. For these I fully see the purpose of unstripped binaries. However, for normal applications it seems to me stripping would make sense, as it reduces binary size and it is unlikely people will run tooling on those.
So what's the best practice regarding stripping application binaries?
Hello Matthias,
Many users do strip binaries, and it is generally considered safe: https://dominik.honnef.co/posts/2016/10/go-and-strip/ (2016)
Hmmm, I hadn't seen this blog post. The author is correct that the official Go compiler has fewer problems, but there definitely is evidence from less than 5 years ago that strip will break stack traces -- we disabled OBS stripping specifically because gcc-go stack traces were broken on s390x and ppc64le (back before Go supported them in their main compiler). Still, using "-s" is stripping just as well as GNU strip and is actually part of the Go toolchain (and is what we use), so I don't think we need to remove the nostrip stuff... -- Aleksa Sarai Senior Software Engineer (Containers) SUSE Linux GmbH https://www.cyphar.com/
On Thu, Nov 8, 2018 at 9:30 PM Aleksa Sarai
On 2018-11-08, Jeff Kowalczyk
wrote: The Packaging Go page in the Wiki [1] recommends to use the %{go_nostrip} macro to prevent issues with Go tooling. However, it is unclear to me whether this recommendation should also be followed for application binaries. After all, the page mostly focuses on the, now deprecated, development packages. For these I fully see the purpose of unstripped binaries. However, for normal applications it seems to me stripping would make sense, as it reduces binary size and it is unlikely people will run tooling on those.
So what's the best practice regarding stripping application binaries?
Hello Matthias,
Many users do strip binaries, and it is generally considered safe: https://dominik.honnef.co/posts/2016/10/go-and-strip/ (2016)
Hmmm, I hadn't seen this blog post. The author is correct that the official Go compiler has fewer problems, but there definitely is evidence from less than 5 years ago that strip will break stack traces -- we disabled OBS stripping specifically because gcc-go stack traces were broken on s390x and ppc64le (back before Go supported them in their main compiler).
Still, using "-s" is stripping just as well as GNU strip and is actually part of the Go toolchain (and is what we use), so I don't think we need to remove the nostrip stuff...
In Fedora, we've allowed automatic stripping of binaries for debuginfo generation on Go for at least the last three years. We've also fixed debuginfo generation in RPM upstream so that binaries with weird data like Rust and Go don't break when we strip and put it aside for debuginfo packages. This works fine with golang and gcc-go, in my experience. If OBS is doing stripping separately from rpmbuild, then I don't know whether the binaries would be sane after that. But at least RPM itself should be fine. -- 真実はいつも一つ!/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
Neal Gompa writes:
In Fedora, we've allowed automatic stripping of binaries for debuginfo generation on Go for at least the last three years. We've also fixed debuginfo generation in RPM upstream so that binaries with weird data like Rust and Go don't break when we strip and put it aside for debuginfo packages.
Thanks for that information, Neal. Do you know the approximate RPM version/date where go and rust -friendly debuginfo support landed, and what nomenclature was used to describe it? I see debuginfo features in http://rpm.org/wiki/Releases/4.14.0 (Oct 12 2017)
This works fine with golang and gcc-go, in my experience. If OBS is doing stripping separately from rpmbuild, then I don't know whether the binaries would be sane after that. But at least RPM itself should be fine.
Also helpful, thanks. Do you know of a Fedora package using gcc-go that I could look at? Jeff -- Jeff Kowalczyk Software Engineer, Go Developer Experience SUSE Linux http://suse.com -- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
On Fri, Nov 9, 2018 at 12:25 AM Jeff Kowalczyk
Neal Gompa writes:
In Fedora, we've allowed automatic stripping of binaries for debuginfo generation on Go for at least the last three years. We've also fixed debuginfo generation in RPM upstream so that binaries with weird data like Rust and Go don't break when we strip and put it aside for debuginfo packages.
Thanks for that information, Neal. Do you know the approximate RPM version/date where go and rust -friendly debuginfo support landed, and what nomenclature was used to describe it? I see debuginfo features in http://rpm.org/wiki/Releases/4.14.0 (Oct 12 2017)
It was all upstreamed with RPM 4.14, and previously existed as Fedora-specific patches on top of our usage of dwz (minidebuginfo). The RPM 4.14 release was when Mark Wielaard and I worked on upstreaming and integrating Fedora and SUSE debuginfo changes. So Fedora's minidebuginfo work[1] and other enhancements[2] along with a completely rewritten implementation of SUSE's split debuginfo[2] were merged together and mainlined into RPM for 4.14. One specific thing was --keep-section as a find-debuginfo.sh option for rust[4]. In Fedora, our go compiler and %gobuild macro have slight tweaks to make it possible to correctly generate debuginfo (we force a build-id to be made, and we have an extension for golang that ensures that crashes give useful traceback output). [1]: https://fedoraproject.org/wiki/Features/MiniDebugInfo [2]: https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo [3]: https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo [4]: https://bugzilla.redhat.com/show_bug.cgi?id=1465997
This works fine with golang and gcc-go, in my experience. If OBS is doing stripping separately from rpmbuild, then I don't know whether the binaries would be sane after that. But at least RPM itself should be fine.
Also helpful, thanks. Do you know of a Fedora package using gcc-go that I could look at?
Unfortunately, since we dropped ppc64be for Go a few releases back, we don't have anything anyone that builds by default with it. But it's trivial to force something to be built with gcc-go in Fedora-style Go packaging. -- 真実はいつも一つ!/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
----- Original Message -----
From: "Neal Gompa"
To: jkowalczyk@suse.com Cc: opensuse-go@opensuse.org Sent: Friday, November 9, 2018 12:43:47 PM Subject: Re: [opensuse-go] Stripping of Go binaries On Fri, Nov 9, 2018 at 12:25 AM Jeff Kowalczyk
wrote: Neal Gompa writes:
In Fedora, we've allowed automatic stripping of binaries for debuginfo generation on Go for at least the last three years. We've also fixed debuginfo generation in RPM upstream so that binaries with weird data like Rust and Go don't break when we strip and put it aside for debuginfo packages.
Thanks for that information, Neal. Do you know the approximate RPM version/date where go and rust -friendly debuginfo support landed, and what nomenclature was used to describe it? I see debuginfo features in http://rpm.org/wiki/Releases/4.14.0 (Oct 12 2017)
It was all upstreamed with RPM 4.14, and previously existed as Fedora-specific patches on top of our usage of dwz (minidebuginfo). The RPM 4.14 release was when Mark Wielaard and I worked on upstreaming and integrating Fedora and SUSE debuginfo changes. So Fedora's minidebuginfo work[1] and other enhancements[2] along with a completely rewritten implementation of SUSE's split debuginfo[2] were merged together and mainlined into RPM for 4.14. One specific thing was --keep-section as a find-debuginfo.sh option for rust[4]. In Fedora, our go compiler and %gobuild macro have slight tweaks to make it possible to correctly generate debuginfo (we force a build-id to be made, and we have an extension for golang that ensures that crashes give useful traceback output).
[1]: https://fedoraproject.org/wiki/Features/MiniDebugInfo [2]: https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo [3]: https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo [4]: https://bugzilla.redhat.com/show_bug.cgi?id=1465997
This works fine with golang and gcc-go, in my experience. If OBS is doing stripping separately from rpmbuild, then I don't know whether the binaries would be sane after that. But at least RPM itself should be fine.
Also helpful, thanks. Do you know of a Fedora package using gcc-go that I could look at?
Unfortunately, since we dropped ppc64be for Go a few releases back, we don't have anything anyone that builds by default with it. But it's trivial to force something to be built with gcc-go in Fedora-style Go packaging.
We never really used gcc-go in Fedora much, apart for bootstrapping GC. IIRC we have used gcc-go transiently(~1 release) for s390x before it got support in upstream GC. Main reason for dropping Go stack on ppc64le, along with maintenance overhead of gcc-go, has been change in minimal ISA version support in GC(IIRC from power6 to power8). JC
-- 真実はいつも一つ!/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
-- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
On Fri, Nov 9, 2018 at 7:09 AM Jakub Cajka
----- Original Message -----
From: "Neal Gompa"
To: jkowalczyk@suse.com Cc: opensuse-go@opensuse.org Sent: Friday, November 9, 2018 12:43:47 PM Subject: Re: [opensuse-go] Stripping of Go binaries On Fri, Nov 9, 2018 at 12:25 AM Jeff Kowalczyk
wrote: Neal Gompa writes:
In Fedora, we've allowed automatic stripping of binaries for debuginfo generation on Go for at least the last three years. We've also fixed debuginfo generation in RPM upstream so that binaries with weird data like Rust and Go don't break when we strip and put it aside for debuginfo packages.
Thanks for that information, Neal. Do you know the approximate RPM version/date where go and rust -friendly debuginfo support landed, and what nomenclature was used to describe it? I see debuginfo features in http://rpm.org/wiki/Releases/4.14.0 (Oct 12 2017)
It was all upstreamed with RPM 4.14, and previously existed as Fedora-specific patches on top of our usage of dwz (minidebuginfo). The RPM 4.14 release was when Mark Wielaard and I worked on upstreaming and integrating Fedora and SUSE debuginfo changes. So Fedora's minidebuginfo work[1] and other enhancements[2] along with a completely rewritten implementation of SUSE's split debuginfo[2] were merged together and mainlined into RPM for 4.14. One specific thing was --keep-section as a find-debuginfo.sh option for rust[4]. In Fedora, our go compiler and %gobuild macro have slight tweaks to make it possible to correctly generate debuginfo (we force a build-id to be made, and we have an extension for golang that ensures that crashes give useful traceback output).
[1]: https://fedoraproject.org/wiki/Features/MiniDebugInfo [2]: https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo [3]: https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo [4]: https://bugzilla.redhat.com/show_bug.cgi?id=1465997
This works fine with golang and gcc-go, in my experience. If OBS is doing stripping separately from rpmbuild, then I don't know whether the binaries would be sane after that. But at least RPM itself should be fine.
Also helpful, thanks. Do you know of a Fedora package using gcc-go that I could look at?
Unfortunately, since we dropped ppc64be for Go a few releases back, we don't have anything anyone that builds by default with it. But it's trivial to force something to be built with gcc-go in Fedora-style Go packaging.
We never really used gcc-go in Fedora much, apart for bootstrapping GC. IIRC we have used gcc-go transiently(~1 release) for s390x before it got support in upstream GC. Main reason for dropping Go stack on ppc64le, along with maintenance overhead of gcc-go, has been change in minimal ISA version support in GC(IIRC from power6 to power8).
You mean ppc64be (big endian), because we still support ppc64le (little endian). -- 真実はいつも一つ!/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
----- Original Message -----
From: "Neal Gompa"
To: jcajka@redhat.com Cc: jkowalczyk@suse.com, opensuse-go@opensuse.org Sent: Friday, November 9, 2018 1:11:00 PM Subject: Re: [opensuse-go] Stripping of Go binaries On Fri, Nov 9, 2018 at 7:09 AM Jakub Cajka
wrote: ----- Original Message -----
From: "Neal Gompa"
To: jkowalczyk@suse.com Cc: opensuse-go@opensuse.org Sent: Friday, November 9, 2018 12:43:47 PM Subject: Re: [opensuse-go] Stripping of Go binaries On Fri, Nov 9, 2018 at 12:25 AM Jeff Kowalczyk
wrote: Neal Gompa writes:
In Fedora, we've allowed automatic stripping of binaries for debuginfo generation on Go for at least the last three years. We've also fixed debuginfo generation in RPM upstream so that binaries with weird data like Rust and Go don't break when we strip and put it aside for debuginfo packages.
Thanks for that information, Neal. Do you know the approximate RPM version/date where go and rust -friendly debuginfo support landed, and what nomenclature was used to describe it? I see debuginfo features in http://rpm.org/wiki/Releases/4.14.0 (Oct 12 2017)
It was all upstreamed with RPM 4.14, and previously existed as Fedora-specific patches on top of our usage of dwz (minidebuginfo). The RPM 4.14 release was when Mark Wielaard and I worked on upstreaming and integrating Fedora and SUSE debuginfo changes. So Fedora's minidebuginfo work[1] and other enhancements[2] along with a completely rewritten implementation of SUSE's split debuginfo[2] were merged together and mainlined into RPM for 4.14. One specific thing was --keep-section as a find-debuginfo.sh option for rust[4]. In Fedora, our go compiler and %gobuild macro have slight tweaks to make it possible to correctly generate debuginfo (we force a build-id to be made, and we have an extension for golang that ensures that crashes give useful traceback output).
[1]: https://fedoraproject.org/wiki/Features/MiniDebugInfo [2]: https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo [3]: https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo [4]: https://bugzilla.redhat.com/show_bug.cgi?id=1465997
This works fine with golang and gcc-go, in my experience. If OBS is doing stripping separately from rpmbuild, then I don't know whether the binaries would be sane after that. But at least RPM itself should be fine.
Also helpful, thanks. Do you know of a Fedora package using gcc-go that I could look at?
Unfortunately, since we dropped ppc64be for Go a few releases back, we don't have anything anyone that builds by default with it. But it's trivial to force something to be built with gcc-go in Fedora-style Go packaging.
We never really used gcc-go in Fedora much, apart for bootstrapping GC. IIRC we have used gcc-go transiently(~1 release) for s390x before it got support in upstream GC. Main reason for dropping Go stack on ppc64le,
^^ ppc64be should be here
along with maintenance overhead of gcc-go, has been change in minimal ISA version support in GC(IIRC from power6 to power8).
You mean ppc64be (big endian), because we still support ppc64le (little endian).
Yup, writing something else than thinking about :) JC
-- 真実はいつも一つ!/ Always, there's only one truth!
-- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
Aleksa Sarai writes:
The official Go compiler has fewer problems, but there definitely is evidence from less than 5 years ago that strip will break stack traces -- we disabled OBS stripping specifically because gcc-go stack traces were broken on s390x and ppc64le (back before Go supported them in their main compiler).
Good point. I would be interested in cataloging the set of (package, OBS target distro, platform, lifecycle) combinations that use gcc-go. Not to take anything away from the value and achievement of gcc-go. Only that we may increasingly find our OBS tooling support evolves in different ways for Go and gcc-go. Jeff -- Jeff Kowalczyk Software Engineer, Go Developer Experience SUSE Linux -- To unsubscribe, e-mail: opensuse-go+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-go+owner@opensuse.org
On 2018-11-07, Matthias Bach
The Packaging Go page in the Wiki [1] recommends to use the %{go_nostrip} macro to prevent issues with Go tooling. However, it is unclear to me whether this recommendation should also be followed for application binaries. After all, the page mostly focuses on the, now deprecated, development packages. For these I fully see the purpose of unstripped binaries. However, for normal applications it seems to me stripping would make sense, as it reduces binary size and it is unlikely people will run tooling on those.
So what's the best practice regarding stripping application binaries?
The best practice is to strip using "go build -s". The reason why we have go_nostrip is because OBS will try to strip binaries to generate debuginfo, and this simply doesn't work with Go binaries (it can break them very badly because GNU strip will happily modify binaries it doesn't understand the structure of -- I submitted some patches for the OBS script to fix this but they weren't accepted for more than a year now). This was a bigger problem with gcc-go, where stripping would result in backtraces being completely broken -- but even with the regular go compiler this is something upstream doesn't support. In particular, the no-strip options are very important for the Go compiler because we ship a bunch of .a files that GNU strip will ruin the metadata of (it changes .__PKGDEF or something like that). -- Aleksa Sarai Senior Software Engineer (Containers) SUSE Linux GmbH https://www.cyphar.com/
participants (5)
-
Aleksa Sarai
-
Jakub Cajka
-
Jeff Kowalczyk
-
Matthias Bach
-
Neal Gompa