[Bug 1140896] New: lto causes minor variations in games/warsow binaries

older
[Bug 1128853] yast2 doesn't handle...

bugzilla_noreply＠novell.com

9 Jul 2019 9 Jul '19

14:49

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 Bug ID: 1140896 Summary: lto causes minor variations in games/warsow binaries Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: openSUSE Factory Status: NEW Severity: Minor Priority: P5 - None Component: Development Assignee: martin.liska@suse.com Reporter: bwiedemann@suse.com QA Contact: qa-bugs@suse.de Found By: Development Blocker: --- While working on reproducible builds for openSUSE, I found that building the games/warsow package with https://build.opensuse.org/request/show/714250 applied still gave minor variations from variations in KVM CPU count. These are ignored by build-compare but prevent us from reaching the goal of bit-identical reproducibility: /usr/bin/warsow: only difference was in build-id, gnu_debuglink or gnu_debugdata , GOOD. /usr/bin/warsow-server: only difference was in build-id, gnu_debuglink or gnu_de bugdata, GOOD. /usr/bin/warsow-tv-server: only difference was in build-id, gnu_debuglink or gnu _debugdata, GOOD. /usr/lib64/warsow/libs/libangelwrap_x86_64.so: only difference was in build-id, gnu_debuglink or gnu_debugdata, GOOD. [snipped more .so files] These variations went away with %define _lto_cflags %{nil} or %define _lto_cflags -flto=2 To reproduce (untested): osc co games/warsow && cd $_ for N in 1 4 ; do osc build --keep-pkg=.rpms$N --vm-type=kvm -j$N \ --define='%clamp_mtime_to_source_date_epoch Y' \ --define='%use_source_date_epoch_as_buildtime Y' done md5sum .rpms*/*.rpm -- You are receiving this mail because: You are on the CC list for the bug.

Attachments:

attachment.htm (text/html — 3.3 KB)

Show replies by date

bugzilla_noreply＠novell.com

9 Jul 9 Jul

14:59

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c1 --- Comment #1 from Bernhard Wiedemann --- somehow -flto=4 appears twice in the build call: + [ 22%] Linking CXX shared library ../libs/libsteamlib_x86_64.so + cd /home/abuild/rpmbuild/BUILD/warsow_21_sdk/source/source/build/steamlib && /usr/bin/cmake -E cmake_link_script CMakeFiles/steamlib.dir/link.txt --verbose=1 + /usr/bin/c++ -fPIC -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -flto=4 -DNDEBUG -pipe -Wall -Wno-unused-function -fvisibility=hidden -Winvalid-pch -std=c++0x -O2 -g -DNDEBUG -flto=4 -Wl,--as-needed -Wl,--no-undefined -Wl,-z,now -Wl,--as-needed -shared -Wl,-soname,libsteamlib_x86_64.so -o ../libs/libsteamlib_x86_64.so CMakeFiles/steamlib.dir/steamlib.cpp.o CMakeFiles/steamlib.dir/steamlib_main.cpp.o CMakeFiles/steamlib.dir/steamlib_syscalls.cpp.o -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

10 Jul 10 Jul

09:33

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c2 Bernhard Wiedemann changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mls@suse.com --- Comment #2 from Bernhard Wiedemann --- games/Veraball is also affected: filterdiff 'rpm -qp --qf %{OPTFLAGS}' */Veraball-20160127-0.0.no*rpm|less --- filter/RPMS.2017/Veraball-20160127-0.0.noarch.rpm +++ filter/RPMS/Veraball-20160127-0.0.noarch.rpm @@ -1 +1 @@ --O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -flto=4 \ No newline at end of file +-O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -flto=1 Do we need to store the OPTFLAGS in rpm headers? Would always using -flto=1 really be worse? or could we invent a -flto=max which is constant but could do the right thing in gcc/binutils -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

09:59

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c3 Bernhard Wiedemann changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |1133809 --- Comment #3 from Bernhard Wiedemann --- -flto seems to affect many/all packages. atftp is nicely small+fast to build for experimenting. I workarounded it for now in https://build.opensuse.org/project/prjconf/home:bmwiedemann:reproducible -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

10:42

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c4 --- Comment #4 from Martin Liška --- (In reply to Bernhard Wiedemann from comment #3)

...

-flto seems to affect many/all packages. atftp is nicely small+fast to build for experimenting.

I workarounded it for now in https://build.opensuse.org/project/prjconf/home:bmwiedemann:reproducible

The workaround is not desirable as we want to use -flto=N with maximum possible parallelism. The original difference comes from: $ objdump -s ./1/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64 > /tmp/1 $ objdump -s ./2/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64 > /tmp/2 $ diff -u /tmp/1 /tmp/2 --- /tmp/1 2019-07-10 12:40:26.746758800 +0200 +++ /tmp/2 2019-07-10 12:40:28.978729823 +0200 @@ -1,10 +1,10 @@ -./1/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64: file format elf64-x86-64 +./2/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64: file format elf64-x86-64 Contents of section .note.gnu.build-id: 0000 04000000 14000000 03000000 474e5500 ............GNU. - 0010 0689672c 6a05b2ea 038e57e3 713069d4 ..g,j.....W.q0i. - 0020 a8f0fa4d ...M + 0010 b3c21a53 324c14ca 20e85ee3 b5f389da ...S2L.. .^..... + 0020 23ceccae #... Contents of section .debug_info: 0000 3b000000 04000000 00000811 00000000 ;............... 0010 63000000 45040799 15000045 01080510 c...E......E.... @@ -810,7 +810,7 @@ 02e0 726f6e6f 75732d75 6e77696e 642d7461 ronous-unwind-ta 02f0 626c6573 202d6673 7461636b 2d636c61 bles -fstack-cla 0300 73682d70 726f7465 6374696f 6e202d66 sh-protection -f - 0310 6c746f3d 31202d66 676e7538 392d696e lto=1 -fgnu89-in + 0310 6c746f3d 32202d66 676e7538 392d696e lto=2 -fgnu89-in 0320 6c696e65 202d6650 4945005f 494f5f46 line -fPIE._IO_F 0330 494c4500 49505052 4f544f5f 47524500 ILE.IPPROTO_GRE. 0340 49505052 4f544f5f 5544504c 49544500 IPPROTO_UDPLITE. Where .note.gnu.build-id HASH is different due to differences in .debug_info section. Let me investigate that. -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

10:48

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c5 --- Comment #5 from Martin Liška --- Which is cause by: $ eu-readelf --debug-dump=info a.out | less ... Compilation unit at offset 144: Version: 4, Abbreviation section offset: 91, Address size: 8, Offset size: 4 [ 9b] compile_unit abbrev: 1 producer (strp) "GNU C17 9.1.1 20190611 [gcc-9-branch revision 272147] -mtune=generic -march=x86-64 -g -flto=1234567890" language (data1) C99 (12) ... So it's encoded in debug info in producer ;) Let me investigate that.. -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

11:09

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c6 Martin Liška changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |IN_PROGRESS --- Comment #6 from Martin Liška --- I've got a patch candidate for it: https://gcc.gnu.org/ml/gcc-patches/2019-07/msg00786.html -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

13 Sep 13 Sep

08:03

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c7 Bernhard Wiedemann changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dleuenberger@suse.com --- Comment #7 from Bernhard Wiedemann --- So it seems, Factory's gcc9 now supports -flto=auto but it is not used yet. -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

12:04

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c8 Martin Liška changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bwiedemann@suse.com Flags| |needinfo?(bwiedemann@suse.c | |om) --- Comment #8 from Martin Liška --- (In reply to Bernhard Wiedemann from comment #7)

...

So it seems, Factory's gcc9 now supports -flto=auto but it is not used yet.

I see it being used: https://build.opensuse.org/build/games/openSUSE_Tumbleweed/x86_64/warsow/_lo... Can we close this issue now? -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

13:06

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c9 --- Comment #9 from Bernhard Wiedemann --- I was looking at Factory and -flto=auto still is not used there: https://build.opensuse.org/package/live_build_log/openSUSE:Factory/libuv/sta... I just dont know why. -- You are receiving this mail because: You are on the CC list for the bug.

bugzilla_noreply＠novell.com

18 Dec 18 Dec

12:26

New subject: [Bug 1140896] lto causes minor variations in games/warsow binaries

http://bugzilla.opensuse.org/show_bug.cgi?id=1140896 http://bugzilla.opensuse.org/show_bug.cgi?id=1140896#c10 Bernhard Wiedemann changed: What |Removed |Added ---------------------------------------------------------------------------- Status|IN_PROGRESS |RESOLVED Resolution|--- |FIXED Flags|needinfo?(bwiedemann@suse.c | |om) | --- Comment #10 from Bernhard Wiedemann --- The rpm patch for flto=auto was merged back into Factory on 2019-10-21 https://build.opensuse.org/request/show/732635 -- You are receiving this mail because: You are on the CC list for the bug.

1598

Age (days ago)

1760

Last active (days ago)

List overview

Download

10 comments

1 participants

participants (1)

bugzilla_noreply＠novell.com

[Bug 1140896] New: lto causes minor variations in games/warsow binaries

tags

participants (1)