Hi.
We as toolchain team are working on enablement of LTO in Factory:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
Our goal is to extend the scope to as many packages as possible. We'll achieve that via a new RPM macro: _lto_cflags. We expect that about 5% of packages will have LTO disabled due to various reasons. A new META bug has been created for that and will link all these packages: https://bugzilla.opensuse.org/show_bug.cgi?id=1133084
I'm also planning to collect all relevant information for package maintainers on the following WIKI page: https://en.opensuse.org/openSUSE:LTO
Note that Debian and Gentoo are also trying to adopt LTO in their corresponding ecosystems: https://wiki.debian.org/LTO https://github.com/InBetweenNames/gentooLTO
Martin
On 23. 04. 19, 14:02, Martin Liška wrote:
Our goal is to extend the scope to as many packages as possible. We'll achieve that via a new RPM macro: _lto_cflags.
Wow. Is there a staging prj with that setting yet? So that we can start fixing things...
thanks,
On 4/23/19 4:23 PM, Jiri Slaby wrote:
On 23. 04. 19, 14:02, Martin Liška wrote:
Our goal is to extend the scope to as many packages as possible. We'll achieve that via a new RPM macro: _lto_cflags.
Wow. Is there a staging prj with that setting yet? So that we can start fixing things...
I've got an almost working staging project: https://build.opensuse.org/project/show/openSUSE:Factory:Staging:N Note that I disabled LTO for about 140 packages in the project.
My plan is to first disable LTO for the packages that fail. It would be very difficult task to migrate all packages to LTO first. Then we can get LTO enabled in Factory and later the problematic packages can be adopted to LTO. For some legacy stuff it does not make much sense to spend an extra time with the porting.
Martin
thanks,
Hello.
Cool job. Thank you very much.
Dne 23. 04. 19 v 14:02 Martin Liška napsal(a):
Hi.
We as toolchain team are working on enablement of LTO in Factory:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
Our goal is to extend the scope to as many packages as possible. We'll achieve that via a new RPM macro: _lto_cflags. We expect that about 5% of packages will have LTO disabled due to various reasons. A new META bug has been created for that and will link all these packages: https://bugzilla.opensuse.org/show_bug.cgi?id=1133084
I'm also planning to collect all relevant information for package maintainers on the following WIKI page: https://en.opensuse.org/openSUSE:LTO
Note that Debian and Gentoo are also trying to adopt LTO in their corresponding ecosystems: https://wiki.debian.org/LTO https://github.com/InBetweenNames/gentooLTO
Martin
Hi,
On Tue, Apr 23, Martin Liška wrote:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
If I read that LTO doesn't work with symbol versioning, isn't then introducing LTO contra productive? We need much more shared libraries with symbol versioning then less or LTO ...
Thorsten
On Thursday 2019-04-25 09:23, Thorsten Kukuk wrote:
On Tue, Apr 23, Martin Liška wrote:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
If I read that LTO doesn't work with symbol versioning, isn't then introducing LTO contra productive? We need much more shared libraries with symbol versioning then less or LTO ...
And what about the extra memory requirements of LTO? We're notoriously short on that in the build service.
On Thu, Apr 25, Jan Engelhardt wrote:
On Thursday 2019-04-25 09:23, Thorsten Kukuk wrote:
On Tue, Apr 23, Martin Liška wrote:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
If I read that LTO doesn't work with symbol versioning, isn't then introducing LTO contra productive? We need much more shared libraries with symbol versioning then less or LTO ...
And what about the extra memory requirements of LTO? We're notoriously short on that in the build service.
Memory requirements can be solved by other ways, crashing applications due to symbol conflicts of different libraries not.
Thorsten
On 4/25/19 9:25 PM, Jan Engelhardt wrote:
On Thursday 2019-04-25 09:23, Thorsten Kukuk wrote:
On Tue, Apr 23, Martin Liška wrote:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
If I read that LTO doesn't work with symbol versioning, isn't then introducing LTO contra productive? We need much more shared libraries with symbol versioning then less or LTO ...
And what about the extra memory requirements of LTO? We're notoriously short on that in the build service.
I know, according to experiments I've done we don't face the issue. Moreover, GCC 9 was significantly improved and LTO mode is not consuming so much memory.
Martin
On 4/25/19 9:23 AM, Thorsten Kukuk wrote:
Hi,
On Tue, Apr 23, Martin Liška wrote:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
If I read that LTO doesn't work with symbol versioning, isn't then introducing LTO contra productive?
No. Right now we're talking about 15-20 packages that use symbol versioning (out of ~3000). Moreover, LTO is the most beneficial for executables of a reasonable size where scope of exported symbols is limited. On the contrary, libraries have a lot of symbols (functions) exported.
Note that we're planning to add support for symber in next GCC release.
We need much more shared libraries with symbol versioning then less or LTO ...
Yes.
Martin
Thorsten
On Mon, 29 Apr 2019, Martin Liška wrote:
On 4/25/19 9:23 AM, Thorsten Kukuk wrote:
Hi,
On Tue, Apr 23, Martin Liška wrote:
LTO stands for Link Time Optimization and it is a GCC optimization technique that improves speed and reduces size of binaries. According to our measurements, ELF binaries will be about 5% smaller and debug info packages by 15%. Now, there are various interesting packages that have been LTO in Factory right now: libreoffice, MozillaFirefox, python3, gcc9.
If I read that LTO doesn't work with symbol versioning, isn't then introducing LTO contra productive?
No. Right now we're talking about 15-20 packages that use symbol versioning (out of ~3000). Moreover, LTO is the most beneficial for executables of a reasonable size where scope of exported symbols is limited. On the contrary, libraries have a lot of symbols (functions) exported.
Note that we're planning to add support for symber in next GCC release.
IIRC symbol-versioning with -Wl,-map should work just fine, just the glibc way of using toplevel asms does not. Hopefully not too many projects copied that way.
For C++ restricting visibility is something that works as well and is similarly useful (just w/o the versions).
We need much more shared libraries with symbol versioning then less or LTO ...
Yes.
I think we need more (C++) shared libraries that make their set of exported symbols explicit. Not sure if we really need more libraries with several variants of the same symbol ;)
Richard.
Martin
Thorsten
Hello.
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default. The snapshot survived the full openQA testing and I've been using the snapshot on both my machines. Users can look forward to smaller and faster binaries. Right now, we've been collecting statistics and we're planning to write a more detail blog post about the benefits.
The enablement wouldn't be possible without a great effort of the people who take of openSUSE:Factory. I thank them for the nice cooperation and special thanks goes to DimStar who was very patient with me.
We are still expecting a fallout of the change and we'll try to fix the potentially broken package quickly. Another unresolved topic are reproducible builds on which we are cooperating with Bernhard Wiedemann.
In order to make the announcement complete, users can expect Chromium package using LTO and a new Firefox release will newly utilize profile guided optimization (PGO).
Martin
[1] https://review.tumbleweed.boombatower.com/2019/07/03/release.html
Sorry for bad link provided. The correct snapshot is Build20190713:
https://openqa.opensuse.org/tests/overview?distri=microos&distri=opensus...
Martin
Am 17.07.19 um 10:22 schrieb Martin Liška:
Hello.
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default. The snapshot survived the full openQA testing and I've been using the snapshot on both my machines. Users can look forward to smaller and faster binaries. Right now, we've been collecting statistics and we're planning to write a more detail blog post about the benefits.
The enablement wouldn't be possible without a great effort of the people who take of openSUSE:Factory. I thank them for the nice cooperation and special thanks goes to DimStar who was very patient with me.
We are still expecting a fallout of the change and we'll try to fix the potentially broken package quickly. Another unresolved topic are reproducible builds on which we are cooperating with Bernhard Wiedemann.
In order to make the announcement complete, users can expect Chromium package using LTO and a new Firefox release will newly utilize profile guided optimization (PGO).
Martin
[1] https://review.tumbleweed.boombatower.com/2019/07/03/release.html
could you also give a link with explanations on what LTO is? Google didn't help me thanks Karl
On 17/07/2019 11.28, Karl Sinn wrote:
Am 17.07.19 um 10:22 schrieb Martin Liška:
Hello.
...
could you also give a link with explanations on what LTO is? Google didn't help me
Probably this: https://en.opensuse.org/openSUSE:LTO
Am 17.07.19 um 11:34 schrieb Carlos E. R.:
On 17/07/2019 11.28, Karl Sinn wrote:
Am 17.07.19 um 10:22 schrieb Martin Liška:
Hello.
...
could you also give a link with explanations on what LTO is? Google didn't help me
Probably this: https://en.opensuse.org/openSUSE:LTO
thanks for the links (both of them) sounds good :)
On 17/07/2019 12.33, Karl Sinn wrote:
Am 17.07.19 um 11:34 schrieb Carlos E. R.:
On 17/07/2019 11.28, Karl Sinn wrote:
Am 17.07.19 um 10:22 schrieb Martin Liška:
Hello.
...
could you also give a link with explanations on what LTO is? Google didn't help me
Probably this: https://en.opensuse.org/openSUSE:LTO
thanks for the links (both of them) sounds good :)
Reminds me of the Smart Linking feature of Borland Pascal many years ago - around 1995 :-)
Basically the link phase would include only code actually used by the programs, not the entire libraries (this required a different object file format with the necessary information). It was thus curious that a helloworld.pas would occupy 2KB when built, while helloworld.c resulted 16 or 30 K binary.
I'm hope that LTO succeeds and we can see it in Leap.
Hello,
On Jul 17 11:28 Karl Sinn wrote (excerpt):
could you also give a link with explanations on what LTO is? Google didn't help me
use https://en.wikipedia.org/wiki/LTO and choose ;-)
Kind Regards Johannes Meixner
On 17/07/2019 10.22, Martin Liška wrote:
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default.
Great work. Thanks for all the effort you put in there.
Users can look forward to smaller and faster binaries.
Here are some numbers for the "smaller" part for i586+x86_64+noarch:
du -s 92569840 20190708 90485340 20190713
So the total collection is 2.3% smaller.
and the file count even went up: find 20190708 |wc -l 56596 find 20190713 |wc -l 57007
and some individual comparisons:
584548 Jun 12 19:00 20190708/tumbleweed/repo/oss/x86_64/bash-5.0-7.3.x86_64.rpm 571504 Jul 9 13:55 20190713/tumbleweed/repo/oss/x86_64/bash-5.0-7.4.x86_64.rpm uncompressed /usr/bin/bash is 1106848 => 1070200
320792 Jun 13 22:57 20190708/tumbleweed/repo/oss/x86_64/curl-7.65.1-1.1.x86_64.rpm 317700 Jul 9 15:22 20190713/tumbleweed/repo/oss/x86_64/curl-7.65.1-1.2.x86_64.rpm uncompressed /usr/bin/curl is 232632 => 224440
25371464 Jun 26 19:28 20190708/tumbleweed/repo/oss/x86_64/gcc9-9.1.1+r272147-1.1.x86_64.rpm 25365880 Jul 9 13:18 20190713/tumbleweed/repo/oss/x86_64/gcc9-9.1.1+r272147-1.2.x86_64.rpm
80580 Jun 13 01:41 20190708/tumbleweed/repo/oss/x86_64/zsync-0.6.2-24.20.x86_64.rpm 78816 Jul 9 13:28 20190713/tumbleweed/repo/oss/x86_64/zsync-0.6.2-24.21.x86_64.rpm
93072 Jun 13 01:41 /usr/bin/zsync 97016 Jun 13 01:41 /usr/bin/zsyncmake 93072 Jul 9 13:28 /usr/bin/zsync 80600 Jul 9 13:28 /usr/bin/zsyncmake
53060 Jun 13 01:41 20190708/tumbleweed/repo/oss/x86_64/zsync-devel-0.6.2-24.20.x86_64.rpm 19608 Jul 9 13:28 20190713/tumbleweed/repo/oss/x86_64/zsync-devel-0.6.2-24.21.x86_64.rpm the large diff here is due to libzsync.a file shrinking 98472 => 31886
So the compressed binary rpms in this small sample seem to be 1-2% smaller and uncompressed binaries are 3-4% smaller.
zsync* is the exception with 0% and 20% diff.
Another unresolved topic are reproducible builds on which we are cooperating with Bernhard Wiedemann.
If you are interested, most of this is tracked in: https://bugzilla.opensuse.org/show_bug.cgi?id=1140896 https://bugzilla.opensuse.org/show_bug.cgi?id=1141323
Ciao Bernhard M.
On 7/17/19 10:22 AM, Martin Liška wrote:
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default.
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before snapshot 20190713.
Installing pdns-recursor 4.2.0 from devel project server:dns fixed it.
Now for tracking down whether it's related to LTO or not:
- Is LTO also enabled on all devel projects?
- Should I try to build the pdns-recursor 4.1.13 with %define _lto_cflags %{nil} ?
Ciao, Michael.
On Wed, 2019-07-17 at 12:25 +0200, Michael Ströder wrote:
On 7/17/19 10:22 AM, Martin Liška wrote:
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default.
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before snapshot 20190713.
Installing pdns-recursor 4.2.0 from devel project server:dns fixed it.
Now for tracking down whether it's related to LTO or not:
Is LTO also enabled on all devel projects?
Should I try to build the pdns-recursor 4.1.13 with %define _lto_cflags %{nil} ?
LTO flags are injected into Optflags in openSUSE:Factory - so every project building against openSUSE:Factory/{standard,snapshot} has LTO enabled - UNLESS the devel project does some weird stuff to disable it again. In this case, shame on the devel project (I'm not aware of any such project configuration, so this is really just hypothetical)
Disabling LTO for the package can be a first test to identify the root cause - but it should only be used as a short term solution, preferably fixing the underlying issue.
Cheers Dominique
On 7/17/19 12:25 PM, Michael Ströder wrote:
On 7/17/19 10:22 AM, Martin Liška wrote:
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default.
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before snapshot 20190713.
Can you please create a bugzilla issue for it?
Installing pdns-recursor 4.2.0 from devel project server:dns fixed it.
Now for tracking down whether it's related to LTO or not:
Is LTO also enabled on all devel projects?
Should I try to build the pdns-recursor 4.1.13 with %define _lto_cflags %{nil} ?
That does not work for some reason. It looks the package inherits somehow $optflags from a different package.
Martin
Ciao, Michael.
On 22/07/2019 13:31, Martin Liška wrote:
On 7/17/19 12:25 PM, Michael Ströder wrote:
On 7/17/19 10:22 AM, Martin Liška wrote:
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default.
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before snapshot 20190713.
Can you please create a bugzilla issue for it?
Installing pdns-recursor 4.2.0 from devel project server:dns fixed it.
Now for tracking down whether it's related to LTO or not:
Is LTO also enabled on all devel projects?
Should I try to build the pdns-recursor 4.1.13 with %define _lto_cflags %{nil} ?
That does not work for some reason. It looks the package inherits somehow $optflags from a different package.
Martin
Ciao, Michael.
Before somebody pointed me to the wiki, I used sed to kill -flto= in the optflags: #Isolate number of threads %define tds %(rpm --eval %_smp_mflags|cut -b3) and #optflags filter out -flt= flag CXXFLAGS="`echo $CXXFLAGS|sed -- 's/-flto=%{tds}//'`" %define optflags $CXXFLAGS
I've just tried the %define _lto_cflags %{nil} in the build section of jack and it works for me.
Jack is also segfaulting with LTO see: https://bugzilla.suse.com/show_bug.cgi?id=1141762
LTO also appears to give build failure to ARM builds.
Regards Dave P
-----Original Message----- From: Dave Plater dplater.list@gmail.com Sent: 24 July 2019 10:56 To: opensuse-factory@opensuse.org Subject: Re: [opensuse-factory] LTO is enabled in Tumbleweed
On 22/07/2019 13:31, Martin Liška wrote:
On 7/17/19 12:25 PM, Michael Ströder wrote:
On 7/17/19 10:22 AM, Martin Liška wrote:
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default.
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before
snapshot 20190713.
Can you please create a bugzilla issue for it?
Installing pdns-recursor 4.2.0 from devel project server:dns fixed it.
Now for tracking down whether it's related to LTO or not:
Is LTO also enabled on all devel projects?
Should I try to build the pdns-recursor 4.1.13 with %define _lto_cflags %{nil} ?
That does not work for some reason. It looks the package inherits somehow $optflags from a different package.
Martin
Ciao, Michael.
Before somebody pointed me to the wiki, I used sed to kill -flto= in the optflags: #Isolate number of threads %define tds %(rpm --eval %_smp_mflags|cut -b3) and #optflags filter out - flt= flag CXXFLAGS="`echo $CXXFLAGS|sed -- 's/-flto=%{tds}//'`" %define optflags $CXXFLAGS
I've just tried the %define _lto_cflags %{nil} in the build section of jack and it works for me.
Jack is also segfaulting with LTO see: https://bugzilla.suse.com/show_bug.cgi?id=1141762
LTO also appears to give build failure to ARM builds.
One known problem on ARM is http://bugzilla.suse.com/show_bug.cgi?id=1140918
Cheers, Guillaume
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. N�����r��y隊Z)z{.���r�+�맲��r��z�^�ˬz��N�(�֜��^� ޭ隊Z)z{.���r�+��0�����Ǩ�
On 7/17/19 12:25 PM, Michael Ströder wrote:
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before snapshot 20190713.
Upstream issue is here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91170
but I can't reproduce it. Michael can you please update the steps needed for exposure of the issue?
Thanks, Martin
On 7/23/19 12:34 PM, Martin Liška wrote:
On 7/17/19 12:25 PM, Michael Ströder wrote:
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before snapshot 20190713.
Upstream issue is here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91170
but I can't reproduce it. Michael can you please update the steps needed for exposure of the issue?
As said I only had this issue when using pdns-recursor 4.1.13 on Tumbleweed, but not with 4.2.0. Since 4.2.0 should reach Tumbleweed really soon I don't know whether it's worth the effort.
Ciao, Michael.
On Tue, 2019-07-23 at 12:45 +0200, Michael Ströder wrote:
As said I only had this issue when using pdns-recursor 4.1.13 on Tumbleweed, but not with 4.2.0. Since 4.2.0 should reach Tumbleweed really soon I don't know whether it's worth the effort.
'soon' was 5 days ago - pdns-recursor 4.2.0 has been part of snapshot 0718
Cheers Dominique
On 7/23/19 12:55 PM, Dominique Leuenberger / DimStar wrote:
On Tue, 2019-07-23 at 12:45 +0200, Michael Ströder wrote:
As said I only had this issue when using pdns-recursor 4.1.13 on Tumbleweed, but not with 4.2.0. Since 4.2.0 should reach Tumbleweed really soon I don't know whether it's worth the effort.
'soon' was 5 days ago - pdns-recursor 4.2.0 has been part of snapshot 0718
Even better. ;-)
(I was confused by package 'pdns-4.1.10' when writing my answer.)
Ciao, Michael.
On Tue, 23 Jul 2019, Martin Liška wrote:
On 7/17/19 12:25 PM, Michael Ströder wrote:
I see some issue with pdns-recursor 4.1.13 getting restarted by systemd all the time due to seg fault. Starting directly on the command-line also leads to seg fault. IIRC this used to work before snapshot 20190713.
Upstream issue is here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91170
but I can't reproduce it. Michael can you please update the steps needed for exposure of the issue?
I have opened this issue because darix ran into it and reported it to me privately. So maybe he can double-check / clarify.
Richard.
On 7/17/19 10:22 AM, Martin Liška wrote:
Hello.
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default. The snapshot survived the full openQA testing and I've been using the snapshot on both my machines. Users can look forward to smaller and faster binaries. Right now, we've been collecting statistics and we're planning to write a more detail blog post about the benefits.
The enablement wouldn't be possible without a great effort of the people who take of openSUSE:Factory. I thank them for the nice cooperation and special thanks goes to DimStar who was very patient with me.
We are still expecting a fallout of the change and we'll try to fix the potentially broken package quickly. Another unresolved topic are reproducible builds on which we are cooperating with Bernhard Wiedemann.
In order to make the announcement complete, users can expect Chromium package using LTO and a new Firefox release will newly utilize profile guided optimization (PGO).
Martin
[1] https://review.tumbleweed.boombatower.com/2019/07/03/release.html
For additional benchmark-driven analysis of LTO in GCC 9 check out Jan Hubička's blog[0].
[0]: https://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html
Hi,
On 17/07/2019 10:22, Martin Liška wrote:
Hello.
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default.
Thanks for the heads up! I was wondering why some of my packages started to fail in OBS, and then I remembered this thread. Disabling LTO as a workaround solved the problem for now. One of the affected upstream already received a github issue, for the other one I will need to package the latest version before submitting one...
Bye,
CzP
17.07.2019 11:22, Martin Liška пишет:
Hello.
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default. The snapshot survived the full openQA testing and I've been using the snapshot on both my machines. Users can look forward to smaller and faster binaries. Right now, we've been collecting statistics and we're planning to write a more detail blog post about the benefits.
The enablement wouldn't be possible without a great effort of the people who take of openSUSE:Factory. I thank them for the nice cooperation and special thanks goes to DimStar who was very patient with me.
We are still expecting a fallout of the change and we'll try to fix the potentially broken package quickly. Another unresolved topic are reproducible builds on which we are cooperating with Bernhard Wiedemann.
In order to make the announcement complete, users can expect Chromium package using LTO and a new Firefox release will newly utilize profile guided optimization (PGO).
Martin
[1] https://review.tumbleweed.boombatower.com/2019/07/03/release.html
I like it, It is funny.
https://bugzilla.opensuse.org/show_bug.cgi?id=1142913
On Thu, Jul 25, 2019 at 10:46 PM Matwey V. Kornilov matwey.kornilov@gmail.com wrote:
17.07.2019 11:22, Martin Liška пишет:
Hello.
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default. The snapshot survived the full openQA testing and I've been using the snapshot on both my machines. Users can look forward to smaller and faster binaries. Right now, we've been collecting statistics and we're planning to write a more detail blog post about the benefits.
Anyone have an issue with wine? After an update at the same time, wine starts with a couple popups with an error. Then the program itself starts and runs. It also happens with winecfg or wineconsole.
No idea if it is specifically the LTO that brought this. But is seems to have started at the same time.
On 2019-07-26 17:32, Roger Oberholtzer wrote:
On Thu, Jul 25, 2019 at 10:46 PM Matwey V. Kornilov matwey.kornilov@gmail.com wrote:
17.07.2019 11:22, Martin Liška пишет:
Hello.
I'm happy to announce that latest ([1]) Tumbleweed release is a first snapshot that uses LTO by default. The snapshot survived the full openQA testing and I've been using the snapshot on both my machines. Users can look forward to smaller and faster binaries. Right now, we've been collecting statistics and we're planning to write a more detail blog post about the benefits.
Anyone have an issue with wine? After an update at the same time, wine starts with a couple popups with an error. Then the program itself starts and runs. It also happens with winecfg or wineconsole.
No idea if it is specifically the LTO that brought this. But is seems to have started at the same time.
No, works as usual on my machine.
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Here's another fallout:
https://build.opensuse.org/package/show/home:frispete:telephony:asterisk/ dahdi-tools
Fails with:
[ 14s] make[3]: Entering directory '/home/abuild/rpmbuild/BUILD/dahdi- tools-3.0.0/xpp/xtalk' [ 14s] CC xlist_test-xlist_test.o [ 14s] CC libxtalk_la-xtalk_sync.lo [ 14s] CC libxtalk_la-xtalk_raw.lo [ 14s] CC libxtalk_la-xtalk_base.lo [ 14s] CC libxtalk_la-debug.lo [ 14s] CC libxtalk_la-xlist.lo [ 14s] CC libxtalk_la-xtalk-xusb.lo [ 14s] CC libxtalk_la-xusb_common.lo [ 14s] CC libxtalk_la-xusb_libusbx.lo [ 14s] CC xusb_test-xusb_test.o [ 14s] CC xusb_test_bypath-xusb_test_bypath.o [ 14s] CC xtalk_test-xtalk_test.o [ 14s] CC xtalk_raw_test-xtalk_raw_test.o [ 14s] CC xtalk_send-xtalk_send.o [ 14s] CCLD libxtalk.la [ 14s] CCLD xusb_test [ 14s] CCLD xlist_test [ 14s] CCLD xusb_test_bypath [ 14s] CCLD xtalk_test [ 14s] CCLD xtalk_raw_test [ 14s] CCLD xtalk_send [ 14s] xtalk_sync.c:63:10: error: type of 'members' does not match original declaration [-Werror=lto-type-mismatch] [ 14s] 63 | } PACKED members; [ 14s] | ^ [ 14s] xtalk_raw.c:51:10: note: 'members' was previously declared here [ 14s] 51 | } PACKED members; [ 14s] | ^ [ 14s] xtalk_raw.c:51:10: note: code may be misoptimized unless '-fno- strict-aliasing' is used [ 14s] lto1: all warnings being treated as errors [ 14s] lto-wrapper: fatal error: gcc returned 1 exit status [ 14s] compilation terminated.
Without lto, it compiles fine.
It throws a couple of "interesting" warnings, but since this is rarely used test code, it's probably not worth it to dig into..
Cheers, Pete
On Jul 25 2019, Hans-Peter Jansen hpj@urpla.net wrote:
Without lto, it compiles fine.
But broken nevertheless. LTO just uncovered it.
Andreas.
Am Donnerstag, 25. Juli 2019, 17:26:27 CEST schrieben Sie:
On Jul 25 2019, Hans-Peter Jansen hpj@urpla.net wrote:
Without lto, it compiles fine.
But broken nevertheless. LTO just uncovered it.
If you could elaborate in which specific way, I could report this to the asterisk developers.
Pete
On Donnerstag, 25. Juli 2019 21:39:37 CEST Hans-Peter Jansen wrote:
Am Donnerstag, 25. Juli 2019, 17:26:27 CEST schrieben Sie:
On Jul 25 2019, Hans-Peter Jansen hpj@urpla.net wrote:
Without lto, it compiles fine.
But broken nevertheless. LTO just uncovered it.
If you could elaborate in which specific way, I could report this to the asterisk developers.
The two files each have a top level variable "members", which have an implicit "extern" storage class. Any other translation unit ("file") could reference it, and the linker would try to resolve it.
As both both TUs end up in the same binary, the linker has two possibilities for each reference.
In this case, probably the best fix is to declare both "members" instances as static.
Kind regards,
Stefan-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am Donnerstag, 25. Juli 2019, 22:43:58 CEST schrieb Brüns, Stefan:
On Donnerstag, 25. Juli 2019 21:39:37 CEST Hans-Peter Jansen wrote:
Am Donnerstag, 25. Juli 2019, 17:26:27 CEST schrieben Sie:
On Jul 25 2019, Hans-Peter Jansen hpj@urpla.net wrote:
Without lto, it compiles fine.
But broken nevertheless. LTO just uncovered it.
If you could elaborate in which specific way, I could report this to the asterisk developers.
The two files each have a top level variable "members", which have an implicit "extern" storage class. Any other translation unit ("file") could reference it, and the linker would try to resolve it.
As both both TUs end up in the same binary, the linker has two possibilities for each reference.
Thanks, Stefan, for your feedback.
In this case, probably the best fix is to declare both "members" instances as static.
It's not that easy (at least with my *very* rusty C knowledge), since xpp/ mpptalk.c uses this union from xtalk_raw.c and xtalk_sync.c.
I solved it by removing the useless members declaration:
Index: b/xpp/mpptalk.c =================================================================== --- a/xpp/mpptalk.c +++ b/xpp/mpptalk.c @@ -237,7 +237,7 @@ union XTALK_PDATA(MPP) { MEMBER(MPP, TWS_PORT_GET_REPLY); MEMBER(MPP, TWS_PWR_GET); MEMBER(MPP, TWS_PWR_GET_REPLY); -} PACKED members; +} PACKED;
/* * Statuses Index: b/xpp/xtalk/xtalk_raw.c =================================================================== --- a/xpp/xtalk/xtalk_raw.c +++ b/xpp/xtalk/xtalk_raw.c @@ -48,7 +48,7 @@ CMD_DEF(XTALK, ACK,
union XTALK_PDATA(XTALK) { MEMBER(XTALK, ACK); -} PACKED members; +} PACKED;
const struct xtalk_protocol xtalk_raw_proto = { .name = "XTALK-RAW", Index: b/xpp/xtalk/xtalk_sync.c =================================================================== --- a/xpp/xtalk/xtalk_sync.c +++ b/xpp/xtalk/xtalk_sync.c @@ -60,7 +60,7 @@ union XTALK_PDATA(XTALK) { MEMBER(XTALK, ACK); MEMBER(XTALK, PROTO_GET); MEMBER(XTALK, PROTO_GET_REPLY); -} PACKED members; +} PACKED;
const struct xtalk_protocol xtalk_sync_proto = { .name = "XTALK-SYNC",
It compiles with lto enabled at least.
What do you think? (I don't have any related hardware, just want asterisk for Tumbleweed..)
Cheers, Pete
On Jul 25 2019, Hans-Peter Jansen hpj@urpla.net wrote:
Am Donnerstag, 25. Juli 2019, 17:26:27 CEST schrieben Sie:
On Jul 25 2019, Hans-Peter Jansen hpj@urpla.net wrote:
Without lto, it compiles fine.
But broken nevertheless. LTO just uncovered it.
If you could elaborate in which specific way, I could report this to the asterisk developers.
The compiler already told you.
Andreas.
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Since that switch, veracrypt doesn't build anymore due to a lto-wrapper error:
https://build.opensuse.org/package/show/security/veracrypt
Any idea anybody?
Cheers, Pete
On Sonntag, 11. August 2019 00:19:19 CEST Hans-Peter Jansen wrote:
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Since that switch, veracrypt doesn't build anymore due to a lto-wrapper error:
https://build.opensuse.org/package/show/security/veracrypt
Any idea anybody?
Cheers, Pete
I suggest replacing VeraCrypt with tc-play and zuluCrypt. VeraCrypt’s code quality is extremely poor and the Linux port does not receive a lot of attention.
On Sun, 11 Aug 2019, Hans-Peter Jansen wrote:
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Since that switch, veracrypt doesn't build anymore due to a lto-wrapper error:
https://build.opensuse.org/package/show/security/veracrypt
Any idea anybody?
[ 246s] ../Crypto/Whirlpool.c:105:45: error: 'Whirlpool_C' causes a section type conflict with 'SHA256_K' [ 246s] 105 | CRYPTOPP_ALIGN_DATA(16) static const uint64 Whirlpool_C[8*256+R] CRYPTOPP_SECTION_ALIGN16 = { [ 246s] | ^ [ 246s] ../Crypto/Sha2.c:321:34: note: 'SHA256_K' was declared here [ 246s] 321 | CRYPTOPP_ALIGN_DATA(16) uint_32t SHA256_K[64] CRYPTOPP_SECTION_ALIGN16 = { [ 246s] | ^
so the question is how CRYPTOPP_SECTION_ALIGN16 / CRYPTOPP_ALIGN_DATA are defined for the respective TUs. One appearant difference is that Whirlpool_C is declared 'const' while SHA256_K is not so if the section is named that might be the issue.
Richard.
On Mon, 12 Aug 2019, Richard Biener wrote:
On Sun, 11 Aug 2019, Hans-Peter Jansen wrote:
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Since that switch, veracrypt doesn't build anymore due to a lto-wrapper error:
https://build.opensuse.org/package/show/security/veracrypt
Any idea anybody?
[ 246s] ../Crypto/Whirlpool.c:105:45: error: 'Whirlpool_C' causes a section type conflict with 'SHA256_K' [ 246s] 105 | CRYPTOPP_ALIGN_DATA(16) static const uint64 Whirlpool_C[8*256+R] CRYPTOPP_SECTION_ALIGN16 = { [ 246s] | ^ [ 246s] ../Crypto/Sha2.c:321:34: note: 'SHA256_K' was declared here [ 246s] 321 | CRYPTOPP_ALIGN_DATA(16) uint_32t SHA256_K[64] CRYPTOPP_SECTION_ALIGN16 = { [ 246s] | ^
so the question is how CRYPTOPP_SECTION_ALIGN16 / CRYPTOPP_ALIGN_DATA are defined for the respective TUs. One appearant difference is that Whirlpool_C is declared 'const' while SHA256_K is not so if the section is named that might be the issue.
OK, so google works and spits out
#ifndef CRYPTOPP_SECTION_ALIGN16 #if defined(__GNUC__) && !defined(__APPLE__) // the alignment attribute doesn't seem to work without this section attribute when -fdata-sections is turned on #define CRYPTOPP_SECTION_ALIGN16 __attribute__((section ("CryptoPP_Align16")))
so indeed a named section. Make SHA256_K 'const' and see if that works (or Whirlpool_C non-const, might be more "mismatches" exist).
Richard.
Am Montag, 12. August 2019, 10:09:15 CEST schrieb Richard Biener:
On Mon, 12 Aug 2019, Richard Biener wrote:
On Sun, 11 Aug 2019, Hans-Peter Jansen wrote:
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Since that switch, veracrypt doesn't build anymore due to a lto-wrapper error:
https://build.opensuse.org/package/show/security/veracrypt
Any idea anybody?
[ 246s] ../Crypto/Whirlpool.c:105:45: error: 'Whirlpool_C' causes a section type conflict with 'SHA256_K' [ 246s] 105 | CRYPTOPP_ALIGN_DATA(16) static const uint64 Whirlpool_C[8*256+R] CRYPTOPP_SECTION_ALIGN16 = { [ 246s] | ^ [ 246s] ../Crypto/Sha2.c:321:34: note: 'SHA256_K' was declared here [ 246s] 321 | CRYPTOPP_ALIGN_DATA(16) uint_32t SHA256_K[64] CRYPTOPP_SECTION_ALIGN16 = { [ 246s] | ^
so the question is how CRYPTOPP_SECTION_ALIGN16 / CRYPTOPP_ALIGN_DATA are defined for the respective TUs. One appearant difference is that Whirlpool_C is declared 'const' while SHA256_K is not so if the section is named that might be the issue.
OK, so google works and spits out
#ifndef CRYPTOPP_SECTION_ALIGN16 #if defined(__GNUC__) && !defined(__APPLE__) // the alignment attribute doesn't seem to work without this section attribute when -fdata-sections is turned on #define CRYPTOPP_SECTION_ALIGN16 __attribute__((section ("CryptoPP_Align16")))
so indeed a named section. Make SHA256_K 'const' and see if that works (or Whirlpool_C non-const, might be more "mismatches" exist).
Thanks to your hint, Richard, I've aligned the section types to "static const", and it builds fine now. Much appreciated.
A SR to Factory is triggered, as well as a PR to the upstream project.
Cheers, Pete
Hi Richard,
Am Montag, 12. August 2019, 10:07:32 CEST schrieb Richard Biener:
On Sun, 11 Aug 2019, Hans-Peter Jansen wrote:
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Another LTO fallout
With LTO enabled, if fails to load /usr/lib64/qt5/plugins/kf5/kio/gdrive.so on TW: https://build.opensuse.org/package/show/openSUSE:Factory/libkgapi
Without LTO, it loads fine: https://build.opensuse.org/package/show/home:wolfi323:branches:KDE:Extra/ libkgapi
The issue cannot be deduced from the warnings this time, AFAICS.
Testing: given, you have added a google account in systemsettings -> Online accounts, run dolphin, click on Network (in remote places list), and expand Google Drive.
Cheers, Pete
On August 26, 2019 5:36:10 PM GMT+02:00, Hans-Peter Jansen hpj@urpla.net wrote:
Hi Richard,
Am Montag, 12. August 2019, 10:07:32 CEST schrieb Richard Biener:
On Sun, 11 Aug 2019, Hans-Peter Jansen wrote:
Am Dienstag, 23. April 2019, 14:02:20 CEST schrieb Martin Liška:
Hi.
We as toolchain team are working on enablement of LTO in Factory:
Another LTO fallout
With LTO enabled, if fails to load /usr/lib64/qt5/plugins/kf5/kio/gdrive.so on TW: https://build.opensuse.org/package/show/openSUSE:Factory/libkgapi
Without LTO, it loads fine: https://build.opensuse.org/package/show/home:wolfi323:branches:KDE:Extra/ libkgapi
The issue cannot be deduced from the warnings this time, AFAICS.
Testing: given, you have added a google account in systemsettings -> Online accounts, run dolphin, click on Network (in remote places list), and expand Google Drive.
Thanks for reporting. Can you open a bugzilla please?
Thanks, Richard.
Cheers, Pete
Am Montag, 26. August 2019, 19:01:45 CEST schrieb Richard Biener:
On August 26, 2019 5:36:10 PM GMT+02:00, Hans-Peter Jansen hpj@urpla.net
wrote:
Another LTO fallout
With LTO enabled, if fails to load /usr/lib64/qt5/plugins/kf5/kio/gdrive.so on TW: https://build.opensuse.org/package/show/openSUSE:Factory/libkgapi
Without LTO, it loads fine: https://build.opensuse.org/package/show/home:wolfi323:branches:KDE:Extra/ libkgapi
The issue cannot be deduced from the warnings this time, AFAICS.
Testing: given, you have added a google account in systemsettings -> Online accounts, run dolphin, click on Network (in remote places list), and expand Google Drive.
Thanks for reporting. Can you open a bugzilla please?
Done: boo#1148217
Cheers, Pete
-
Andreas Schwab -
Bernhard M. Wiedemann -
Brüns, Stefan -
Carlos E. R. -
Carlos E.R. -
Dave Plater -
Dominique Leuenberger / DimStar -
Guillaume Gardet -
Hans-Peter Jansen -
Jan Engelhardt -
Jiri Slaby -
Johannes Meixner -
Karl Sinn -
kv -
Martin Herkt -
Martin Liška -
Matwey V. Kornilov -
Michael Ströder -
Oleksii Vilchanskyi -
Peter Czanik -
Richard Biener -
Roger Oberholtzer -
Thorsten Kukuk -
Vit Pelcak