[opensuse-packaging] problems with -lto-auto in R-RcppParallel
Hi! This package: https://build.opensuse.org/package/show/devel:languages:R:released/R-RcppPar... does not compile in OBS OpenSUSE. It works for Fedora, debian etc. The only possible problem we could find, was the usage of -flto-auto for the included tbb library. Upstream is https://github.com/RcppCore/RcppParallel, an issue was filed some time ago, but upstream has no idea either. Thankful for any idea, because some very interesting R-packages depend on RcppParallel, that I would like to add to OBS. Regards Detlef -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Saturday, 2 May 2020 14:53:42 CEST Detlef Steuer wrote:
Hi!
This package: https://build.opensuse.org/package/show/devel:languages:R:released/R-RcppPar allel
does not compile in OBS OpenSUSE.
It works for Fedora, debian etc.
The only possible problem we could find, was the usage of -flto-auto for the included tbb library.
Upstream is https://github.com/RcppCore/RcppParallel, an issue was filed some time ago, but upstream has no idea either.
The preferred approach would be to just use the system tbb library instead of using a bundled copy. Maybe its sufficient to add tbb-devel to the BuildRequires, but a short glimpse at the build system suggests it insists on building its on tbb, so you likely have to patch the tbb building part out. Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen phone: +49 241 53809034 mobile: +49 151 50412019
Hello, (Martin: CCed you for a real lto-wrapper problem you might be interested in :) ) On Sat, 2 May 2020, Detlef Steuer wrote:
This package: https://build.opensuse.org/package/show/devel:languages:R:released/R-RcppPar...
does not compile in OBS OpenSUSE.
It works for Fedora, debian etc.
The only possible problem we could find, was the usage of -flto-auto for the included tbb library.
(see below for the solution) You can verify if this is the case by following https://en.opensuse.org/openSUSE:LTO for disabling LTO for this one package. (Add '%define _lto_cflags %{nil}' in your spec file). Note that if this helps it's really only a work-around for something that should be fixed correctly. ... hmm, looking at the build log (note that it also fails on SLE-12, for package policy reasons), it has: [ 225s] g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -flto=auto -o RcppParallel.so init.o options.o -L/usr/lib64/R/lib -lR [ 225s] lto-wrapper: fatal error: execvp: No such file or directory [ 225s] compilation terminated. [ 225s] /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: error: lto-wrapper failed [ 225s] collect2: error: ld returned 1 exit status which is really strange, it's not an usual problem with LTO itself, but rather that it can't call a helper that's needed for the linking process, which should come from the gcc9 package itself. Digging deeper... The message is misleading, the parallel LTO compilation internally uses make and some temporary makefiles, and the lto-wrapper tries to call make like so: 20182 execve("/usr/bin/make -e -s", ["make -e -s", "-f", "/tmp/ccSBBfs4.mk", "-j8", "all"], 0x7ffe9d9873c0 /* 98 vars */) = -1 ENOENT (No such file or directory) Note that the executable it's trying to call is 'make -e -s'. This comes from the setting of 'MAKE' in src/Makevars: ... MAKE += -e -s MAKE_CMD = \ CONLY="$(CC) $(CPPFLAGS)" \ ... This is an unusual use of the $(MAKE) variable for makefiles. The more traditionl one to change setting for the current invocation of make is to use MAKEFLAGS (like further up is already done for Windows and -j1). So changing this to MAKEFLAGS += -e -s solves the problem you have. This _does_ point to a problem in GCC, though, because the above usage of MAKE to smuggle in command line args for make does work in general. So lto-wrapper needs to do word splitting on spaces for that environment variable, when it uses it. (CCing Martin for this) Ciao, Michael. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Thank you for your help! Works! :-)
Detlef
Am Tue, 5 May 2020 15:56:30 +0000 (UTC)
schrieb Michael Matz
Hello,
(Martin: CCed you for a real lto-wrapper problem you might be interested in :) )
On Sat, 2 May 2020, Detlef Steuer wrote:
This package: https://build.opensuse.org/package/show/devel:languages:R:released/R-RcppPar...
does not compile in OBS OpenSUSE.
It works for Fedora, debian etc.
The only possible problem we could find, was the usage of -flto-auto for the included tbb library.
(see below for the solution)
You can verify if this is the case by following https://en.opensuse.org/openSUSE:LTO for disabling LTO for this one package. (Add '%define _lto_cflags %{nil}' in your spec file). Note that if this helps it's really only a work-around for something that should be fixed correctly.
... hmm, looking at the build log (note that it also fails on SLE-12, for package policy reasons), it has:
[ 225s] g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -flto=auto -o RcppParallel.so init.o options.o -L/usr/lib64/R/lib -lR [ 225s] lto-wrapper: fatal error: execvp: No such file or directory [ 225s] compilation terminated. [ 225s] /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: error: lto-wrapper failed [ 225s] collect2: error: ld returned 1 exit status
which is really strange, it's not an usual problem with LTO itself, but rather that it can't call a helper that's needed for the linking process, which should come from the gcc9 package itself.
Digging deeper... The message is misleading, the parallel LTO compilation internally uses make and some temporary makefiles, and the lto-wrapper tries to call make like so:
20182 execve("/usr/bin/make -e -s", ["make -e -s", "-f", "/tmp/ccSBBfs4.mk", "-j8", "all"], 0x7ffe9d9873c0 /* 98 vars */) = -1 ENOENT (No such file or directory)
Note that the executable it's trying to call is 'make -e -s'. This comes from the setting of 'MAKE' in src/Makevars:
... MAKE += -e -s MAKE_CMD = \ CONLY="$(CC) $(CPPFLAGS)" \ ...
This is an unusual use of the $(MAKE) variable for makefiles. The more traditionl one to change setting for the current invocation of make is to use MAKEFLAGS (like further up is already done for Windows and -j1). So changing this to
MAKEFLAGS += -e -s
solves the problem you have.
This _does_ point to a problem in GCC, though, because the above usage of MAKE to smuggle in command line args for make does work in general. So lto-wrapper needs to do word splitting on spaces for that environment variable, when it uses it. (CCing Martin for this)
Ciao, Michael.
-- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Tue, 5 May 2020, Michael Matz wrote:
Hello,
(Martin: CCed you for a real lto-wrapper problem you might be interested in :) )
On Sat, 2 May 2020, Detlef Steuer wrote:
This package: https://build.opensuse.org/package/show/devel:languages:R:released/R-RcppPar...
does not compile in OBS OpenSUSE.
It works for Fedora, debian etc.
The only possible problem we could find, was the usage of -flto-auto for the included tbb library.
(see below for the solution)
You can verify if this is the case by following https://en.opensuse.org/openSUSE:LTO for disabling LTO for this one package. (Add '%define _lto_cflags %{nil}' in your spec file). Note that if this helps it's really only a work-around for something that should be fixed correctly.
... hmm, looking at the build log (note that it also fails on SLE-12, for package policy reasons), it has:
[ 225s] g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -flto=auto -o RcppParallel.so init.o options.o -L/usr/lib64/R/lib -lR [ 225s] lto-wrapper: fatal error: execvp: No such file or directory [ 225s] compilation terminated. [ 225s] /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: error: lto-wrapper failed [ 225s] collect2: error: ld returned 1 exit status
which is really strange, it's not an usual problem with LTO itself, but rather that it can't call a helper that's needed for the linking process, which should come from the gcc9 package itself.
Digging deeper... The message is misleading, the parallel LTO compilation internally uses make and some temporary makefiles, and the lto-wrapper tries to call make like so:
20182 execve("/usr/bin/make -e -s", ["make -e -s", "-f", "/tmp/ccSBBfs4.mk", "-j8", "all"], 0x7ffe9d9873c0 /* 98 vars */) = -1 ENOENT (No such file or directory)
Note that the executable it's trying to call is 'make -e -s'. This comes from the setting of 'MAKE' in src/Makevars:
... MAKE += -e -s MAKE_CMD = \ CONLY="$(CC) $(CPPFLAGS)" \ ...
This is an unusual use of the $(MAKE) variable for makefiles. The more traditionl one to change setting for the current invocation of make is to use MAKEFLAGS (like further up is already done for Windows and -j1). So changing this to
MAKEFLAGS += -e -s
solves the problem you have.
This _does_ point to a problem in GCC, though, because the above usage of MAKE to smuggle in command line args for make does work in general. So lto-wrapper needs to do word splitting on spaces for that environment variable, when it uses it. (CCing Martin for this)
I think it should check $(MAKE) actually points to an executable ... (or simply somehow try and then fall back to 'make'). It could be MAKE = echo as well if the project is not using make after all. It simply points to the fact that we are relying on heuristics here and the user playing well with us. Not entirely unreasonable IMHO. Richard.
Ciao, Michael.
--
Richard Biener
participants (4)
-
Detlef Steuer -
Michael Matz -
Richard Biener -
Stefan Brüns