| On Sat, Nov 12, 2022 at 2:37 PM Hans-Peter Jansen <hpj@urpla.net> wrote:
* How to address this mess?
It shouldn't be that difficult to write a meson.build and ninja build it.
Probably preparing the environment before in order to avoid those shell expansions within the Makefile will cut it * The new make behaviour is kind of questionable, * since it can be easily tricked into a DoS, since it might behave like a fork bomb now * feels like a regression nevertheless
Reminds me of a saying:
Just don't. Normal human beings and even skilled developers suck at writing makefiles. if it is not this it will be another problem. On Sat, Nov 12, 2022 at 2:37 PM Hans-Peter Jansen <hpj@urpla.net> wrote:
Am Samstag, 12. November 2022, 16:56:35 CET schrieb Jan Engelhardt:
On Friday 2022-11-11 20:56, Hans-Peter Jansen wrote:
some late changes in the build chain degrades build performance to a crawl: e.g. a package typically builds in under two minutes:
https://build.opensuse.org/package/live_build_log/vdr:plugins/vdr-plugin-li ve/ openSUSE_Tumbleweed/x86_64
shows this now:
[ 2624s] CC live.o [ 3161s] CC thread.o
1. `osc build ...` in one terminal
2. Second terminal,
strace -fe execve -p 24582
(pid of make)
and I observed that make re-runs pkg-config a damn lot.
What recently changed in openSUSE is the make program (4.3 -> 4.4).
From make's changelog, I identify two changes that seem like they
could be relevant:
https://lists.gnu.org/archive/html/info-gnu/2022-10/msg00008.html
>Previously each target in a explicit grouped target rule was >considered individually: if the targets needed by the build >were not out of date the recipe was not run even if other >targets in the group were out of date
or
>Previously makefile variables marked as export were not >exported to commands started by the $(shell ...) function. >Now, all exported variables are exported to $(shell ...).
Of course, these alone do not constitute problem - we have over 1000 packages that exercise `make` one way or the other, but they do so through a proper means of Makefile generators like configure or cmake or something of the sort.
3. So we're looking at a shitty Makefile.
vdr$ make -ddd clean Reading makefile 'Makefile'... grep: warning: stray \ before # Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Reading makefile 'global.mk' (search path) (no ~ expansion)... Makefile:27: not recursively expanding CFLAGS to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function ...
Just look at the mess:
PKGCFG = $(if $(VDRDIR),$(shell pkg-config --variable=$(1) $(VDRDIR)/vdr.pc),$(shell PKG_CONFIG_PATH="$$PKG_CONFIG_PATH:../../.." pkg-config --variable=$(1) vd... LIBDIR = $(call PKGCFG,libdir) LOCDIR = $(call PKGCFG,locdir) PLGCFG = $(call PKGCFG,plgcfg) RESDIR = $(call PKGCFG,resdir)
### The compiler options: export CFLAGS = $(call PKGCFG,cflags) export CXXFLAGS = $(call PKGCFG,cxxflags)
...
ifeq ($(TNTNET-CONFIG),) TNTVERSION = $(shell pkg-config --modversion tntnet | sed -e's/\.// g' | sed -e's/pre.*//g' | awk '/^..$$/ { print $$1."000"} /^...$$/ { print $$1."00"} /^....$ CXXFLAGS += $(shell pkg-config --cflags tntnet) LIBS += $(shell pkg-config --libs tntnet) else TNTVERSION = $(shell tntnet-config --version | sed -e's/\.//g' | sed -e's/pre.*//g' | awk '/^..$$/ { print $$1."000"} /^...$$/ { print $$1."00"} /^....$$/ { pr CXXFLAGS += $(shell tntnet-config --cxxflags) LIBS += $(shell tntnet-config --libs)
So remember what the changelog said: export all variables whenever $(shell ..) is run. The irony is that, to set up the environment for that shell run, it has to expand PKGCFG, which involves calling at least another $(shell ...).
And that multiplies fast.
*yuck*
First at all, thanks Jan! Your analysis is much appreciated..
Oh my, wth.
This reveals a bunch of questions: * How to address this mess? Probably preparing the environment before in order to avoid those shell expansions within the Makefile will cut it * The new make behaviour is kind of questionable, * since it can be easily tricked into a DoS, since it might behave like a fork bomb now * feels like a regression nevertheless
Reminds me of a saying:
You don't understand recursion unless you understand recursion!
Thanks again, Pete