| On Sat, Nov 12, 2022 at 2:37 PM Hans-Peter Jansen <hpj@urpla.net> wrote:

• How to address this mess?

It shouldn't be that difficult to write a meson.build and ninja build it.

Probably preparing the environment before in order to avoid those shell expansions within the Makefile will cut it

• The new make behaviour is kind of questionable,
  • since it can be easily tricked into a DoS, since it might behave like a fork bomb now
  • feels like a regression nevertheless

Reminds me of a saying:

Just don't. Normal human beings and even skilled developers suck at writing makefiles. if it is not this it will be another problem. On Sat, Nov 12, 2022 at 2:37 PM Hans-Peter Jansen <hpj@urpla.net> wrote:

Am Samstag, 12. November 2022, 16:56:35 CET schrieb Jan Engelhardt:

...

On Friday 2022-11-11 20:56, Hans-Peter Jansen wrote:

...

some late changes in the build chain degrades build performance to a crawl: e.g. a package typically builds in under two minutes:

https://build.opensuse.org/package/live_build_log/vdr:plugins/vdr-plugin-li ve/ openSUSE_Tumbleweed/x86_64

shows this now:

[ 2624s] CC live.o [ 3161s] CC thread.o

1. `osc build ...` in one terminal

2. Second terminal,

   strace -fe execve -p 24582

   (pid of make)

and I observed that make re-runs pkg-config a damn lot.

What recently changed in openSUSE is the make program (4.3 -> 4.4).

...

From make's changelog, I identify two changes that seem like they

could be relevant:

  https://lists.gnu.org/archive/html/info-gnu/2022-10/msg00008.html

  >Previously each target in a explicit grouped target rule was
  >considered individually: if the targets needed by the build
  >were not out of date the recipe was not run even if other
  >targets in the group were out of date

  or

  >Previously makefile variables marked as export were not
  >exported to commands started by the $(shell ...) function.
  >Now, all exported variables are exported to $(shell ...).

Of course, these alone do not constitute problem - we have over 1000 packages that exercise `make` one way or the other, but they do so through a proper means of Makefile generators like configure or cmake or something of the sort.

3. So we're looking at a shitty Makefile.

   vdr$ make -ddd clean Reading makefile 'Makefile'... grep: warning: stray \ before # Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Reading makefile 'global.mk' (search path) (no ~ expansion)... Makefile:27: not recursively expanding CFLAGS to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function Makefile:18: not recursively expanding PKGCFG to export to shell function ...

Just look at the mess:

  PKGCFG = $(if $(VDRDIR),$(shell pkg-config --variable=$(1)

$(VDRDIR)/vdr.pc),$(shell PKG_CONFIG_PATH="$$PKG_CONFIG_PATH:../../.." pkg-config --variable=$(1) vd... LIBDIR = $(call PKGCFG,libdir) LOCDIR = $(call PKGCFG,locdir) PLGCFG = $(call PKGCFG,plgcfg) RESDIR = $(call PKGCFG,resdir)

  ### The compiler options:
  export CFLAGS   = $(call PKGCFG,cflags)
  export CXXFLAGS = $(call PKGCFG,cxxflags)

  ...

  ifeq ($(TNTNET-CONFIG),)
  TNTVERSION = $(shell pkg-config --modversion tntnet | sed -e's/\.//

g' | sed -e's/pre.*//g' | awk '/^..$$/ { print $$1."000"} /^...$$/ { print $$1."00"} /^....$ CXXFLAGS += $(shell pkg-config --cflags tntnet) LIBS += $(shell pkg-config --libs tntnet) else TNTVERSION = $(shell tntnet-config --version | sed -e's/.//g' | sed -e's/pre.*//g' | awk '/^..$$/ { print $$1."000"} /^...$$/ { print $$1."00"} /^....$$/ { pr CXXFLAGS += $(shell tntnet-config --cxxflags) LIBS += $(shell tntnet-config --libs)

So remember what the changelog said: export all variables whenever $(shell ..) is run. The irony is that, to set up the environment for that shell run, it has to expand PKGCFG, which involves calling at least another $(shell ...).

And that multiplies fast.

*yuck*

First at all, thanks Jan! Your analysis is much appreciated..

Oh my, wth.

This reveals a bunch of questions:

• How to address this mess? Probably preparing the environment before in order to avoid those shell expansions within the Makefile will cut it
• The new make behaviour is kind of questionable,
  • since it can be easily tricked into a DoS, since it might behave like a fork bomb now
  • feels like a regression nevertheless

Reminds me of a saying:

You don't understand recursion unless you understand recursion!

Thanks again, Pete

On Sunday 2022-11-13 15:06, Stefan Seyfried wrote:

On 13.11.22 12:59, Cristian Rodríguez wrote:

...

On Sat, Nov 12, 2022 at 2:37 PM Hans-Peter Jansen <hpj@urpla.net> wrote:

...

• How to address this mess?

It shouldn't be that difficult to write a meson.build and ninja build it.

You are free to try convincing upstream to switch to that stuff :-)

Well if they don't, upstream is just going to play themselves. Because make 4.4 will make an entrance into practically all distros eventually.