Neal Gompa schrieb:

On Mon, Dec 9, 2019 at 7:21 AM Ludwig Nussel wrote:

...

Trying to figure out a way to use multibuild for mingw packages. The challenge there is that almost all packages need to be built for 32bit as well as 64bit Windows. So currently we have mingw32-foo and mingw64-foo packages. With exactly identical spec files except for s/mingw64/mingw32/ and vice versa. So sounds like a job for multibuild with flavors mingw32 and mingw64 to produce those packages from a common mingw-foo package. Now there are also packages that have multiple spec files though like gcc and pkgconf. Looks like that case was somehow forseen for multibuild as there are 'flavor' and 'package' tags. In the end both are treated the same though and end up as @BUILD_FLAVOR@ in spec files.

Are there any plans to change that? So one could write something like

<multibuild> <flavor>mingw64</flavor> <flavor>mingw32</flavor> <package>mingw-gcc</package> <package>mingw-cross-gcc</package> <package>mingw-cross-gcc-bootstrap</package> </multibuild>

to produce six packages?

Short of that would it work to have both links and multibuild? Ie links according to spec file names and only mingw flavors?

Why not just define the flavors in the spec itself? It seems crazy to try to use features like multibuild for this.

It's not difficult, see: https://src.fedoraproject.org/rpms/mingw-openal-soft/blob/master/f/mingw-ope...

Alternatively, something like what we do for Python and Ruby builds could do the trick here (macros that expand to flavors).

I am aware of those methods. Looks like it is a matter of choosing your favorite way of getting tortured. The price of those is a lot of (black) magic with macros. Tricky to implement and hard to debug. Compared to that, multibuild looks rather straight forward to me. That's why I'm exploring it first. Requires some clarification of where multibuild leads to though. Dominique also filed an issue for that: https://github.com/openSUSE/open-build-service/issues/8847 When thinking about it I wonder if a _multibuild file is needed at all. The spec file selection could be solved with classical links just fine. Maybe with some more code in OBS to create the requires packages automatically on request accept, like external scripts do (or did?) for staging. The flavors with the @BUILD_FLAVOR@ substitution feels like it should be done in a more rpmish way. Thinking of extending bcond, eg assuming we have %bcond_with bootstrap %bcond_with small %bcond_with full [...] %if %{with bootstrap} foo %else bar %endif One could come up with a macro to replace the %bcond_with lines, let's say %{bcond_xor bootstrap small full} So that bcond_xor would have to make sure one of the options is actually selected and fail rpmbuild if not. OBS would then have to parse the %bcond_xor macro in the spec to generate those virtual packages with colon separated flavor we know from multibuild today. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Jan Engelhardt schrieb:

On Monday 2019-12-09 15:08, Ludwig Nussel wrote:

...

When thinking about it I wonder if a _multibuild file is needed at all. The spec file selection could be solved with classical links just fine.

Links don't play nice (i.e. they get expanded sometimes and sometimes differently) when you are about to copypac or sr a package. And sometimes, the sr gets rejected by the bot because it's a link.

Inconsistent handling of local links sound like bugs. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Adrian Schröter schrieb:

On Montag, 9. Dezember 2019, 13:21:13 CET Ludwig Nussel wrote:

...

Hi,

Trying to figure out a way to use multibuild for mingw packages. The challenge there is that almost all packages need to be built for 32bit as well as 64bit Windows. So currently we have mingw32-foo and mingw64-foo packages. With exactly identical spec files except for s/mingw64/mingw32/ and vice versa. So sounds like a job for multibuild with flavors mingw32 and mingw64 to produce those packages from a common mingw-foo package. Now there are also packages that have multiple spec files though like gcc and pkgconf. Looks like that case was somehow forseen for multibuild as there are 'flavor' and 'package' tags. In the end both are treated the same though and end up as @BUILD_FLAVOR@ in spec files.

Are there any plans to change that? So one could write something like

<multibuild> <flavor>mingw64</flavor> <flavor>mingw32</flavor> <package>mingw-gcc</package> <package>mingw-cross-gcc</package> <package>mingw-cross-gcc-bootstrap</package> </multibuild>

to produce six packages?

Short of that would it work to have both links and multibuild? Ie links according to spec file names and only mingw flavors?

hm, not sure if it makes sense to use _multibuild here. Isn't this more like python2 vs python3 stuff?

So, one spec, one build enviroment where you could pull in both works and build multiple binary rpm's?

After looking at a more elaborate spec file I'm quite sure that using multibuild would be superior to complex macro magic. Much easier to handle and understand for packagers. Rpm macros like done with python and also how Fedora does mingw have to hide the fact that things are done multiple times. So when calling something like eg. %make that wouldn't actually call "make" but eg cd build_win32 make cd ../build_win64 make cd .. All commands to deal with the sources have to be duplicated that way, either hidden by macros or manually. Look at this for example: https://src.fedoraproject.org/rpms/mingw-qt5-qtbase/blob/master/f/mingw-qt5-... The spec is long enough without the duplication already. Also, by building both flavors in the same spec the build gets serialized whereas multibuild could build on two hosts in parallel. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Neal Gompa schrieb:

On Mon, Dec 16, 2019 at 4:59 AM Ludwig Nussel wrote:

...

[...] After looking at a more elaborate spec file I'm quite sure that using multibuild would be superior to complex macro magic. Much easier to handle and understand for packagers. [...]

There are major downsides too: using multibuild means that local development of such packages is functionally impossible.

You mean because of that nasty @BUILD_FLAVOR@ substitution? I really hope that we can find a sane alternative, like the one I suggest earlier in this thread.

You are also saying it's okay to occupy more builders at a time (which is probably not actually okay to do at scale). You also lose the

OBS is all about parallelization. That's a feature that needs to be leveraged as much as possible :-)

guarantee that the binary artifacts are always in sync, since a failure for one flavor doesn't stop another. That is a subtle kind of breakage that's hard to perceive and fix.

May happen but is a pretty standard problem to have. In fact we have that all the time with different architectures for example. And in the end mingw32 and mingw64 are nothing else than architectures. Just weirdly packaged as there's no general concept of handling cross arch packages. So no nothing special here. Doesn't sound like a big deal to me.

Duplication in the spec is something that is solvable (cf. SUSE-style single-spec for Python and Ruby). But losing understandability and coherence is something I'd rather not see us do.

Like I said, at the expense of more macro magic. I wouldn't consider myself a total novice in rpm macros but what python is doing there is close to witchcraft. That works in a situation where most packages are standard situations, using the same build system etc. Mingw packages are just like any other package in the distro. Ie has to deal with all build systems anyone ever invented means all the quirks and hacks needed to deal with them. On top of that cross compilation hacks. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org