Mailinglist Archive: opensuse-buildservice (105 mails)

< Previous Next >
Re: [opensuse-buildservice] multibuild flavor vs package
On Tuesday, December 17, 2019 12:36:30 AM CET Neal Gompa wrote:
On Mon, Dec 16, 2019 at 4:59 AM Ludwig Nussel <ludwig.nussel@xxxxxxx> wrote:
Adrian Schröter schrieb:
On Montag, 9. Dezember 2019, 13:21:13 CET Ludwig Nussel wrote:
Hi,

Trying to figure out a way to use multibuild for mingw packages. The
challenge there is that almost all packages need to be built for 32bit
as well as 64bit Windows. So currently we have mingw32-foo and
mingw64-foo packages. With exactly identical spec files except for
s/mingw64/mingw32/ and vice versa. So sounds like a job for multibuild
with flavors mingw32 and mingw64 to produce those packages from a
common
mingw-foo package.
Now there are also packages that have multiple spec files though like
gcc and pkgconf. Looks like that case was somehow forseen for
multibuild
as there are 'flavor' and 'package' tags. In the end both are treated
the same though and end up as @BUILD_FLAVOR@ in spec files.

Are there any plans to change that? So one could write something like

<multibuild>

<flavor>mingw64</flavor>
<flavor>mingw32</flavor>
<package>mingw-gcc</package>
<package>mingw-cross-gcc</package>
<package>mingw-cross-gcc-bootstrap</package>

</multibuild>

to produce six packages?

Short of that would it work to have both links and multibuild? Ie links
according to spec file names and only mingw flavors?

hm, not sure if it makes sense to use _multibuild here. Isn't this more
like python2 vs python3 stuff?

So, one spec, one build enviroment where you could pull in both works
and
build multiple binary rpm's?

After looking at a more elaborate spec file I'm quite sure that using
multibuild would be superior to complex macro magic. Much easier to
handle and understand for packagers.

Rpm macros like done with python and also how Fedora does mingw have to
hide the fact that things are done multiple times.

So when calling something like eg. %make that wouldn't actually call
"make" but eg

cd build_win32
make
cd ../build_win64
make
cd ..

All commands to deal with the sources have to be duplicated that way,
either hidden by macros or manually. Look at this for example:

https://src.fedoraproject.org/rpms/mingw-qt5-qtbase/blob/master/f/mingw-qt
5-qtbase.spec

The spec is long enough without the duplication already.

Also, by building both flavors in the same spec the build gets
serialized whereas multibuild could build on two hosts in parallel.

There are major downsides too: using multibuild means
that local development of such packages is functionally impossible.

Care to elaborate? Whats wrong with "osc build -M <flavor>"?

You are also saying it's okay to occupy more builders at a time (which
is probably not actually okay to do at scale).

Often it is possible to reduce the peak resources required for package to
build, which improves scaling. Also, when the build is split up, you hold on
to these resources for shorter time. At least for the public OBS, scaling
horizontally (more builders) is no problem at all, but blocking the few "fat"
workers definitely is.

Also, often the flavors have vastly different build requirements package wise.
These requirements cause build loops, or delay the build of a package so doing
a rebuild (which happens on every snapshot) takes much longer.

You also lose the
guarantee that the binary artifacts are always in sync, since a
failure for one flavor doesn't stop another. That is a subtle kind of
breakage that's hard to perceive and fix.

In most cases this is not a problem:
- as soon as a package is submitted from one of its development repositories,
all flavors should build.
- often there is no requirement for the packages to be in sync. Why should
e.g. the API documentation depend on a specific rebuilt?
- synchronized versions, in case these are necessary, can be enforced by
appropriate Requires. Build synchronization does *not* guarantee same versions
on the installed system - in case you do e.g. a "zypper in foo", the pulled in
dependencies may be newer than already fulfilled (installed) ones.

Duplication in the spec is something that is solvable (cf. SUSE-style
single-spec for Python and Ruby). But losing understandability and
coherence is something I'd rather not see us do.

IMHO _multibuild packages are much easier to understand than the magic
happening in e.g. the python singlespec macros. And why should one rebuild
*all* the python3 packages and its dependees when only python2 has changed?
What a waste of resources ...

Kind regards,

Stefan

--
Stefan Brüns / Bergstraße 21 / 52062 Aachen
phone: +49 241 53809034 mobile: +49 151 50412019


--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-buildservice+owner@xxxxxxxxxxxx

< Previous Next >