Mailinglist Archive: opensuse-buildservice (214 mails)

< Previous Next >
Re: [opensuse-buildservice] Cross compilation dependency handling
Hi Carsten, Hi Adrian,

On Thursday, February 16, 2012 11:01:15 AM Adrian Schröter wrote:
Am Donnerstag, 16. Februar 2012, 10:43:01 schrieb Carsten Munk:
Daniel Gollub suggested that we work with N amount of 'sysroots' and N
rpm databases and I tend to agree. Right now in SB2 I distinguish
between host (/) and target /target, but it might be more flexible to
do with N targets instead.

We started last week finally to do some prototyping of this idea.

Added Christian and Uwe into the loop, since booth of them implemented most of
the stuff I'm mentioning here.
This is a "brief" (sorry, at least I tried to keep it brief) what we have
already in place and is working to some point (we have no build results yet).

Scheduler and Worker modification by Christian:

* Add new parameter to prjconf for additional architectures

New parameter 'AdditionalArch' added to projectconfiguration (prjconf). This
parameter identifies additional architectures with its sysroot that
should be downloaded during build preparation.

AdditionalArch: <arch> <sysroot>

In the prjconf we put something like this:

Required: gcc gcc%{gcc_version} glibc rpm tar patch

%ifarch armv5el armv7l
Hostarch: x86_64
%obs_crossbuild 1 # not yet used
AdditionalArch: x86_64 / # actually not required, IIR
AdditionalArch: armv7l /sysroot/armv7l
AdditionalArch: armv5el /sysroot/armv5el
Required(target): tar

* determine packages used in hostsystem or in target-sysroot

To determine which package has to be installed where, in the host system
or in a sysroot for a crossbuild. Therefor the XML for the buildinfo got
a new field for packages. 'neededbyhost' has to be set to 1 if a package
has to be installed in the host system. If this parameter is not set or
0 the package will be downloaded for the architecture the buildjob is
for and for all additional architectures into their sysroots.

With the example prjconf from above, the scheduler sends an enhanced buildjob
file which looks something like this:

<buildinfo project="home:Admin" repository="openSUSE_Factory_ARM_standard"
package="foo" srcserver="";

<additional_archs arch="x86_64" sysroot="/" /> <!--- actually not
required, IIRC --->
<additional_archs arch="armv7l" sysroot="/sysroot/armv7l" />
<additional_archs arch="armv5el" sysroot="/sysroot/armv5el" />
<bdep name="aaa_base" preinstall="1" runscripts="1" notmeta="1"
neededbyhost="1" />
<bdep name="tar" preinstall="1" neededbyhost="1" />
<bdep name="libsepol1" vminstall="1" target_arch="x86_64" />
<bdep name="libblkid1" vminstall="1" notmeta="1" target_arch="x86_64" />
<bdep name="libuuid1" vminstall="1" notmeta="1" target_arch="x86_64" />
<bdep name="gcc" />
<bdep name="gcc46" />
<bdep name="patch" />
<bdep name="cpp" />
<bdep name="tar" />
<path project="home:Admin" repository="openSUSE_Factory_ARM_standard"
server=""; />
<path project="openSUSE:Factory:ARM" repository="standard"
server=""; />

* Adjustments on the worker

The worker is now able to download packages for N-architectures and put
them into separate directories and RPMlists. Both are given to the buid
script using the new parameters --rpmlist-target. Also it sets --target
accordingly to the architecture the job does the build for.

srsyy:/var/cache/obs/worker/root_1 # ls -1 .build*rpmlist

The new "Required(target):" allows mutliple occurences of the same package-name
(e.g. "tar") to serve hostarch and targets with the same dependency (but
different target)
-> this will result in a tar package for the hostarch x86_64 and the targets
armvl and armv5el - in seperated directories:

srsyy:/var/cache/obs/worker/root_1 # find -name tar.rpm

Currently we only assign preinstall: vminstall: to the hostarch rpmlist. All
other packages get assigend to the target rpmlists.
So the hostarch is a minimal set to bootup a VM or switch into a chroot.

Following obs-build script modifications got introduced so far by Uwe:

* Add initialization of separate build root for cross compiling targets

The build script from obs-build has been enhanced, to setup additional
build root environments. To achieve this, a new parameter
--rpmlist-target has been introduced for setting up additional build
environments. Currently the name of the rpmlist file is taken as
architecture name is created at $BUILD_ROOT/sys-root/<arch>. In the
build script a new function setup_sys_root() has been added, which uses
the build_initscript from obs-build to setup additional build
enviroments. The build_initscript has been enhanced to allow the setup
of addtional environments underneath the host target environment. A new
parameter --root was introduced, which overwrites the ROOT_BUILD
environment variable.

(We are currently discussing if we can avoid the introduction of this --root

* Add cross build command

A new function cross_build() has been introduced in the build script
from obs-build. It is responsible for executing the commands to cross
build a package for the given architecture. Right now the functionality
is copied from the build commands for the host architecture, and have to
be adapted for necessary commands for cross build a package.

* Add initialization of rpm database for a cross build environment

The build script from obs-build will be enhanced to allow the
initialization of rpm database in a cross build environment.

(Instead of chroot $BUILD_ROOT rpm --init-db... we want to call the hostarch
rpm binary without chroot use)

There are still places where we need to touch code in init_buildsystem or build
script to not make use of chroot and instead call hostarch "toolchain" to setup
the buildenv.
We have not reached the final "rpmbuild" call yet. That's the reason why we
have no build results yet.

(this cmdline is from a test without armv5el - just armv7l)

/bin/bash /var/run/obs/worker/1/build/build --root /var/cache/obs/worker/root_1
--clean --changelog --oldpackages
/var/cache/obs/worker/root_1/.build.oldpackages --norootforbuild
--lint --dist /var/cache/obs/worker/root_1/.build.config --rpmlist
/var/cache/obs/worker/root_1/.build.rpmlist --rpmlist-target
/var/cache/obs/worker/root_1/.build.armv7l.rpmlist --target armv7l --
logfile /var/cache/obs/worker/root_1/.build.log --release 15.1 --arch armv7l
--jobs 1 --reason Building foo for project 'home:Admin' repository
'openSUSE_Factory_ARM_standard' arch 'armv7l' srcmd5
'6763ac6ae31fbc1a89ed98c53d443ce8' --disturl

* --rpmlist-target /var/cache/obs/worker/root_1/.build.armv7l.rpmlist
# --rpmlist-target can be invovked multiple times
# --rpmlist is used for hostarch

* --target armv7l
# sets the ABUILD_TARGET ... even if we support N-Target-Sysroots actual
build happens only for one target

We are think of introducing --crossbuild to indicate that we are doing no
The target-sysroot-pathes got not yet passed along to the build script. But we
plan to do so.

The question is of course, can we effectively work with multiple
dependency trees (maybe even across schedulers?) within the current
code and how do we do it best?

Our prototype managed to assemble a buildjob which a patched worker understands
and retrieves RPMs from different trees.

But all this requries that those RPMs from the different architectures are
already build and available.
We still need to introduce something that the scheduler schedules the cross
build job only if all cross-target dependencies are solved.
So there is the need that the scheduler check the meta-data/status of the other
architectures - if the all cross dependencies are ready.

For example: if a pre-install: for the Hostarch x86_64 is failing/missing, the
armv7l scheduler would still schedule the buildjob.

Adrian suggested something like Preinstall($hostarch) to indicate that
this needs to be preinstalled on host architecture.

I guess introducing "preinstall($hostarch)" is harder then "preinstall(host)".
Working with our current prototype it really looks like you do not really need
to care about what your hostarch is.
You just need to distinguish between: is it a package for the host or for one
of the targets?

So we currently follow this approach:

Imagine you need have as Hostarch: x86_64 (<hostarch />) and need additionally
those Targets: armv7l, i586, armv5el. (<additional_archs />)
Your primary build target (to build a package for) is: armv7l (<arch />)

- preinstall: / vminstall: packages will only be installed for the hostarch.
- all other build dependencies will be installed for each additional_arch -
depending on the BuildRequires in your spec-file.

All this will result in a $BUILD_ROOT which looks something like this:

$BUILD_ROOT/ <- stuffed with set of preinstall: / vminstall:
$BUILD_ROOT/sysroot-armv7l/ <- no preinstall/vminstall, but all other package
dependencies for armv7l installed in a seperated package database
$BUILD_ROOT/sysroot-armv5el/ <- no preinstall/vminstall, but all other package
dependencies for armv5el installed in a seperated package database
$BUILD_ROOT/sysroot-i586/ <- no preinstall/vminstall, but all other package
dependencies for i586 installed in a seperated package database

What is not covered yet is to introduce special dependencies for crossbuild
For testing we introduced this one:


If you run a crossbuild, those required packages will only get installed in
your target-sysroots. For non-crossbuilds not.

Thinking again about it, specify an arch here makes not much sense.

Better something like this in spec files:

BuildRequire(host): gcc
BuildRequire(target): libqt4-devel

Since this is not supported by rpm yet, we would need to have something

#!BuildRequire(host): gcc
#!BuildRequire(target): libqt4-devel

So when a

BuildRequire: gcc

appears later in the spec file (for plain rpm non-cross building), OBS can
see that it has it already, but it knows already for which architecture it
needs it.

I thought about doing this in another way or even provide something
Instead of putting special crossbuild dependencies in each spec files (by
default), use some kind of Substitute(host): / Substitute(target): /
Required(target): / Required(host): combination.

Something like this:

Substitute(target): gcc %{crosstag}gcc

-> Drops gcc from the target dependency list and will replace it with a
dependency of %{crosstag}gcc in the hostarch tree.

Substitute(crossdep): libqt4-devel

-> if libqt4-devel shows up in the build-depenency of a target (in a
crossbuild), libqt4-devel gets a "cross dependency": it gets additionally a
dependency for the hostarch.
(e.g. if you want to use pre-processors and run them with native
performance, but need target specific headers, static libraries, ...)

The prjconf Preinstall, Support and Require lines could also support the
(host) and (target) extensions.


So far we only needed Required(target). But very likely we need more of them
while keep working on that.

My suggestion
would be perhaps to indicate through configuration where to put
things, for example, Roots: host=/ armv7l=/opt/cross/armv7l, etc, so
we can do dynamic labeling and telling what location on the build
filesystem things should go in.

Yes, similar to Hostarch: directive, we could add a Targetpath:

What do you think about the "AdditionalArch: $ARCH $TARGETPATH" directive?
Or should we change this to something else?

Btw, our git branches can be found here:

Best Regards,

Daniel Gollub
Linux Consultant & Developer
Tel.: +49-160 47 73 970
Mail: gollub@xxxxxxxxxxxxx

B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg /
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537
< Previous Next >