Mailinglist Archive: opensuse-buildservice (214 mails)

< Previous Next >
Re: [opensuse-buildservice] RFC: patches for SB2 integration into OBS for better cross compilation,
Hi Crasten, Hi Uwe,


Uwe, Carsten is addressing similar issue as we do in the obs-build scripts -
you might want to have a look on them so we can collaborate with Carsten.

On Friday, February 10, 2012 09:33:02 AM Carsten Munk wrote:
Hi all,

I'd like to request comments and review on my upcoming patches which
makes it possible to integrate SB2 (Scratchbox2, not related to
Scratchbox1 in usage or code) into OBS for much cleaner, flexible and
faster cross compilation approaches that can be utilized in multiple
distributions.
[...]
Now, on to the patches -- I'm touching three components, osc,
obs-build, open-build-service, with the most invasive patches in
'obs-build'.

Please review the following commits:

https://github.com/stskeeps/obs-build/commit/0687c00013901bdba893f2b69289919c6d6388bd
https://github.com/stskeeps/open-build-service/commit/7fced08b7c2eecf93e82b53c397acd7f3d72d86a
https://github.com/stskeeps/osc/commit/f0ba509e308d630345d0e064d7aa3856c5ae5457

To understand the logic/approach in these patches:
[....]
* Instead of utilizing chroot command everywhere, a wrapper has been
made, called enter_target, see
https://github.com/stskeeps/obs-build/commit/0687c00013901bdba893f2b69289919c6d6388bd#L3R126
- It classifies chroot calls into three kinds, needroot,
needuser and needscript.

needroot implies that root is needed to run the command, needuser
implies that we should 'su' to the user to run the command, and
needscript is needed to run su to the user with '-s' parameter.

- Most of the changes within the patch has been rewrites in order to
switch from chroot calls to enter_target. When SB2 is not enabled,
these are translated straight to chroot / su / su -s calls.

Here we hit the same challenge. Inside init_buildsystem and other places where
a lot of chroot are done.
Which makes it hard to use some parts of the scripts for "classic" crossbuilds.

In our case we call multiple times init_buildsystem - to setup the actual
BUILD_ROOT + for each target sysroot we want to setup.
But for the init_buildsystem call for the target sysroots we want to avoid the
chroot calls. In some cases some people might want
to have very limited sysroots which are not suiteable for chroots at all.

For instance all the rpm calls (intalling packages, init-db, quries, ...) could
be done by using the host rpm binary and pointing with "rpm --dbpath" to target
rpm database.
E.g. rpm --dbpath $SYSROOTPATH/var/lib/rpm ... or even use that rpm --root
parameter (I'm still not quite sure if rpm really makes use of chroot on its
own then).


With SB2 are you calling the target rpm binary (and it deps) to do this initial
setup?
Or do you call somehow the native rpm?

(Analog to dpkg)


Maybe additionally to a generic wrapper (which should also support plain calls
and maybe NOOP) we should write for more frequent calls, specific wrapper -
e.g. for rpm / dpkg.
So the rpm / dpkg specific wrapper use the hostarch binaries and point to the
sysroot package database.


obs-build> git show 99417ba6c225c58dbc240720da4b6a245f753123 | grep
enter_target | egrep -v '(rpm |dpkg)' | wc -l
26
projects/obs-build> git show 99417ba6c225c58dbc240720da4b6a245f753123 | grep
enter_target | wc -l
44




* There has been introduced the concept of a 'target', and we split
BUILD_ROOT contents into being under BUILD_ROOT and BUILD_TARGET. When
SB2 is not enabled, BUILD_ROOT=BUILD_TARGET=BUILD_ROOT, in order to
not change behavior. Most of the patches are related to putting some
things under BUILD_TARGET instead of BUILD_ROOT
[...]

For the init_buildsystem script we just introduce a --root parameter to
overwrite the BUILD_ROOT variable just for the init_buildsystem script (not for
the buildscript). So we can call
init_buildsystem additionally for each target. But we are still in middle of
adapting the init_buildsystem to setup a seperated sysroots. So not quite sure
yet if this solves all our problems with
BUILD_TARGET/BUILD_ROOT.

Have you tried to just overwrite the BUILD_ROOT in the init_buildsystem to some
"target" direcotry.



This diff hunk is very useful:

https://github.com/stskeeps/obs-build/commit/0687c00013901bdba893f2b69289919c6d6388bd#L7L697

Without that using init_buildsystem to setup a seperated target sysroot fails
to detect the right package-system (deb || rpm).
The target architectures would require the rpm or the deb package as native
dependency to indicating which kind of disto-package-system should be setup:
deb or rpm distro-package-system.

You might want to isolate this diff hunk as seperated patch and and ask to get
this one pulled.

Maybe isolating those new features (e.g. chroot wrapper,
BUILD_ROOT->BUILD_TARGET migration, SB2 introduction, ...) might speed up
getting this pulled.
I guess it would make review easier - since you have smallers diffs which
introduce one feature.



With regards to your open-build-service change:
https://github.com/stskeeps/open-build-service/commit/7fced08b7c2eecf93e82b53c397acd7f3d72d86a

Instead of introducing "SB2install" I guess a generic "Required(host):" might
also work for you - right?

Required(host): (which is not yet implemented) would add dependencies for the
host "target" only.


Same for your osc change:
https://github.com/stskeeps/osc/commit/f0ba509e308d630345d0e064d7aa3856c5ae5457

If "Required(host):" would work for you - we could go with that in the osc as
well.



Best Regards,
Daniel

--
Daniel Gollub
Linux Consultant & Developer
Tel.: +49-160 47 73 970
Mail: gollub@xxxxxxxxxxxxx

B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537
< Previous Next >
Follow Ups
References