[opensuse-buildservice] different packages installed in local build chroot than in OBS build
Hi! As already initially discussed on IRC I have observed that my local chroot build environments get different packages installed than the build environment on the OBS workers. I think the discussion there confirmed my assumption that the local chroot should have exactly the same contents as the build environment on the host[1]. Both OBS and my local environment are just chroot, no virtualization[2]. I have a hierarchy (<path> in buildinfo) of 4 projects, the "lowest" one has a project configuration. I'm using the highest one and there I see the difference in the build roots. The main properties are: - worker architecture is x86_64, same for my local machine - target architecture is arm, all execection uses binfmt_misc and qemu-arm-static The differences are related to some preinstalls. The preinstalls are defined in the project configuration of the "lowest" project, e. g. like this %if %speedbash Preinstall: bash-x86-arm ncurses-libs-x86-arm Runscripts: bash-x86-arm %endif (This is just one example, there are several similar cases in the whole project config) This configuration is copied correctly into .osc/_buildconfig... when I run osc build. However, it appears that on the OBS the %speedbash macro is true (resulting in some x86 preinstalls), in my local builds it must be false (these preinstalls are missing). So I end up with different preinstalls locally than remote. The %speedbash macro depends on other %if macros, but in the beginning of the chain some macro must just be initialized differently on the worker host than locally. How can I find out where these macros are actually intialized and where the difference comes from? When looking at http://en.opensuse.org/openSUSE:Build_Service_prjconf, I read:
The prjconf denotes the (build) configuration of a project. The main part of this file (/usr/lib/build/configs/$distro.conf) is the definition of the build environment
So do these conf files actually play a role in the initialization of the macros? If yes, is it the one on the host or the one in the chroot? I don't really understand how that should work: We are talking about the initialization of the chroot, so I can't see how the conf files from inside the chroot could be used, because they are not there yet when we need to decide our preinstalls. But on the other side if the ones of the host are used, the host could influence the contents of the chroot, contradicting the purpose of the chroot being a well-defined isolated build envoironment. Any explanations to solve this knot are welcome. Or are these conf files for the "second" kind of macros only, which are used by rpmbuild, not those used in the prjconf? Any hints to trace down were the difference is created? I have already tried to debug osc a bit, but at least so far I have not found any place where these macros are handled. When I execute "osc build", is the difference likely to be caused already on the OBS (I don't have any access to the installation than a normal user account) and included e.g. in the buildinfo sent from the OBS to my local build or does the relevant processing happen first locally in my own machine (where I could debug it)? Regards, Uwe Geuder Nomovok Ltd. Tampere, Finland uwe.gxuder@xxxxxxxxxxx (bot check: humans correct 1 obvious spelling error) [1] As far as installed packages are concerned. Of course there will be other differences like the kernel, the amount of memory and a million other parameters that could cause nasty problems if things go really wrong. [2] chroot in OBS is a security risk. This is a private OBS instance in a closed intranet with only "trusted" users. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am Dienstag, 6. März 2012, 12:44:00 schrieb Uwe Geuder: ...
The %speedbash macro depends on other %if macros, but in the beginning of the chain some macro must just be initialized differently on the worker host than locally. How can I find out where these macros are actually intialized and where the difference comes from?
Check with eg: osc api /build/openSUSE:Tools/openSUSE_Factory/_buildconfig what your configured build config is. There are also comments from which project they exactly come from.
When looking at http://en.opensuse.org/openSUSE:Build_Service_prjconf, I
read:
The prjconf denotes the (build) configuration of a project. The main part of this file (/usr/lib/build/configs/$distro.conf) is the definition of the build environment
So do these conf files actually play a role in the initialization of the macros? If yes, is it the one on the host or the one in the chroot?
Not in the OBS case. They are only there for usage of "build" standalone.
don't really understand how that should work: We are talking about the initialization of the chroot, so I can't see how the conf files from inside the chroot could be used, because they are not there yet when we need to decide our preinstalls. But on the other side if the ones of the host are used, the host could influence the contents of the chroot, contradicting the purpose of the chroot being a well-defined isolated build envoironment. Any explanations to solve this knot are welcome. Or are these conf files for the "second" kind of macros only, which are used by rpmbuild, not those used in the prjconf?
Any hints to trace down were the difference is created? I have already tried to debug osc a bit, but at least so far I have not found any place where these macros are handled. When I execute "osc build", is the difference likely to be caused already on the OBS (I don't have any access to the installation than a normal user account) and included e.g. in the buildinfo sent from the OBS to my local build or does the relevant processing happen first locally in my own machine (where I could debug it)?
Regards,
Uwe Geuder Nomovok Ltd. Tampere, Finland uwe.gxuder@xxxxxxxxxxx (bot check: humans correct 1 obvious spelling error)
[1] As far as installed packages are concerned. Of course there will be other differences like the kernel, the amount of memory and a million other parameters that could cause nasty problems if things go really wrong.
[2] chroot in OBS is a security risk. This is a private OBS instance in a closed intranet with only "trusted" users. -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 7 March 2012 11:36, Adrian Schröter <adrian suse.de> wrote:
Check with eg:
osc api /build/openSUSE:Tools/openSUSE_Factory/_buildconfig
what your configured build config is. There are also comments from which project they exactly come from.
Thanks for your reply. Well, this api call returns exactly what "osc build" stores into the local .osc/_buildconfig... file, which I have already looked at.
There are also comments from which project they exactly come from.
Yes, I had already seen them and it looked all correct to me. But the issue is still the project configuration contains conditional code like the mentioned
%if %speedbash Preinstall: bash-x86-arm ncurses-libs-x86-arm Runscripts: bash-x86-arm %endif
(This part comes from the "root" project, which I "inherit" from. I guess it should not matter where it comes from besides that "trial and error testing" is even more limited, because I cannot directly edit these parts.) As I wrote before the OBS worker seems to evaluate the condition to true (at least it installs these Preinstalls) and my local build seems to evaluate the same condition to false (it does not install those). If I run "osc -d build" I can see the steps. I guess the evaluation happens somewhere between the messages
Getting buildconfig from server and store to /home/geuder/projects/xyz/nvc/bini/home:uwe.geuder:test1/test-release/.osc/_buildconfig-release-1-2-armv7el GET https://api.foo.nomovok.info/build/home:uwe.geuder:test1/release-1-2/_buildc...
and
Updating cache of required packages
Looks like I have to dig deeper what happens between these 2 messages (didn't have time to do so yesterday). I guess somehow there must be a macro expansion step if this buildconfig info is at all locally used to determine which packages to preinstall. However, the "is at all used" seems to be a very big "IF" to me. Somehow I have difficulties to see why "osc build" would at all use buildconfig data to determine what packages are needed. After all it has already fetched the buildinfo
Getting buildinfo from server and store to /home/geuder/projects/xyz/nvc/bini/home:uwe.geuder:test1/meego-release/.osc/_buildinfo-release-1-2-armv7el.xml POST https://api.foo.nomovok.info/build/home:uwe.geuder:test1/release-1-2/armv7el...
which seems to contain a list of what to install and even a flag for each package whether it's a preinstall or not. So I could guess the macro expansion in the buildconfig has already happened in the OBS (scheduler?) and there is little I could do locally anymore (both debugging and influencing the outcome). Or do packages inferred from buildconfig add up to the list contained in buildinfo? Regards, Uwe Geuder Nomovok Ltd. Tampere, Finland uwe.gxuder@xxxxxxxxxxx (bot check: humans correct 1 obvious spelling error) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (2)
-
Adrian Schröter
-
Uwe Geuder