Hi Carsten, On Monday, February 27, 2012 02:17:34 PM Christian Schneemann wrote: [...]
Preinstall: and VMinstall: will always go into hostarch (which makes sense) in order to have a bootable environment
Right.
What scope will BuildRequires: dependencies from the package be resolved with? Will it look within the <scheduler> that it is being built within?
Will BuildRequires: be dep resolved on all architectures or just one?
I just pushed out something which starts addressing this issues: https://github.com/b1-systems/obs-build/commit/930554fe36992097c2b200e6db74a... https://github.com/b1-systems/open-build-service/commit/91bece36f12a8095d6cf... Since we want to have _Transpartent_ cross builds (which means not too much fidlling in each spec file with regards to BuildRequires and other things) I enabled Build::get_build and Build::get_deps to: 1. Substitue cross build dependency e.g. toolchain. Like this (in your prjconf): Substitute: gcc cross-gcc-armv7l(HOST) (no modification on Build.pm required for this) The idea is to assign dependencies to a specific dependency-tree/sysroot by adding a suffix "package(SYSROOT_LABEL)". In the above example and this first the build dependency would end up in: cross-gcc-armv7l(HOST) 2. Additional code directly after the Build::do_subst function get called: - Build::extract_crossdeps extracts cross dependencies, which are marked like this "package(SYSROOT_LABEL)". This function will then return a hash with the SYSROOT_LABEL as key and an array/list of package name which are dependencies for this SYSROOT_LABEL. So this is suitable for N-sysroots. This hash get stored as seperated seperated cross dependency variable and returnted later. 3. In the next step Build::drop_crossdeps will drop cross dependecies from the original dependecy list. Since the expander function would fail to resolve the dependency of: "coss-gcc-armv7l(HOST)" So all those "package(SYSROOT_LABEL)" get dropped from the original list. 4. unmodified Build::expand or BSSolv.xs exapnder get called. The dependency list will only cover native dependencies - so it should expand as always. This requires that the (cross) Substitute from did not break anything. In some cases you might do something like this to no break the expander: Substitute: libmy-devel libmy-devel libmy-devel(HOST) 5. Build::get_deps / Build::get_build returns the expanded dependency list for the target sysroot. The other additional dependencies for the host or other sysroot got not expanded or resolved by Build::get_deps / Build::get_build. I have not tried to resolve them on purpose inside Build:: since it would not use the expander pool from BSSolv of different scheduler instances. The return value slightly changed of Build::get_deps / Build::get_build function: - return ($eok, @deps) + return ($eok, \@deps, \%crossdeps) (if anyone has an idea how to avoid this - please let us know) 6. All the bs_sched (one per architecture) instances call Build::get_deps / Build::get_build once a build need or so get kicked off. Now the scheduler retrives multiple dependency lists: - one resolved for the native architecture for the sysroot target -> @bdeps (like it was before) - one or more unresolved for other sysroots (incl. the host sysroot) -> %crossdeps TODO: find a way how to handle the dependencies for other sysroots Some rough ideas: - dependencies of sysroots with the same architecture could be resolved in the same scheduler instance and block the build or let it fail due unresolved package - dependencies of sysroots with other architecture: * probe/call the other scheduler instances and ask to resolve the dependency? * check if we can get this information from the meta data on the disk? Not yet solved: blocking builds if a dependency from a other architectures is not ready/available. 7. set_building (from bs_sched) get called with an additional parameter: %crossdeps (Since %crossbuild is not yet expanded/resolved correctly - your cross dependecies might not prevent the scheduler to kick off a build which will fail to due missing cross dependencies) Christian is working right now on checking if there %crossdeps is filled with additional dependencies and add those as dependecy into the buildjob file: <bdep name="cross-gcc-armv7l" sysroot="HOST" /> 8. bs_worker retrieves the job and assembles for each sysroot a differnet package list and stores the package also in seperated directories. 9. build script will get called with additional parameters to setup different sysroots. IIRC something like this: build [..] --rpmlist /path/to/worker/package/host/packagelist --sysroot /path/to/sysroot:/path/to/worker/target/pkglist \ --sysroot /path/sysroot2:/path/to/worker/target2/pkglist \ --sysroot /path/sysrootN:/path/to/worker/targetN/pkglist \ --target-sysroot /path/to/sysroot \ --target armv7l 10. init_buildsystem (WIP) setups for each sysroot a seperated package database. Installs package into each sysroot. 11. build kicksoff rpmbuild [...] --target armv7l --root $TARGET_SYSROOT (WIP) ...
Have you considered how to deal with x86_64 cross compiling to x86_64 target as well? (or i586->i586) - that would be a typical use case for 'daily development' kind of things, demo embedded system in a X86 VM for example. That's why I was thinking of labels instead for scheduler.
Not considered yet, but that should be possible. We are using now labels too :)
Right. Since we moved now from AdditoinalArch: to Sysroot: it should be possible to have whatever archiecture you want as additional sysroot. I guess you could even map the Host-Sysroot to something else then /, but then you still need to make sure that your BUILD_ROOT is still works with the buildscript somehow.
Won't you be having an inconsistent RPM database in the target, as in, 'gcc' package name is not represented in package names when doing substitutions?
If we subsitute, lets say "gcc", so it results in cross-gcc-armv7l or something like that - then there would be no BuildRequires/Dependency anylonger for the target once the the job file get assemabled by the scheduler. But as already said, you could also try do some substitution tricks like: Substitute: gcc cross-gcc-armv7l(HOSTLABEL) gcc Which would add cross-gcc-armv7l as dependency for the sysroot with the label HOSTLABLEL and it would be still a regular buildrequires for the target. This way could populated the other sysroots as well by doing this: Subsitute: libfoo libfoo libfoo(petproject_armv7l) libfoo(petproject_i586) or even do this directly in your spec file: BuildRequires: libfoo(petproject_armv7l) libfoo(petproject_i586) [..] Best Regards, Daniel -- Daniel Gollub Linux Consultant & Developer Tel.: +49-160 47 73 970 Mail: gollub@b1-systems.de B1 Systems GmbH Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537