19. jan. 2012 08.24 skrev Daniel Gollub :

Hi, Hi,

we currently plan to enable OBS with an additional cross-compilation feature. Which is referred in the "Build Service Concept CrossDevelopment" [1] as Type 3: "[...] the build system is modified so original source and binary packages can be build and used in a repository ".

So instead of using QEMU to emulate a different architecture completely, we want to make use of cross-compilation using the host toolchain.

Later: for executing tests or other "target"-binaries during the %build/%install/%check there will be still the need to use QEMU - which would be then a mix of Type 3 and Type 4.

We have not started yet any coding, just have done some rough concepts and wanted to make our planned effort public to involve the community in the design, to make sure everyone is happy with that effort and useful not only for us.

I should probably have noted this before on this mailing list, but I've been working on a cross compilation solution surrounding Scratchbox2 (SB2) and OBS for some months now, with an actual working solution. I haven't yet upstreamed it as we've spent time validating the solution and cleaning it up, optimizing - it currently builds my 322 package Mer Core without a hitch or source changes to all packages. The patches are available on https://github.com/stskeeps and an out-dated description page at http://wiki.merproject.org/wiki/SB2 Scratchbox2 (not really related to Scratchbox 1, as some might know) is a flexible mapping engine. that you can use it to alter the perception of file systems and ability to be choosing to execute other binaries (such as X86 binaries instead of ARM), enabling quite flexible and interesting new ways of cross compiling. It works without binfmt_misc as well, tested both in kvm and chroot. And can accelerate things like autoreconf, etc. And can even export the running of target binaries to a native host through 'sbrsh' What we do in our build patches, when SB2 is enabled (flagged by a prjconf setting), is to seperate into a build target and a build root. When SB2 isn't enabled, BUILD_TARGET == BUILD_ROOT. The 'build' script then extracts dependencies noted as SB2install: into the BUILD_ROOT and the rest into BUILD_TARGET. Currently the solution is SB2 + QEMU + toolchain + x86 tools exported using baselibs.conf, but it's sure that a more elegant solution could be done. It then proceeds to installing the target packages utilizing SB2 and then executing the build, enabling flexible control of what we want to accelerate. I just briefly wanted to introduce this work, so let me know if you'd like to know more. I'm personally planning on putting it into production with actual images and devices running it over the next weeks - I think your idea could even be done using our SB2-OBS solution. Reviews on my patches to build/obs/osc / constructive flames etc, most welcome. BR Carsten Munk -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Hi Carsten, Hi Martin, On Monday, January 23, 2012 10:27:39 PM Martin Mohring wrote:

What is missing for an upstream push is to review the code wrt. to any security violations in the pre- virtual machine phase. Side effects are unlikeley, an own code path with own "prjconf commands" are used (so no side effects to "preinstall:" or "CBinstall:" / "CBpreinstall:")

I just briefly checked the obs SB2 git repo from Carsten. Since you touch lot of code in the init_buildsystem, next to LXC conditionals, it looks like you use SB2 as VM-type. So my question is: would it still be possible to run SB2 cross-compile builds with OBS_VM_TYPE={xen,kvm,none,lxc}? Would the SB2 implementation also allow local cross compliation if someone wants to run a local obs build by calling "osc build"?

Also, the inclusion of packages from one namespace into another as they are (e.g. x86 packages to arm) needs some consideration, so meta data namespaces are not poisoned.

What about vice-versa direction: arm packages to x86?

Other than that, the current solution shows exactly what we expected from it. Anyway, QEMU is still involved in those cases as a fallback where no speedup is present. This is needed to fool spec files in cases when they call things like uname or similiar (can name here a list of other exceptions we found).

Our goal was actually to be generic as possible and to fake and emulate less then possible - and just do "real" cross-compiliation. Since most build- environments already support somehow cross-compiliation (autotools, cmake, ...). So this would very likely cause a lot of "stock" packages in the openSUSE (ARM) distro fail, if the build environment in the package is really not aware of cross-compiliation and needs to call things like uname. Of course it would be nice if everything would build without touching any project, but that is not our goal. Primary goal is to build packages/projects which are fully aware of cross- compiliation. To be clear: I see relevance for SB2 and it looks quite interesting and avoid defintely a lot of problems by make it easy as possible to get things just build, especially for packages which require some faking of the environment. So there is very likely room for your idea of cross compiliation and our idea of cross compiliation. OBS users can have the freedom to choose between different type of cross-compiliation, which is a good thing. Best Regards, Daniel -- Daniel Gollub Linux Consultant & Developer Tel.: +49-160 47 73 970 Mail: gollub@b1-systems.de B1 Systems GmbH Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537