[opensuse-buildservice] How bs_worker works?
Hi, I'm trying to understood architecture of OBS. I find that /etc/init.d/obsworker starts /usr/lib/obs/server/bs_worker and that worker can be configure to build in kvm sandbox. But I got lost in bs_worker code. Can someone please enlighten me (or just point to specific parts of code) how is sources passed to that kvm sandbox and how is retrieved the result from the virtual machine? Mirek -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 03.07.2013, at 20:46, Miroslav Suchy <msuchy@redhat.com> wrote:
Hi, I'm trying to understood architecture of OBS. I find that /etc/init.d/obsworker starts /usr/lib/obs/server/bs_worker and that worker can be configure to build in kvm sandbox. But I got lost in bs_worker code. Can someone please enlighten me (or just point to specific parts of code) how is sources passed to that kvm sandbox and how is retrieved the result from the virtual machine?
Mirek -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
There is no architecture docs (the same for fedora build processes). obsworker init script reads /etc/sysconfig/obs-server and starts builds (it fetches bs worker code from src server).-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am Mittwoch, 3. Juli 2013, 23:18:08 schrieb Kanstantsin Shautsou:
On 03.07.2013, at 20:46, Miroslav Suchy <msuchy@redhat.com> wrote:
Hi, I'm trying to understood architecture of OBS. I find that /etc/init.d/obsworker starts /usr/lib/obs/server/bs_worker and that worker can be configure to build in kvm sandbox. But I got lost in bs_worker code. Can someone please enlighten me (or just point to specific parts of code) how is sources passed to that kvm sandbox and how is retrieved the result from the virtual machine?
All that handling is done by the build script (openSUSE/obs-build on github). The worker is just downloading sources and binaries, calls build and returns the result. ./bs_worker --help may give an idea about the options. -- Adrian Schroeter email: adrian@suse.de SUSE LINUX GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Also it listen tcp ports (btw why?) in case you want to configure firewall on worker. On 04.07.2013, at 10:53, Adrian Schröter <adrian@suse.de> wrote:
Am Mittwoch, 3. Juli 2013, 23:18:08 schrieb Kanstantsin Shautsou:
On 03.07.2013, at 20:46, Miroslav Suchy <msuchy@redhat.com> wrote:
Hi, I'm trying to understood architecture of OBS. I find that /etc/init.d/obsworker starts /usr/lib/obs/server/bs_worker and that worker can be configure to build in kvm sandbox. But I got lost in bs_worker code. Can someone please enlighten me (or just point to specific parts of code) how is sources passed to that kvm sandbox and how is retrieved the result from the virtual machine?
All that handling is done by the build script (openSUSE/obs-build on github). The worker is just downloading sources and binaries, calls build and returns the result.
./bs_worker --help
may give an idea about the options.
--
Adrian Schroeter email: adrian@suse.de
SUSE LINUX GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Wed, Jul 03, 2013 at 07:46:15PM +0200, Miroslav Suchy wrote:
I'm trying to understood architecture of OBS. I find that /etc/init.d/obsworker starts /usr/lib/obs/server/bs_worker and that worker can be configure to build in kvm sandbox. But I got lost in bs_worker code. Can someone please enlighten me (or just point to specific parts of code) how is sources passed to that kvm sandbox and how is retrieved the result from the virtual machine?
The worker just fetches the sources from the source server and the binary packages from the repo server and calls /usr/bin/build from the "build" package to do the building and sandboxing. "build" is like Fedora's "mock", but a couple of years older ;) The build program creates the file system, mounts it, copies the sources and binary packages into it and does the "preinstall" phase, i.e. it unpacks the packages needed to get rpm/bash running. This unpacking is done by using "rpm2cpio" and "bsdtar" for security reasons. It then unmounts the file system and starts the virtual machine. The "init" program used in the vm is actually the build script again, it then installs all the rpm packages and runs rpmbuild. Getting the build result out of the vm is a bit tricky, as we don't want to mount the filesystem again after the vm was running due to security reasons. Instead, the build script running inside the vm disables swap and writes a list of file system blocks to the swap device. Then the vm terminates, and the outer build script reads the swap space to get the block list and extracts the built rpms. Then the build script terminates and the worker sends the result back to the repo server, which sends an event to the scheduler process to make it pick up the build result. (The worker also listens to a port, so you can stream the build log and kill running jobs.) Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 07/04/2013 11:21 AM, Michael Schroeder wrote:
The worker just fetches the sources from the source server and the binary packages from the repo server and calls /usr/bin/build from the "build" package to do the building and sandboxing. "build" is like Fedora's "mock", but a couple of years older;)
The build program creates the file system, mounts it, copies the sources and binary packages into it and does the "preinstall" phase, i.e. it unpacks the packages needed to get rpm/bash running. This unpacking is done by using "rpm2cpio" and "bsdtar" for security reasons.
It then unmounts the file system and starts the virtual machine. The "init" program used in the vm is actually the build script again, it then installs all the rpm packages and runs rpmbuild.
Getting the build result out of the vm is a bit tricky, as we don't want to mount the filesystem again after the vm was running due to security reasons. Instead, the build script running inside the vm disables swap and writes a list of file system blocks to the swap device. Then the vm terminates, and the outer build script reads the swap space to get the block list and extracts the built rpms.
Then the build script terminates and the worker sends the result back to the repo server, which sends an event to the scheduler process to make it pick up the build result.
(The worker also listens to a port, so you can stream the build log and kill running jobs.)
Thanks Michael, that was the answer I was looking for. -- Miroslav Suchy Red Hat, Software Engineer -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (5)
-
Adrian Schröter
-
Kanstantsin Shautsou
-
Michael Schroeder
-
Miroslav Suchy
-
Miroslav Suchý