[opensuse-buildservice] Questions about build workers (with some answers)
Hi all, after my question the other day about builds in state "broken", I realized I have a bunch of questions about build workers. Background: I'm working on a private OBS instance that was setup by a colleague. He's moving on, so I don't want to go running to him every time I have an OBS question. Instead, I'm digging into the docs and the config files and the source code, and running here when I get stumped. Here are the questions I have come up with. (Incidentally, I have already answered some of these myself by reading source code. I have not found any answers in the manuals or with Google.) Q1) What defines a build worker? A config file? Something in the database? It just shows up, announces itself, and starts building? (A: from reading /etc/init.d/obsworker, it looks like a variety of config files in /etc, mostly /etc/sysconfig/obs-server, define the build workers *on this host*.) Q2) I have 4 build workers defined on my OBS server: are they interchangeable, or do they all build for different targets? (A: from /etc/init.d/obsworker plus config files, it looks very much like they are interchangeable.) Q3) How do I know which build targets (operating systems, architectures, etc.) a given build worker -- or perhaps I should say, all the build workers on a certain host -- can handle? IOW, why are my build workers capable of building for Oracle Linux 7 and CentOS 7, but not Scientific Linux 6.7? They are all on an openSUSE host, the OBS appliance. Q4) How do I find out if I have any build workers that can handle a given target, e.g. Scientific Linux 6.7? Q5) Can I run i586 builds on an x86_64 machine? Or do I need a 32-bit build host? (I only care about i586 and x86_64, so I should not need anything fancier than a chroot for builds.) Q6) How do I add a new build worker that can handle builds for Scientific Linux 6.7? Q7) How do I add new build workers on a separate host? Does that host have to be running openSUSE, or will the worker run on any sufficiently compatible OS? (Like, say, CentOS 7.) (A: oops, I just discovered the Admin Guide. It was not very well hidden at all. I guess I'm just blind. I'm hoping I will find the answer in there.) Thanks! Greg -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Mittwoch, 15. Februar 2017, 17:02:22 CET wrote Greg Ward: ..
Q1) What defines a build worker? A config file? Something in the database? It just shows up, announces itself, and starts building? (A: from reading /etc/init.d/obsworker, it looks like a variety of config files in /etc, mostly /etc/sysconfig/obs-server, define the build workers *on this host*.)
the bs_worker code gets downloaded from the server by the obsworker init script and it is registering itself at the server.
Q2) I have 4 build workers defined on my OBS server: are they interchangeable, or do they all build for different targets? (A: from /etc/init.d/obsworker plus config files, it looks very much like they are interchangeable.)
in general they are used for all builds. Check the chapter about build constraints to learn how to filter them.
Q3) How do I know which build targets (operating systems, architectures, etc.) a given build worker -- or perhaps I should say, all the build workers on a certain host -- can handle? IOW, why are my build workers capable of building for Oracle Linux 7 and CentOS 7, but not Scientific Linux 6.7? They are all on an openSUSE host, the OBS appliance.
there is no limit, except the worker capabilities like architecture, disk space, memory and so on. Again, the constraints chapter should be helpful.
Q4) How do I find out if I have any build workers that can handle a given target, e.g. Scientific Linux 6.7?
all can usually
Q5) Can I run i586 builds on an x86_64 machine? Or do I need a 32-bit build host? (I only care about i586 and x86_64, so I should not need anything fancier than a chroot for builds.)
yes, just the "linux32" binary needs to be available.
Q6) How do I add a new build worker that can handle builds for Scientific Linux 6.7?
only general workers exist.
Q7) How do I add new build workers on a separate host? Does that host have to be running openSUSE,
no
or will the worker run on any sufficiently compatible OS? (Like, say, CentOS 7.) (A: oops, I just discovered the Admin Guide. It was not very well hidden at all. I guess I'm just blind. I'm hoping I will find the answer in there.)
good spot ;) -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Thu, Feb 16, 2017, at 02:49, Adrian Schröter wrote:
On Mittwoch, 15. Februar 2017, 17:02:22 CET wrote Greg Ward:
Q7) How do I add new build workers on a separate host? Does that host have to be running openSUSE,
no
So I've found this bit of documentation on adding worker hosts: http://openbuildservice.org/help/manuals/obs-admin-guide/obs.cha.installatio... but I cannot figure out how to do it on a non-openSUSE system. Here's what I've done: * copied /etc/init.d/obsworker from my OBS server appliance to a different host, which is running Scientific Linux 6.7 (I have a couple of CentOS 7 boxes around too if that would help, but most of our infrastructure is still on SL 6) * copied /etc/sysconfig/obs-server from the OBS server to the build host * created /data/obs (/data has lots of space) and tweaked various settings in /etc/sysconfig/obs-server to use dirs under /data/obs * ran "/etc/init.d/obsworker status", expecting a "not running" message Problems: * no such file /etc/rc.status. So I commented out the line that sources it. * /etc/init.d/obsworker: line 234: rc_reset: command not found -> defined empty rc_reset function * Checking for obsworker: /etc/init.d/obsworker: line 401: checkproc: command not found -> hmmm: this probably contains non-trivial logic Conclusion: /etc/init.d/obsworker really only works on openSUSE. I guess that is why the Admin Guide refers to a "worker appliance". But I cannot find such an appliance on the download page, and the Admin Guide does not link to anything. Should I be using this: https://en.opensuse.org/openSUSE:Build_Service_Appliance#Worker_Installation ? Has anyone built a container for this? I want as few layers of virtualization between my builds and the hardware as possible! Thanks, Greg -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Donnerstag, 16. Februar 2017, 16:48:27 CET wrote Greg Ward:
On Thu, Feb 16, 2017, at 02:49, Adrian Schröter wrote:
On Mittwoch, 15. Februar 2017, 17:02:22 CET wrote Greg Ward:
Q7) How do I add new build workers on a separate host? Does that host have to be running openSUSE,
no
So I've found this bit of documentation on adding worker hosts:
http://openbuildservice.org/help/manuals/obs-admin-guide/obs.cha.installatio...
but I cannot figure out how to do it on a non-openSUSE system. Here's what I've done:
* copied /etc/init.d/obsworker from my OBS server appliance to a different host, which is running Scientific Linux 6.7 (I have a couple of CentOS 7 boxes around too if that would help, but most of our infrastructure is still on SL 6)
* copied /etc/sysconfig/obs-server from the OBS server to the build host
* created /data/obs (/data has lots of space) and tweaked various settings in /etc/sysconfig/obs-server to use dirs under /data/obs
* ran "/etc/init.d/obsworker status", expecting a "not running" message
Problems:
* no such file /etc/rc.status. So I commented out the line that sources it. * /etc/init.d/obsworker: line 234: rc_reset: command not found -> defined empty rc_reset function * Checking for obsworker: /etc/init.d/obsworker: line 401: checkproc: command not found -> hmmm: this probably contains non-trivial logic
Try to use the bs_worker directly. (or write your own init/systemd file). You can run it with bs_worker --root /space/... --statedir /space/...statedir --reposerver $your_obs_server:5152 ...
Has anyone built a container for this? I want as few layers of virtualization between my builds and the hardware as possible!
We have the OBS-WorkerOnly appliances in our projects. A container (like Docker without an own kernel) is often not good enough, since builds can depend on a kernel. The build instance also tries to use an own kernel (within KVM or XEN) matching for the build process. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (2)
-
Adrian Schröter -
Greg Ward