Making self-hosted OBS not rely on build.opensuse.org availability
Hello, for the Osmocom project we are hosting our own OBS instance. The projects in our instance depend on build.opensuse.org's instance for the base distributions. For example, from one meta config: <repository name="Debian_12"> <path project="openSUSE.org:Debian:12" repository="standard"/> <arch>x86_64</arch> </repository> As I understand, this is the reason that during recent outages related to openSUSE's OBS instance moving to a different datacenter, our OBS didn't build packages. After trying to start a build, in scheduler_x86_64.log: sorting 68 packages checking packages - libasn1c (interconnect error: rpc timeout) - libgtpnl (interconnect error: rpc timeout) - libosmo-abis (interconnect error: rpc timeout) … So I'm wondering, is there a way to cache/sync the base distribution information so our builds keep working even if build.opensuse.org is currently not reachable / we have a network outage etc? Thanks! Best regards, -- - Oliver Smith <osmith@sysmocom.de> https://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschaeftsfuehrer / Managing Director: Harald Welte
On Thu, Aug 31, 2023 at 9:29 AM Oliver Smith <osmith@sysmocom.de> wrote:
Hello,
for the Osmocom project we are hosting our own OBS instance. The projects in our instance depend on build.opensuse.org's instance for the base distributions. For example, from one meta config:
<repository name="Debian_12"> <path project="openSUSE.org:Debian:12" repository="standard"/> <arch>x86_64</arch> </repository>
As I understand, this is the reason that during recent outages related to openSUSE's OBS instance moving to a different datacenter, our OBS didn't build packages. After trying to start a build, in scheduler_x86_64.log:
sorting 68 packages checking packages - libasn1c (interconnect error: rpc timeout) - libgtpnl (interconnect error: rpc timeout) - libosmo-abis (interconnect error: rpc timeout) …
So I'm wondering, is there a way to cache/sync the base distribution information so our builds keep working even if build.opensuse.org is currently not reachable / we have a network outage etc?
It's been an open RFE for a while now: https://github.com/openSUSE/open-build-service/issues/7027 I would also be curious if there's been any progress on this, since openSUSE Build Service outages are so highly disruptive. -- 真実はいつも一つ！/ Always, there's only one truth!
Hey, On 31.08.23 15:23, Oliver Smith wrote:
for the Osmocom project we are hosting our own OBS instance. The projects in our instance depend on build.opensuse.org's instance for the base distributions. ... So I'm wondering, is there a way to cache/sync the base distribution information so our builds keep working even if build.opensuse.org is currently not reachable / we have a network outage etc?
Host the repos you need to build against yourself? Especially for the distribution (Download on Demand) repos (like Debian:12 or any other of the non openSUSE distros) it doesn't make much sense to go through build.o.o The prjconf changes are also infrequently, I'm sure you can mirror these with a simple cronjob. There is no magic to it. Just copy and adapt the Debian:12 repository setup to your local instance and build against Debian:12 instead of the interconnect. Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson
On Donnerstag, 31. August 2023, 17:59:16 CEST Henne Vogelsang wrote:
Hey,
On 31.08.23 15:23, Oliver Smith wrote:
for the Osmocom project we are hosting our own OBS instance. The projects in our instance depend on build.opensuse.org's instance for the base distributions. ... So I'm wondering, is there a way to cache/sync the base distribution information so our builds keep working even if build.opensuse.org is currently not reachable / we have a network outage etc?
Host the repos you need to build against yourself?
Especially for the distribution (Download on Demand) repos (like Debian:12 or any other of the non openSUSE distros) it doesn't make much sense to go through build.o.o
The prjconf changes are also infrequently, I'm sure you can mirror these with a simple cronjob.
erm, that is only true when you build against the GA repos. The moving repos like update and backports are require changes from time to time. Also we just modified the setup for arm builds atm in case you are interessted in these.
There is no magic to it. Just copy and adapt the Debian:12 repository setup to your local instance and build against Debian:12 instead of the interconnect.
That does not work atm, you need to replace the short keys for the gpg keys in the config. A remote instance does not know these (and 2.10 can not handle these at all). -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
participants (4)
-
Adrian Schröter -
Henne Vogelsang -
Neal Gompa -
Oliver Smith