[opensuse-buildservice] Bootstrapping OBS
Hello all, I’m installing a private instance of OBS for a customer, and so far I managed to bring up all the services (including the frontend). However, I’m not having much success on bootstrapping it. After creating an empty SUSE:SLE-12-SP1:GA project, added a project config (pulled from the corresponding project on build.opensuse.org), I tried two approaches to bootstrapping it: 1) created a “standard” x86_64 DoD repository via the UI, pointing the URL to a local HTTP server hosting the installation image for SLES 12 SP1. I noticed errors on the logs trying to download “repodata”. Ok, it expects a proper repository created by createrepo, which is not the case. Let’s try something else. 2) copied all the binary RPMs to a staging area and proceeded as noted on the documentation (stop the scheduler and move everything to the corresponding :full directory, each package renamed to just “.rpm” without a version). Even though a :full.solv file was created after a while, I still see no packages at all for the project. What am I missing here? I also activated the OBS interconnect with openSUSE.org, but I cannot find any of the projects from build.opensuse.org on the local search. Is that intended? I wanted to branch a package hosted on OBS. — Erico
On Dienstag, 10. Mai 2016, 19:25:01 wrote Erico Mendonca:
Hello all,
I’m installing a private instance of OBS for a customer, and so far I managed to bring up all the services (including the frontend). However, I’m not having much success on bootstrapping it.
After creating an empty SUSE:SLE-12-SP1:GA project, added a project config (pulled from the corresponding project on build.opensuse.org), I tried two approaches to bootstrapping it:
1) created a “standard” x86_64 DoD repository via the UI, pointing the URL to a local HTTP server hosting the installation image for SLES 12 SP1. I noticed errors on the logs trying to download “repodata”. Ok, it expects a proper repository created by createrepo, which is not the case. Let’s try something else.
we have not published repos, so DoD is not a good idea. interconnect is better ...
2) copied all the binary RPMs to a staging area and proceeded as noted on the documentation (stop the scheduler and move everything to the corresponding :full directory, each package renamed to just “.rpm” without a version). Even though a :full.solv file was created after a while, I still see no packages at all for the project.
source package containers? no, you don't see them.
What am I missing here?
I also activated the OBS interconnect with openSUSE.org, but I cannot find any of the projects from build.opensuse.org on the local search. Is that intended? I wanted to branch a package hosted on OBS.
Yes, but you can anyway access it. Try for example: osc api /build/openSUSE.org:SUSE:SLE-12:GA/standard/x86_64/zip you should see a listing via the interconnect. You can also build against that repo in your instance. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
2) copied all the binary RPMs to a staging area and proceeded as noted on the documentation (stop the scheduler and move everything to the corresponding :full directory, each package renamed to just “.rpm” without a version). Even though a :full.solv file was created after a while, I still see no packages at all for the project.
source package containers? no, you don't see them.
How should I proceed to replicate these locally without interconnect? I may be having firewall/proxy issues here.
What am I missing here?
I also activated the OBS interconnect with openSUSE.org, but I cannot find any of the projects from build.opensuse.org on the local search. Is that intended? I wanted to branch a package hosted on OBS.
Yes, but you can anyway access it. Try for example:
osc api /build/openSUSE.org:SUSE:SLE-12:GA/standard/x86_64/zip
you should see a listing via the interconnect. You can also build against that repo in your instance.
Hmm, not working here. I get a 502 error after a long time. Does it need access to anything other than api.opensuse.org on port 443? I’m behind an authenticated proxy + firewall... -- — Erico Mendonça Dedicated Support Engineer SUSE N�����r��y隊Z)z{.���Wlz��qﮞ˛���m�)z{.��+�: �{Zr�az�'z��j)h���Ǜ�)]���Ǿ� ޮ�^�ˬz��
On Dienstag, 10. Mai 2016, 20:42:28 wrote Erico Mendonca:
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
2) copied all the binary RPMs to a staging area and proceeded as noted on the documentation (stop the scheduler and move everything to the corresponding :full directory, each package renamed to just “.rpm” without a version). Even though a :full.solv file was created after a while, I still see no packages at all for the project.
source package containers? no, you don't see them.
How should I proceed to replicate these locally without interconnect? I may be having firewall/proxy issues here.
hm, best way is maybe for you to copy the binaries to some directory, call "createrepo" manual there and use that as DoD URL (we should really enable rpm-md meta data on our SLE medias ...) Please note that you will need at least also the SDK, depending on what you want to build.
What am I missing here?
I also activated the OBS interconnect with openSUSE.org, but I cannot find any of the projects from build.opensuse.org on the local search. Is that intended? I wanted to branch a package hosted on OBS.
Yes, but you can anyway access it. Try for example:
osc api /build/openSUSE.org:SUSE:SLE-12:GA/standard/x86_64/zip
you should see a listing via the interconnect. You can also build against that repo in your instance.
Hmm, not working here. I get a 502 error after a long time. Does it need access to anything other than api.opensuse.org on port 443? I’m behind an authenticated proxy + firewall...
Do you have configured the proxy either in /etc/sysconfig/proxy or /usr/lib/obs/server/BSConfig.pm our BSConfig::proxy = ...; ? Do you see error messages in /srv/obs/log/src_server.log ? Can you reach api.opensuse.org manualy with curl using the proxy from that server? -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
On Dienstag, 10. Mai 2016, 20:42:28 wrote Erico Mendonca:
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
2) copied all the binary RPMs to a staging area and proceeded as noted on the documentation (stop the scheduler and move everything to the corresponding :full directory, each package renamed to just “.rpm” without a version). Even though a :full.solv file was created after a while, I still see no packages at all for the project.
source package containers? no, you don't see them.
How should I proceed to replicate these locally without interconnect? I may be having firewall/proxy issues here.
hm, best way is maybe for you to copy the binaries to some directory, call "createrepo" manual there and use that as DoD URL (we should really enable rpm-md meta data on our SLE medias ...)
Please note that you will need at least also the SDK, depending on what you want to build.
Working on that right now. I noticed that it won’t accept file:/// URLs, I had to place the files accessible via http.
Hmm, not working here. I get a 502 error after a long time. Does it need access to anything other than api.opensuse.org on port 443? I’m behind an authenticated proxy + firewall...
Do you have configured the proxy either in
/etc/sysconfig/proxy
or
/usr/lib/obs/server/BSConfig.pm our BSConfig::proxy = ...;
?
Do you see error messages in /srv/obs/log/src_server.log ?
Can you reach api.opensuse.org manualy with curl using the proxy from that server?
Yes, I had the proxy defined in sysconfig, and curl works. But repeating the osc api command with the —http-full-debug parameter, I was able to see the proxy error messages. I added “our $proxy = …” and a corresponding “our $noproxy=…” to BSConfig.pm, and things started working. I’m now able to perform operations o openSUSE.org projects via interconnect. However, the frontend search won’t return results from build.opensuse.org. Is that intentional? -- — Erico Mendonça Dedicated Support Engineer SUSE N�����r��y隊Z)z{.���Wlz��qﮞ˛���m�)z{.��+�: �{Zr�az�'z��j)h���Ǜ�)]���Ǿ� ޮ�^�ˬz��
On Mittwoch, 11. Mai 2016, 17:50:25 wrote Erico Mendonca:
On Dienstag, 10. Mai 2016, 20:42:28 wrote Erico Mendonca:
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
2) copied all the binary RPMs to a staging area and proceeded as noted on the documentation (stop the scheduler and move everything to the corresponding :full directory, each package renamed to just “.rpm” without a version). Even though a :full.solv file was created after a while, I still see no packages at all for the project.
source package containers? no, you don't see them.
How should I proceed to replicate these locally without interconnect? I may be having firewall/proxy issues here.
hm, best way is maybe for you to copy the binaries to some directory, call "createrepo" manual there and use that as DoD URL (we should really enable rpm-md meta data on our SLE medias ...)
Please note that you will need at least also the SDK, depending on what you want to build.
Working on that right now. I noticed that it won’t accept file:/// URLs, I had to place the files accessible via http.
right, this is currently a wanted limitation. We may add some option later to allow it, but we think it should not be used in production...
Hmm, not working here. I get a 502 error after a long time. Does it need access to anything other than api.opensuse.org on port 443? I’m behind an authenticated proxy + firewall...
Do you have configured the proxy either in
/etc/sysconfig/proxy
or
/usr/lib/obs/server/BSConfig.pm our BSConfig::proxy = ...;
?
Do you see error messages in /srv/obs/log/src_server.log ?
Can you reach api.opensuse.org manualy with curl using the proxy from that server?
Yes, I had the proxy defined in sysconfig, and curl works. But repeating the osc api command with the —http-full-debug parameter, I was able to see the proxy error messages. I added “our $proxy = …” and a corresponding “our $noproxy=…” to BSConfig.pm, and things started working. I’m now able to perform operations o openSUSE.org projects via interconnect.
However, the frontend search won’t return results from build.opensuse.org. Is that intentional?
yes -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hi Erico, Am 10.05.2016 um 21:25 schrieb Erico Mendonca:
Hello all,
I’m installing a private instance of OBS for a customer, and so far I managed to bring up all the services (including the frontend). However, I’m not having much success on bootstrapping it.
After creating an empty SUSE:SLE-12-SP1:GA project, added a project config (pulled from the corresponding project on build.opensuse.org), I tried two approaches to bootstrapping it:
1) created a “standard” x86_64 DoD repository via the UI, pointing the URL to a local HTTP server hosting the installation image for SLES 12 SP1. I noticed errors on the logs trying to download “repodata”. Ok, it expects a proper repository created by createrepo, which is not the case. Let’s try something else.
2) copied all the binary RPMs to a staging area and proceeded as noted on the documentation (stop the scheduler and move everything to the corresponding :full directory, each package renamed to just “.rpm” without a version). Even though a :full.solv file was created after a while, I still see no packages at all for the project.
What am I missing here?
Nothing I think, you simple do not see these build repositories, since they do not have any source packages. Here is what I'm doing on sites which cannot use interconnect or want local build repositories: I create the build project as you did, use "standard" as repository name, add the project config. Note that SLE12-SP1 also need the SLE12 project config. Then I populate the /srv/obs/build/DISTRI/standard/x86_64/:full directory with symlinks to the loop mounted install media, with SLE you also need the SDK and maybe addons, you can also use symlinks to a local SMT if you have one. Here is no need to strip the version info. You need to add x86_64 RPMs and the noarch RPMS here. Make sure the full path is owned by obsrun:obsrun and writable (if not the scheduler will crash). Then run obs_admin --rescan-repository DISTRI standard x86_64 This will create the :full.solv file and now you can add this Project as build target to your local projects via the advanced interface in the GUI. Cheers Karsten -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
Nothing I think, you simple do not see these build repositories, since they do not have any source packages.
Here is what I'm doing on sites which cannot use interconnect or want local build repositories:
...
Thanks! That was really useful and got things going here. I was able to do an osc copypac from build.opensuse.org to my home dir, checked it out with osc and built the package locally using these repos. However, I still cannot build it remotely. Via osc, I can see that it’s been requested (a star shows beside each platform in “osc results”), but nothing happens. The frontend shows only “No build result available” instead of the repositories enabled for the package. Haven’t seen error messages in the logs. Any idea? -- — Erico Mendonça Dedicated Support Engineer SUSE
On Mittwoch, 11. Mai 2016, 17:57:18 wrote Erico Mendonca:
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
Nothing I think, you simple do not see these build repositories, since they do not have any source packages.
Here is what I'm doing on sites which cannot use interconnect or want local build repositories:
...
Thanks! That was really useful and got things going here.
I was able to do an osc copypac from build.opensuse.org to my home dir, checked it out with osc and built the package locally using these repos.
However, I still cannot build it remotely. Via osc, I can see that it’s been requested (a star shows beside each platform in “osc results”), but nothing happens. The frontend shows only “No build result available” instead of the repositories enabled for the package. Haven’t seen error messages in the logs.
Any idea?
usually the scheduler is not running for this architecture you may want to check your /monitor page for missing services For reference: https://build.opensuse.org/monitor -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Em [DATE], "[NAME]" <[ADDRESS]> escreveu:
usually the scheduler is not running for this architecture
you may want to check your /monitor page for missing services For reference:
Hmmm my monitor page shows active processes for deltastore, armv7l, i586, x86_64, dispatcher, publisher, signer, warden and dodup. OpenSUSE’s show me extra processes, like repserver, service and srcserver. When I trigger rebuild from the UI or OSC I see it arriving on rep_server.log but I see this on the corresponding scheduler log: -home:Admin/SLE_11_SP4 Repository ‘SUSE:SLE-11-SP4/standard’ is unavailable And this repeats for all the repositories I have. The weird thing is, the repositories are populated, have a :full.solv file, and work when I ask for a local build via OSC. However, I have five workers that refuse to work. How can I whip them into working? :P -- — Erico Mendonça Dedicated Support Engineer SUSE
Hmmm my monitor page shows active processes for deltastore, armv7l, i586, x86_64, dispatcher, publisher, signer, warden and dodup.
OpenSUSE’s show me extra processes, like repserver, service and srcserver.
When I trigger rebuild from the UI or OSC I see it arriving on rep_server.log but I see this on the corresponding scheduler log:
-home:Admin/SLE_11_SP4 Repository ‘SUSE:SLE-11-SP4/standard’ is unavailable
And this repeats for all the repositories I have. The weird thing is, the repositories are populated, have a :full.solv file, and work when I ask for a local build via OSC.
However, I have five workers that refuse to work. How can I whip them into working? :P
Answering to myself, I found the problem: the projects to which the repositories were linked to had a “<access><disabled/></access>” on their Meta configuration :) After I removed these tags, the workers started building again. -- — Erico Mendonça Dedicated Support Engineer SUSE
N�����r��y隊Z)z{.���Wlz��qﮞ˛���m�)z{.��+�: �{Zr�az�'z��j)h���Ǜ�)]���Ǿ� ޮ�^�ˬz��
participants (3)
-
Adrian Schröter -
Erico Mendonca -
Karsten Keil