Re: [opensuse-buildservice] setting up an Ubuntu/Debian distribution project on a local OBS installation

On Mon, May 7, 2012 at 6:11 PM, wrote:

Replies inline.

...

On Fri, May 4, 2012 at 8:43 PM, wrote:

...

How are the distribution projects for Ubuntu (like Ubuntu:10.04) set up on build.opensuse.org? Of course, I see the project and raw configs, but the rest is unclear to me.

This is the 64 thousand dollar question. :-) While we've done this internally at our company, there is not a lot of documentation on how to do this in OBS. This is one of the great drawbacks of OBS -- it has very little documentation. Secondly, OBS comes from OpenSUSE (mostly) and this means a subtle bias towards rpm and a lack of familiarity with debs. These two things make it difficult to do easy deb based distro set ups.

If you've done this internally at your company, I'd love to know what steps were taken to do this if you can and wouldn't mind.

Sure, we try to contribute to upstream, but most of what we've done is merely add additional open source tools around the OBS, so its not really a contribution directly to OBS. None of the changes are proprietary, its just a question of priority and documentation often is low on the prio list. :-/

It's true that the deb support is somewhat non-native. The fact that it's supported at all is commendable.

Yes, I don't mean to disparage the OBS. It is good that deb repos are supported and it is clearly a priority for the OBS team. This is where one should be grateful for the support: thanks OBS team!

...

That said, there has been some good work done in the latest OBS release (2.3.0) that addresses some of the shortcomings on the deb side. It is possible to do things like sign the Packages file for example.

This is good to know. Thank you for pointing that out.

...

...

I'm trying to set up an Ubuntu distribution project in a local OBS install to have locally for quicker access and avoid the intermittent but frequent proxy/latency issues we see when just pointing to distribution projects on build.opensuse.org. We can't use download on demand in this case, as we'd like to also have the ability to utilize updated packages from the -updates and -security Ubuntu repositories (e.g. precise-updates and precise-security), and the download on demand feature doesn't like it when repositories change at all.

To do this, the instructions at http://gitorious.org/opensuse/build- service/blobs/master/dist/README.SETUP recommend (in section 4.2) copying the distribution project over from build.opensuse.org using obs_mirror_project and /srv/www/obs/api/script/import. However, I have an internal Ubuntu (non-OBS) mirror, so it really makes no sense to pull all these packages from outside.

It's also unclear to me how to keep a deb-based distribution project up-to- date after the initial import.

What do you mean by "up-to-date"? Do you mean how do you add new packages that have been patched and have new versions? Are you keeping the repo up-to-date or trying to sync the underlying OS with upstream? If you're doing the latter, you'll like want to look at some tools from the Debian world where this problem has been largely solved. Take a look at reprepro for example which has an active maintainer (who is also the author) as well as other tools.

I just meant, "Can I just sync the packages in the distribution project using a standard apt mirror sync tool or do I have to do something special because it's an OBS distribution project?"

Using an apt mirror tool should be trivial to integrate. The deb repos from OBS are becoming much more like the upstream Debian repo layouts so it should be just a matter of rsync and not much more. Though apt-ftparchiver or similar might give you more bang for your buck.

...

...

I've looked through the documentation (and done lots of web searches) for how to do this. The best I can find is to either use download on demand, just point to build.opensuse.org or just copy the distribution project from build.opensuse.org. As I said, though, none of those are good options for me. I've spent time looking through the OBS source, and it looks like bs_admin is what parses repositories for sourcing, but it doesn't necessarily seem to indicate how OBS wants me to set up the actual Ubuntu distribution project. I see that bs_srcserver seems to also do some of the work. I am not terribly familiar with the OBS source, so it's likely that I've missed something. I may also be incorrect in the assumption that OBS is parsing the Packages.gz file for a repository or expecting an ordinary apt repository.

Those are good questions -- I think one has to go code diving to determine the answers, I've never seen any documentation on how OBS parses Debian-based repos.

I was afraid that this would be the answer. I was hoping that the person who set up the Ubuntu and Debian distribution projects on build.opensuse.org might chime in with how to set up one. I suppose I don't really need to know how OBS parses Debian-based repos so long as I can get OBS to use one.

Yeah, I don't think you need to know the details since OBS is trying to do that for you transparently. I think the goal is to have a public OBS that exports debs into a repo in such a fashion that it is indistinguishable from an ordinary deb repo so you won't need to sync at all, just build and publish. (Those in the know please correct me if I'm wrong.) Regards, Jeremiah -- ============================================= Jeremiah C. Foster エレミア フォスター Open Source Technologist Pelagicore AB Ekelundsgatan 4, 6tr, SE-411 18 Gothenburg, Sweden Mobile: +46 (0)730 93 0506 E-Mail: jeremiah.foster@pelagicore.com ============================================= === NOTE === The information contained in this E-mail message is intended only for use of the individual or entity named above. If the reader of this message is not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. =============