Hi Adrian, Arnaud,
I'd like to follow up on this thread about OBS aptly integration.
Arnaud Ferraris wrote:
Hi Adrian, Thanks for your comments. Le 28/05/2021 à 16:26, Adrian Schröter a écrit :
On Donnerstag, 27. Mai 2021, 12:31:27 CEST Arnaud Ferraris wrote: Hi, We currently use OBS for building and publishing APT packages for the Apertis project[1] (a Debian derivative distribution). At the moment we use downstream patches for the publisher (src/backend/bs_publish) in order to use reprepro instead of dpkg-scanpackages/dpkg-scansources. As we're considering implementing a new APT publisher based on aptly[2], we'd like to do so in a way that can be upstreamed to OBS so it can benefit the whole community. This obviously raises several questions as to how we should proceed, so I'm sending this email as a call for advice/comments/suggestions. great .... even when I have no clue what it is :) We're considering the following:
this new publisher would be implemented in addition to the existing
dpkg-scan* based one and be selectable through global configuration options. fine
using aptly opens up possibilities, but it isn't always as
straightforward as a few system() call; we would therefore need to either ship a helper tool or add a new BSPublisher submodule (the latter is the preferred option atm) that is all fine, but keep in mind that the code (helper tool or submodule) needs to get hardened against malicious binaries. So, a review of security team is most likely needed. Understood, I'll keep this in mind.
I'm currently working on a downstream open-build-service project adding support for the new aptly publisher. For reference, work-in-progress branch can be found here: https://gitlab.collabora.com/adalessandro/open-build-service-rebase/-/tree/w...
Note that this e-mail is a brief introduction to how the integration is structured. More details and documentation is available in the wip branch and will be posted once this thread follows up. I could submit an RFC patchset to the mailing list for discussion, or submit a merge request to the github repository. @Adrian, what would be the better way for upstreaming?
This aptly integration supports the following features:
* Aptly backend is used in a synchronous way, calling local aptly command line client.
* Repositories to be published by aptly can be defined in the OBS configuration files. This configuration is a hash following how aptly structures distribution publishing for multi-component repositories. E.g.:
our $aptly_config = { "prefix_path" => { "distribution_name" => { "gpg-key" => "optional_gpg_hash", "components" => { "component_name" => { "project" => "obs_project_id", "repository" => "obs_repository_id", }, [...] }, }, [...] }, };
* From the above, note that several OBS project/repositories can be grouped in a single multi-component distribution.
* Adding/removing an OBS project will create and publish the related aptly repositories. Only those repositories with publish flag enabled will be created and published. Repositories architectures are automatically translated from OBS to aptly and set accordingly.
* Publishing an OBS package will add the related binaries to its aptly repository. Previous versions of the same package are automatically removed.
* As aptly support snapshots, a reference OBS publish_hook is provided to take snapshots each time a package is published.
* As aptly requires the database to get cleaned up periodically, a reference script is provided to be scheduled and called externally.
Again, this e-mail is an RFC on its own :-) in order to kick-off the discussion to a first aptly publisher integration.
Please, don't hesitate to ask and let me know how would be the best way to proceed on this topic.
Regards, Ariel D'Alessandro www.collabora.com