On Mon, Dec 2, 2013 at 1:52 PM, Michal Hrusecky <mhrusecky@suse.cz> wrote:
Hi,
we are having a nice torrent of ideas in this mailing list since coolo presented this proposal last Thursday: https://progress.opensuse.org/workflow/factory-proposal.html
Some of them are related to staging projects, so I will try to expand further one of the options regarding what could be happening inside that dashed box in the diagram.
The main goal of staging projects is preventing Factory from breaking so often by creating a "temporary and alternative Factory" where dangerous and conflicting submissions have to learn to live in peace and harmony before entering the real Factory. The original mail from coolo contains a great example with new automake.
So let's look closer into all those boxes, arrows and decision boxes:
1) Developer sends SR#1 to Factory and review team decides it needs staging, so staging project gets created (let's assume the decision is made by the review team, even if this is currently being discussed in the mailing list). The staging project initially contains only the package that is being sent and rebuilds everything from Factory that depend on it.
A couple questions/comments: === - how SR batches are handled? For instance, if I have even just 2 SRs that I know depend on each other but I want them staged because I don't know what else may depend on them. How do I tell OBS to create a staging tree and include both of my SRs. Obviously for KDE or Gnome, these might be very large SR batches. === - I like the idea of every SR (or small SR batch) going through a staging project. But I think we need 2 solutions: short term (hours) staging projects and long term (days/weeks) staging projects. -- long term staging projects - this has been the focus of discussion, so I will ignore it here. -- short term (hours) staging projects I propose every SR (or SR batch) not tagged for long term staging be evaluated for short term staging viability. The criteria could be "causes 5 or less package builds and all of the package builds take less than 1 hour each". Once a SR / SR batch is tagged for short term staging: --- create short term staging project - A BtrFS read/write snapshot of factory is created. This should only take a couple seconds --- Update the affected packages based on the SR / SR batch - I assume a few more seconds --- Run the build logic and build the updated packages and anything that depends on them - this is the time consumer, put hopefully this class of build can be given top priority. --- If no build errors, forward original SR / SR batch to factory reviewers and destroy BtrFS snapshot. --- if build errors, hoover up the build logs from all the builds and attach them to the SR, reject the SR, and destroy the BtrFS snapshot. If the above is totally automated, it seems the BtrFS snapshot may only live for a an hour or less typically. With that process it seems the overhead of creating a staging projects is minimal and even single package SRs of leaf packages could be run through a quick staging compile / dependency compile. === -- How many "leaf" SRs per day are there anyway? -- How many simultaneous BtrFS snapshots can exist and still get reasonable performance? --- If one assumes 10 simultaneous BtrFS snapshots is reasonable to maintain, then a queue could be established to push leaf package staging projects through. If the average leaf staging project lives for 1 hour, then that's 240 SR / SR batches per day that could be treated this way.. Greg -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org