[opensuse-factory] Converting products to the new system
Hi, Dimstar has started to convert the product builds to the new product-builder in Factory yesterday. Andreas Schwab did start also for ARM. I created the new definitions also for zSystems and PowerPC now. Please note the new "000product" package container. I like to remove the _product container ASAP what will help me to adapt all the stuff for factory publishing. Factory people are collecting all the needed tasks here: https://en.opensuse.org/openSUSE:Migrate_to_product_builder#Factory_process_... JFYI ;) adrian -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, 2017-07-12 at 08:13 +0200, Adrian Schröter wrote:
Hi,
Dimstar has started to convert the product builds to the new product-builder in Factory yesterday.
Andreas Schwab did start also for ARM.
I created the new definitions also for zSystems and PowerPC now.
Shouldn't the package from openSUSE:Factory trickle through to the ports? They are all project lniks against openSUSE:Factory. Seems odd that there is extra work involved in the ports (other than keeping the .group files updated, which is done by bots - update pending) As for removing _product from openSUSE:Factory - I think that's a good thing to do, BUT it will, as far as I've seen, likely crash the factory dashobard (is that finally under maintenance of the OBS team, see fate#320770) Cheers, Dominique
On Mittwoch, 12. Juli 2017, 08:19:02 CEST wrote Dominique Leuenberger / DimStar:
On Wed, 2017-07-12 at 08:13 +0200, Adrian Schröter wrote:
Hi,
Dimstar has started to convert the product builds to the new product-builder in Factory yesterday.
Andreas Schwab did start also for ARM.
I created the new definitions also for zSystems and PowerPC now.
Shouldn't the package from openSUSE:Factory trickle through to the ports?
Yes, and it does.
They are all project lniks against openSUSE:Factory. Seems odd that there is extra work involved in the ports (other than keeping the .group files updated, which is done by bots - update pending)
We do maintain the product configurations currently seperatly. You could of course also merge ARM/PPC/zSystems into your main package. We would not need the additional container then. However, that means you may need to change stuff in openSUSE:Factory to fix product builds in openSUSE:Factory:ARM.
As for removing _product from openSUSE:Factory - I think that's a good thing to do, BUT it will, as far as I've seen, likely crash the factory dashobard (is that finally under maintenance of the OBS team, see fate#320770)
I don't see that we committed to that, but we can take a look ;) -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, 2017-07-12 at 08:39 +0200, Adrian Schröter wrote:
As for removing _product from openSUSE:Factory - I think that's a good thing to do, BUT it will, as far as I've seen, likely crash the factory dashobard (is that finally under maintenance of the OBS team, see fate#320770)
I don't see that we committed to that, but we can take a look ;)
Just for the record: the tate entry was done on request of you so that this can formally be taken over - would be a good moment to look at it, indeed Cheers, Dominique
On Mittwoch, 12. Juli 2017, 08:42:30 CEST wrote Dominique Leuenberger / DimStar:
On Wed, 2017-07-12 at 08:39 +0200, Adrian Schröter wrote:
As for removing _product from openSUSE:Factory - I think that's a good thing to do, BUT it will, as far as I've seen, likely crash the factory dashobard (is that finally under maintenance of the OBS team, see fate#320770)
I don't see that we committed to that, but we can take a look ;)
Just for the record: the tate entry was done on request of you so that this can formally be taken over - would be a good moment to look at it, indeed
That is true, but it still doesn't say that we have committed to it;) I made a hot fix here for now: https://github.com/openSUSE/obs_factory/pull/90 But merging (and supporting) the code would take way more effort, since we need to find a concept how to allow to configure the setup. Currently this is all hardcoded in the code. Sorry, just not our #1 prio atm :/ -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, 2017-07-12 at 08:39 +0200, Adrian Schröter wrote:
They are all project lniks against openSUSE:Factory. Seems odd that there is extra work involved in the ports (other than keeping the .group files updated, which is done by bots - update pending)
We do maintain the product configurations currently seperatly. You could of course also merge ARM/PPC/zSystems into your main package. We would not need the additional container then.
However, that means you may need to change stuff in openSUSE:Factory to fix product builds in openSUSE:Factory:ARM.
In the long run, I think this is the best thing to do (and quite some stuff needed by ports IS in openSUSE:Factoru_product, like the arch includes and the like). This is needed in order to finally get around the branches 'brekaing' over and over (as they update files that also exist in oS:F, it does happen every now and then that we have merge conflicts, which in turn makes 'osc staging accept' fail (as it triggers a service run in the ports/_product - to be changed to 000product) (but let's make the 'even better' after the migration to the new builder - while this is WIP, we do not produce snapshots after all) Cheers, Dominique
On Jul 12 2017, Adrian Schröter
On Mittwoch, 12. Juli 2017, 08:19:02 CEST wrote Dominique Leuenberger / DimStar:
Shouldn't the package from openSUSE:Factory trickle through to the ports?
Yes, and it does.
Does it? I don't see how. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mittwoch, 12. Juli 2017, 10:16:58 CEST wrote Andreas Schwab:
On Jul 12 2017, Adrian Schröter
wrote: On Mittwoch, 12. Juli 2017, 08:19:02 CEST wrote Dominique Leuenberger / DimStar:
Shouldn't the package from openSUSE:Factory trickle through to the ports?
Yes, and it does.
Does it? I don't see how.
it does not anymore, since I re-created the 000product container in the ports projects. But it would get the one from openSUSE:Factory if you drop that one again from openSUSE:Factory:$Port -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Jul 12 2017, Adrian Schröter
On Mittwoch, 12. Juli 2017, 10:16:58 CEST wrote Andreas Schwab:
On Jul 12 2017, Adrian Schröter
wrote: On Mittwoch, 12. Juli 2017, 08:19:02 CEST wrote Dominique Leuenberger / DimStar:
Shouldn't the package from openSUSE:Factory trickle through to the ports?
Yes, and it does.
Does it? I don't see how.
it does not anymore, since I re-created the 000product container in the ports projects.
That the point. If you would have branched it it would get updated. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hi,
2017-07-12 11:46 GMT+02:00 Andreas Schwab
it does not anymore, since I re-created the 000product container in the ports projects. That the point. If you would have branched it it would get updated.
is there a problem with just rebranching it and repairing the messup? I agree that it probably makes a lot more sense to have them branched (ideally with a next-to-zero diff). Greetings, Dirk -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mittwoch, 12. Juli 2017, 13:29:06 CEST wrote Dirk Müller:
Hi,
2017-07-12 11:46 GMT+02:00 Andreas Schwab
: it does not anymore, since I re-created the 000product container in the ports projects. That the point. If you would have branched it it would get updated.
is there a problem with just rebranching it and repairing the messup? I agree that it probably makes a lot more sense to have them branched (ideally with a next-to-zero diff).
no, as written before, it is a good idea, _if_ you plan to merge them. So far we had complete seperated setups and I did not want to create new conflicts therefore. Just getting the status quo using the new product files. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, 2017-07-12 at 14:01 +0200, Adrian Schröter wrote:
On Mittwoch, 12. Juli 2017, 13:29:06 CEST wrote Dirk Müller:
Hi,
2017-07-12 11:46 GMT+02:00 Andreas Schwab
: it does not anymore, since I re-created the 000product container in the ports projects.
That the point. If you would have branched it it would get updated.
is there a problem with just rebranching it and repairing the messup? I agree that it probably makes a lot more sense to have them branched (ideally with a next-to-zero diff).
no, as written before, it is a good idea, _if_ you plan to merge them.
So far we had complete seperated setups and I did not want to create new conflicts therefore. Just getting the status quo using the new product files.
_product in :Ports were branches with small diffs - and osc staging accept does a trigger servicerun in all ports' _product to make sure the oS:F changes fly through Cheers, Dominique
On Jul 12 2017, Adrian Schröter
So far we had complete seperated setups
So far the _product packages were branched. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mittwoch, 12. Juli 2017, 11:46:58 CEST wrote Andreas Schwab:
On Jul 12 2017, Adrian Schröter
wrote: On Mittwoch, 12. Juli 2017, 10:16:58 CEST wrote Andreas Schwab:
On Jul 12 2017, Adrian Schröter
wrote: On Mittwoch, 12. Juli 2017, 08:19:02 CEST wrote Dominique Leuenberger / DimStar:
Shouldn't the package from openSUSE:Factory trickle through to the ports?
Yes, and it does.
Does it? I don't see how.
it does not anymore, since I re-created the 000product container in the ports projects.
That the point. If you would have branched it it would get updated.
yes, but this doesn't make sense in the way we maintain it atm. It would just regulary break, because of changes in openSUSE:Factory. This makes only sense when we plan to submit the changes back to openSUSE:Factory. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hi Adrian,
2017-07-12 13:29 GMT+02:00 Adrian Schröter
It would just regulary break, because of changes in openSUSE:Factory.
This makes only sense when we plan to submit the changes back to openSUSE:Factory.
Not sure I understand that. 90%+ of the changes we do are in the DVD5-aarch64.group file, which is by definition conflict free to the base. I don't see a _servicedata file that would e.g. constantly conflict. I would actually welcome the change that the tumblweed team would maintain the DVD5-aarch64.group file in the base product, so far that wasn't wanted but since this is mostly mechanical I think it might help us in reducing the workload. Greetings, Dirk -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, 2017-07-12 at 13:34 +0200, Dirk Müller wrote:
Not sure I understand that. 90%+ of the changes we do are in the DVD5-aarch64.group file, which is by definition conflict free to the base. I don't see a _servicedata file that would e.g. constantly conflict.
I would actually welcome the change that the tumblweed team would maintain the DVD5-aarch64.group file in the base product, so far that wasn't wanted but since this is mostly mechanical I think it might help us in reducing the workload.
That's not entirely accurate - the major problem we have is that the ports have decoupled build times and anytime we update the .group file, we risk a rebuild in openSUSE:Factory, potentially delaying a release of x86/x64 even further, even though it 'should' not impact those archs (but that would be source changes, service rerun, and generally rebuild) Imho, the .group files are 'ok' to be placed directly in :ARM/:PowerPC/:FOO - as you already state, they do not conflict (they do not exist in openSUSE:Factory) what we SHOULD reduce though is the need to modify the .product files to get the right things done in the various projects - THAT is where the conflicts happen every now and then. Cheers Dominique
On Mittwoch, 12. Juli 2017, 13:34:46 CEST wrote Dirk Müller:
Hi Adrian,
2017-07-12 13:29 GMT+02:00 Adrian Schröter
: It would just regulary break, because of changes in openSUSE:Factory.
This makes only sense when we plan to submit the changes back to openSUSE:Factory.
Not sure I understand that. 90%+ of the changes we do are in the DVD5-aarch64.group file, which is by definition conflict free to the base. I don't see a _servicedata file that would e.g. constantly conflict.
no, the repo, media and architecture definitions are more likely to conflict. However, it is up to you to try it. Just keep in mind that the source service will not run on underlying changes and you need still manually to merge them.
I would actually welcome the change that the tumblweed team would maintain the DVD5-aarch64.group file in the base product, so far that wasn't wanted but since this is mostly mechanical I think it might help us in reducing the workload.
IMHO the way better approch .... In this case you might indeed want to branch from the 000product from openSUSE:Factory and submit request your changes back. It seems Dominique is also fine with this. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, 2017-07-12 at 14:00 +0200, Adrian Schröter wrote:
On Mittwoch, 12. Juli 2017, 13:34:46 CEST wrote Dirk Müller:
Hi Adrian,
2017-07-12 13:29 GMT+02:00 Adrian Schröter
: It would just regulary break, because of changes in openSUSE:Factory.
This makes only sense when we plan to submit the changes back to openSUSE:Factory.
Not sure I understand that. 90%+ of the changes we do are in the DVD5-aarch64.group file, which is by definition conflict free to the base. I don't see a _servicedata file that would e.g. constantly conflict.
no, the repo, media and architecture definitions are more likely to conflict.
However, it is up to you to try it. Just keep in mind that the source service will not run on underlying changes and you need still manually to merge them.
I would actually welcome the change that the tumblweed team would maintain the DVD5-aarch64.group file in the base product, so far that wasn't wanted but since this is mostly mechanical I think it might help us in reducing the workload.
IMHO the way better approch .... In this case you might indeed want to branch from the 000product from openSUSE:Factory and submit request your changes back.
It seems Dominique is also fine with this.
The tricky part will be to not break inside oS:F.. let's for example look at the diff of the old _product container: osc rdiff openSUSE:Factory _product openSUSE:Factory:ARM - <url name="repository">http://download.opensuse.org/tumble weed/repo/oss/</url>; + <url name="repository">http://download.opensuse.org/ports/ %{_target_cpu}/tumbleweed/repo/oss/</url>; => we need a way to unify this / conditionalize this in the .product file (the format does not match for port and non-port archs) the archset changes should be safe to be moved to oS:F already the various package name = FOO should be fine - not sure why they are tagged arch / onlyarch - they are obviously the same on other archs!?! the use group=FOO are 'trickier' to have in oS:F/_product / 000product, as adding the .group file to oS:F would mean various bots updating .group files for the respective ports and triggering rebuilds in oS:F - so those are not really a good thing to have (the include will fail, as we don't have the .group file present) THOSE are the things in need of a solution before we can move 'all' into the main product definition - but then, the .product does not change that often and we can survive with this part (and the .group files) being added in the :Ports but nothing else should be in the diff. Cheers, Dominique
On Jul 12 2017, Adrian Schröter
no, the repo, media and architecture definitions are more likely to conflict.
Such conflicts happen only very rarely. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
How can we can get rid of the broken build result "no architectures for packages"? Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Jul 12 2017, Adrian Schröter
I created the new definitions also for zSystems and PowerPC now.
Any reason they aren't branched off the Factory package? Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, 2017-07-12 at 08:13 +0200, Adrian Schröter wrote:
Hi,
Dimstar has started to convert the product builds to the new product-builder in Factory yesterday.
Andreas Schwab did start also for ARM.
I created the new definitions also for zSystems and PowerPC now.
Please note the new "000product" package container. I like to remove the _product container ASAP what will help me to adapt all the stuff for factory publishing.
Factory people are collecting all the needed tasks here:
https://en.opensuse.org/openSUSE:Migrate_to_product_builder#Factory_ process_tooling_to_validate
- As a followup - to keep our community informed on progress - For now this migration has been put on hold for openSUSE:Factory (aka Tumbleweed for Intel x86/x86_64), as we encountered some issues that would impact users directly. Technically, the Tumbleweed repo type will change from 'susetags' (aka repo type=yast2) to a standard rpm-md repo. Zypp can hanlde both equally well and going to standardized way is generally favored Sadly, at this moment, libzypp does not support a 'repo type change' happening, unless the users remove the "type=" line from the .repo files (see https://bugzilla.suse.com/show_bug.cgi?id=1048315) This is, obviously, nothing we want to ask of all our users and thus we decided to halt the migration for now, update libzypp to cope with this transparently and then restart this migration. We undrestand that users that do not update their Tumbleweed systems regularly (enough) will still run into this kind of issue, but at one point we do have to move forward. The current time plan we have is to provide a libzypp update and then roughly 4 weeks later redo the migration. Most active TW users would hopefully get the updated zypp stack in this period. We will keep you posted about any new attempt of this migration. Cheers, Dominique
participants (4)
-
Adrian Schröter
-
Andreas Schwab
-
Dirk Müller
-
Dominique Leuenberger / DimStar