[opensuse-arm] aarch64 in openQA
Hi, As you may have noticed openqa.opensuse.org has some aarch64 jobs. It was quite some effort to get there but it's far from done yet. 1. Build Power At the moment there's one openQA worker running on a PPC64 host (power8). That host is fast enough to run the emulation reasonably well. To do serious tests we need real hardware with kvm though. Not just for openQA but also for OBS. The image in openQA is just the NET iso that installs from a static ftp repo from two weeks ago. The ftp tree wasn't published since then as it never finished building. Factory is moving too fast, aarch64 can't catch up. Meanwhile maybe an installation DVD could help. The limited set of packages on the DVD probably has a chance to finish in time. 2. Project Setup To get a real Tumbleweed for aarch64 the OBS setup needs to be changed: - openSUSE:Factory:ARM/_product need to become a link to openSUSE:Factory/_product. That way arm inherits the Tumbleweed version number. - openSUSE:Factory:ARM:ToTest needs to be created to hold the snapshots that go into openQA 3. Scripts The magic factory scripts (totest manager, pattern generator) need to be adjusted. Dinar did most of the heavy lifting for ppc so the scripts are now able to handle another architecture. The spots needed for aarch64 should be easy to find by searching for ppc. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.de/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5; 90409 Nürnberg; Germany -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Hi Ludwig, Thanks a lot for your efforts on openQA for aarch64!
1. Build Power
At the moment there's one openQA worker running on a PPC64 host (power8). That host is fast enough to run the emulation reasonably well. To do serious tests we need real hardware with kvm though.
We have some hardware available.. does it need to be in the external network?
just for openQA but also for OBS. The image in openQA is just the NET iso that installs from a static ftp repo from two weeks ago. The ftp tree wasn't published since then as it never finished building. Factory is moving too fast, aarch64 can't catch up.
We're building with native hardware since ~ 12 hours on Factory after some hickups (and still sorting out the fallout of that). So that problem should be solved hopefully. The tree is about to be ready to be published (if only the OBS scheduler would actually notice it). Maybe its the "watching a pot with water never boilds" phenomenon but whenever I check it, there are 10-20 min pauses where the OBS scheduler just does nothing at all, including not noticing that it could publish the tree.
Meanwhile maybe an installation DVD could help. The limited set of packages on the DVD probably has a chance to finish in time.
We can do that as well, unfortunately the DVD builds require the pattern/package list magic which I still don't know about. its certainly not ideal to maintain those package lists (that are slightly different than x86_64) ourselves manually. Any idea where to start on that?
2. Project Setup
To get a real Tumbleweed for aarch64 the OBS setup needs to be changed:
- openSUSE:Factory:ARM/_product need to become a link to openSUSE:Factory/_product. That way arm inherits the Tumbleweed version number.
you mean the date based version number like 20150331 ? is that all? when does it need to be bumped? is there a script involved that bumps it once openqa passes? It used to be a link but we had tons of merge conflicts all the time, which is why we removed the link at some point. it would be nicer to maintain the ARM related things in openSUSE:Factory but SR's don't get accepted and bribing by email to get one liner fixes committed in less than a week of roundtrip is also not always the best thing to spend time on.
- openSUSE:Factory:ARM:ToTest needs to be created to hold the snapshots that go into openQA
created. who needs to have write privileges?
3. Scripts
The magic factory scripts (totest manager, pattern generator) need to be adjusted. Dinar did most of the heavy lifting for ppc so the scripts are now able to handle another architecture. The spots needed for aarch64 should be easy to find by searching for ppc.
git url? TIA, Dirk -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Dirk Müller schrieb:
1. Build Power
At the moment there's one openQA worker running on a PPC64 host (power8). That host is fast enough to run the emulation reasonably well. To do serious tests we need real hardware with kvm though.
We have some hardware available.. does it need to be in the external network?
Jain. I agreed with Lars to create a separate openQA worker VLAN. This new network will not be reachable from the internet. The setup will allow to easily switch workers between OBS and openQA without physical rewiring.
Meanwhile maybe an installation DVD could help. The limited set of packages on the DVD probably has a chance to finish in time.
We can do that as well, unfortunately the DVD builds require the pattern/package list magic which I still don't know about. its certainly not ideal to maintain those package lists (that are slightly different than x86_64) ourselves manually. Any idea where to start on that?
See below.
2. Project Setup
To get a real Tumbleweed for aarch64 the OBS setup needs to be changed:
- openSUSE:Factory:ARM/_product need to become a link to openSUSE:Factory/_product. That way arm inherits the Tumbleweed version number.
you mean the date based version number like 20150331 ? is that all? when does it need to be bumped? is there a script involved that bumps it once openqa passes?
There are two scripts involved. The staging plugin and the totest manager.
It used to be a link but we had tons of merge conflicts all the time, which is why we removed the link at some point.
Looking at PowerPC I guess some of those conflicts could be avoided by splitting more things out of openSUSE.product and include architecture specific files instead.
it would be nicer to maintain the ARM related things in openSUSE:Factory but SR's don't get accepted and bribing by email to get one liner fixes committed in less than a week of roundtrip is also not always the best thing to spend time on.
One of the reasons why it took so long to get a testable aarch64 into openQA... The separate project is fine as long as it only contains the products most of the time. PowerPC is doing the same after all.
- openSUSE:Factory:ARM:ToTest needs to be created to hold the snapshots that go into openQA
created. who needs to have write privileges?
The user who runs the totest manager.
3. Scripts
The magic factory scripts (totest manager, pattern generator) need to be adjusted. Dinar did most of the heavy lifting for ppc so the scripts are now able to handle another architecture. The spots needed for aarch64 should be easy to find by searching for ppc.
git url?
totest manager: https://github.com/openSUSE/osc-plugin-factory pattern generator: https://github.com/openSUSE/package-lists cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.de/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5; 90409 Nürnberg; Germany -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Dirk Müller <dirk@dmllr.de> writes:
We're building with native hardware since ~ 12 hours on Factory after some hickups (and still sorting out the fallout of that). So that problem should be solved hopefully. The tree is about to be ready to be published (if only the OBS scheduler would actually notice it).
Does it actually publish if the repository isn't ready yet? We still have the emulated armv6l which blocks the images/local repo from finishing. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On Donnerstag, 2. April 2015, 09:19:44 wrote Andreas Schwab:
Dirk Müller <dirk@dmllr.de> writes:
We're building with native hardware since ~ 12 hours on Factory after some hickups (and still sorting out the fallout of that). So that problem should be solved hopefully. The tree is about to be ready to be published (if only the OBS scheduler would actually notice it).
Does it actually publish if the repository isn't ready yet? We still have the emulated armv6l which blocks the images/local repo from finishing.
kiwi type repos get always published immediatly after each build job. So one image can not block the others. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Dirk Müller schrieb:
just for openQA but also for OBS. The image in openQA is just the NET iso that installs from a static ftp repo from two weeks ago. The ftp tree wasn't published since then as it never finished building. Factory is moving too fast, aarch64 can't catch up.
We're building with native hardware since ~ 12 hours on Factory after some hickups (and still sorting out the fallout of that). So that problem should be solved hopefully. The tree is about to be ready to be published (if only the OBS scheduler would actually notice it). Maybe its the "watching a pot with water never boilds" phenomenon but whenever I check it, there are 10-20 min pauses where the OBS scheduler just does nothing at all, including not noticing that it could publish the tree.
According to coolo the publisher could benefit from some performance improvements. Looks like it's rather inefficient when faced with large trees. He suggested that limiting the packages in the ftp tree instead of creating an actual dvd tree could help too. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.de/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5; 90409 Nürnberg; Germany -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
participants (4)
-
Adrian Schröter -
Andreas Schwab -
Dirk Müller -
Ludwig Nussel