[opensuse-buildservice] Best option to integrate StartlingX CI with OBS
Hello, I have spent a few hours at looking at various options on how best integrate StartlingX CI with OBS for our multiOS subproject. For info, StarlingX is a pilot project of OpenStack. https://www.starlingx.io/ I notice when looking at information on OBS and CI integration published by projects like OpenStack or obs-server that that prefer process seems to be, first build in a branch, followed by a second build in staging. It leads me to a few questions: 1) Why 2 builds ------------------------ What s the value of building of a package in Staging if that was already validated in a Branch ? Is there something that I missed that is possible in Staging but not in a branch ? 2) Managing builds ------------------------------ I do not have the knowledge of a build ID concept on OBS and that make a few things more difficult. I would be interested to understand the best way to work around. After creating my branch and changing the required files, my build can be tracked by osc. osc results -w --xml home:dominig:branches:....... The -w option might not be a good idea as I can see that SSL Error: (104, 'Connection reset by peer') are quite frequent. Is there a better asynchronous way to track the package build success/failure? If multiple changes are triggered in parallel on the same package, I would have a mess with my branches (naming conflict). What would be the best method to avoid serialisation on the test builds in that case ? 3) Cancelling build ------------------------------ It is not uncommon in a CI, that a new patch set is added before that the build result is known. What would be the best way to manage such situation ? Would an "osc restartbuild" on the same branch after source update the right solution ? Any other hints for CI integration would be welcome. -- Dominig ar Foll Senior Software Architect Intel Open Source Technology Centre -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Donnerstag, 12. September 2019, 16:51:34 CEST Dominig ar Foll (Intel Open Source) wrote:
Hello,
I have spent a few hours at looking at various options on how best integrate StartlingX CI with OBS for our multiOS subproject. For info, StarlingX is a pilot project of OpenStack. https://www.starlingx.io/
I notice when looking at information on OBS and CI integration published by projects like OpenStack or obs-server that that prefer process seems to be, first build in a branch, followed by a second build in staging.
It leads me to a few questions:
1) Why 2 builds ------------------------ What s the value of building of a package in Staging if that was already validated in a Branch ? Is there something that I missed that is possible in Staging but not in a branch ?
You could build in project A and use the release mechanism to move sources and binaries to project B. Would that solve your issue?
2) Managing builds ------------------------------ I do not have the knowledge of a build ID concept on OBS and that make a few things more difficult. I would be interested to understand the best way to work around. After creating my branch and changing the required files, my build can be tracked by osc. osc results -w --xml home:dominig:branches:....... The -w option might not be a good idea as I can see that SSL Error: (104, 'Connection reset by peer') are quite frequent. Is there a better asynchronous way to track the package build success/failure?
hm, osc should maybe handle such an errror better, no idea where it comes from atm ...
If multiple changes are triggered in parallel on the same package, I would have a mess with my branches (naming conflict). What would be the best method to avoid serialisation on the test builds in that case ?
you speak about to build each commit independend in own projects? You could hand over a project name which includes the pull request number?
3) Cancelling build ------------------------------ It is not uncommon in a CI, that a new patch set is added before that the build result is known. What would be the best way to manage such situation ? Would an "osc restartbuild" on the same branch after source update the right solution ?
Hm, you could set it up in way that OBS gets notifications on source changes and is doing so automatically...
Any other hints for CI integration would be welcome.
some old blog about that: https://openbuildservice.org/2013/11/22/source-update-via_token/ https://openbuildservice.org/help/manuals/obs-reference-guide/ cha.obs.authorization.token.html#id-1.7.18.4 https://openbuildservice.org/help/manuals/obs-user-guide/ cha.obs.source_service.html -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 13/09/2019 08:00, Adrian Schröter wrote:
On Donnerstag, 12. September 2019, 16:51:34 CEST Dominig ar Foll (Intel Open Source) wrote:
1) Why 2 builds ------------------------ What s the value of building of a package in Staging if that was already validated in a Branch ? Is there something that I missed that is possible in Staging but not in a branch ? You could build in project A and use the release mechanism to move sources and binaries to project B. Would that solve your issue? An easy way to get all the projects config(s) correctly copied, is to create a branch. Unfortunately I did not find a way to create a branch which would only build on an external trigger that would be activated only after that the _service file to be tested has been installed.
Alternatively, what is the best solution to create a test package which would inherit all the config(s) of a given project ? Would copying meta prj and meta prjconf cover all the requrement to build as in a branch ? -- Dominig ar Foll Senior Software Architect Intel Open Source Technology Centre -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hey Dominig, On 12.09.19 16:51, Dominig ar Foll (Intel Open Source) wrote:
I notice when looking at information on OBS and CI integration published by projects like OpenStack or obs-server that that prefer process seems to be, first build in a branch, followed by a second build in staging.
Lots of people do lots of different things. Maybe you should explain to us what you are trying to achieve? What is it you want to do? Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 13/09/2019 15:24, Henne Vogelsang wrote:
Lots of people do lots of different things. Maybe you should explain to us what you are trying to achieve? What is it you want to do? What we want to do if fairly simple on the paper.
Today, each that a Commit is push to StartlinX Gerrit, we do an automatic local compilation which add a -1 or +1 automatically to the review. Tomorrow we would would like to follow that check, positive, by a real build via the OBS and automate a -1/+1 vote depending of the result. Some packages description (spec) already integrate a test phase and that number should increase in the future, increasing the value of the best build. Most of the time submissions affect only one sub project (a package for OBS). it does also happens that a grouped submission is created affected multiple project at the same time. We could delay the support of that type of build in a first phase if too complex to implement. -- Dominig ar Foll Senior Software Architect Intel Open Source Technology Centre -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hey, On 16.09.19 10:43, Dominig ar Foll (Intel Open Source) wrote:
On 13/09/2019 15:24, Henne Vogelsang wrote:
Lots of people do lots of different things. Maybe you should explain to us what you are trying to achieve? What is it you want to do?
What we want to do if fairly simple on the paper.
Today, each that a Commit is push to StartlinX Gerrit, we do an automatic local compilation which add a -1 or +1 automatically to the review.
Tomorrow we would would like to follow that check, positive, by a real build via the OBS and automate a -1/+1 vote depending of the result.
Okay so the package build on OBS is one of the checks inside your CI cycle right? There are people that do this with a local build (osc build) on their CI host and report back the status of that. There are also people who do the package build on OBS instead (create a test project/package with the new commit) and report back the status of the package build on OBS.
Most of the time submissions affect only one sub project (a package for OBS). it does also happens that a grouped submission is created affected multiple project at the same time. We could delay the support of that type of build in a first phase if too complex to implement.
With a local build this is a bit more complicated as you have to make sure how the two changes/packages influence each other. If you do your test build on the OBS it's a matter of creating both and then report back the state of the project, not the package.
1) Why 2 builds
Staging is just a way people handle their projects in OBS. Has little to do with the CI. It's just about accumulating changes before you push them out the your users.
2) Managing builds
You can listen to events on our event bus instead of querying the API repeatedly. https://rabbit.opensuse.org/
3) Cancelling build Sure restarting the build with the new changes sounds like the way to go.
HTH Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Thanks, for all these info. The last one on the vent Chanel was something that I had not yet seen anywhere. Dominig On 16/09/2019 11:30, Henne Vogelsang wrote:
Hey,
On 16.09.19 10:43, Dominig ar Foll (Intel Open Source) wrote:
On 13/09/2019 15:24, Henne Vogelsang wrote:
Lots of people do lots of different things. Maybe you should explain to us what you are trying to achieve? What is it you want to do?
What we want to do if fairly simple on the paper.
Today, each that a Commit is push to StartlinX Gerrit, we do an automatic local compilation which add a -1 or +1 automatically to the review.
Tomorrow we would would like to follow that check, positive, by a real build via the OBS and automate a -1/+1 vote depending of the result.
Okay so the package build on OBS is one of the checks inside your CI cycle right?
There are people that do this with a local build (osc build) on their CI host and report back the status of that. There are also people who do the package build on OBS instead (create a test project/package with the new commit) and report back the status of the package build on OBS.
Most of the time submissions affect only one sub project (a package for OBS). it does also happens that a grouped submission is created affected multiple project at the same time. We could delay the support of that type of build in a first phase if too complex to implement.
With a local build this is a bit more complicated as you have to make sure how the two changes/packages influence each other. If you do your test build on the OBS it's a matter of creating both and then report back the state of the project, not the package.
1) Why 2 builds
Staging is just a way people handle their projects in OBS. Has little to do with the CI. It's just about accumulating changes before you push them out the your users.
2) Managing builds
You can listen to events on our event bus instead of querying the API repeatedly. https://rabbit.opensuse.org/
3) Cancelling build Sure restarting the build with the new changes sounds like the way to go.
HTH
Henne
-- Dominig ar Foll Senior Software Architect Intel Open Source Technology Centre -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (3)
-
Adrian Schröter -
Dominig ar Foll (Intel Open Source) -
Henne Vogelsang