[yast-devel] Yast and CI
Hi developers, we agreed that current CI in jenkins have some problems and I would like to face it. But at first I would like to hear your opinions on current setup, what is weak points and what is nice features we like. Few questions I have in my mind: testing: current it test only changes in master branch. How important for you if we automatic test also pull requests and maintenance branches? Related question is how important for us is to have generated <package>.spec file as it quite block more generic solutions. autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV? involvement: Do you want actively help with CI? ( current status is that usually I fix it myself when I see fail ) Do you have own idea what can be automatic or what can help with removing common task? openess: Problem of current status is that yast CI lives in suse network ( benefit is that we have direct access to our test machines and can easy modify it ). What about using http://ci.opensuse.org/ instance? where then must live workers? I see that cloud guys use it, so maybe we can discuss it with them (mvidner do you have closer information?)? Thanks for feedback, Josef -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 09/11/2013 02:58 PM, Josef Reidinger wrote:
Hi developers, we agreed that current CI in jenkins have some problems and I would like to face it. But at first I would like to hear your opinions on current setup, what is weak points and what is nice features we like.
It's too fragile and due to its implementation it gets obsolete pretty quickly as the list of installed packages and their versions in the check/build environment are static. The current solution also doesn't care about BuildRequires.
Few questions I have in my mind:
testing: current it test only changes in master branch. How important for you if we automatic test also pull requests and maintenance branches? Related question is how important for us is to have generated <package>.spec file as it quite block more generic solutions.
All maintained branches would make sense. That's the only way how we can make sure nothing breaks by our change somewhere. Additionally, once a week (for instance), all the checks could be rerun even if no Yast packages were changed - it's because of the possible environment changes.
autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV?
Yast:Head is nice, Factory was a bit problematic as Coolo was against that - there were too many changes in Yast and Factory was rebuilding over and over. On the other hand - how to to submit to factory then? Who and when?
involvement: Do you want actively help with CI? ( current status is that usually I fix it myself when I see fail ) Do you have own idea what can be automatic or what can help with removing common task?
If scripts etc. were somewhere at GitHub, it would be easier. If I could checkout these scripts and run them locally, I can cooperate on the solution (in fact, I sometimes do anyway).
openess: Problem of current status is that yast CI lives in suse network ( benefit is that we have direct access to our test machines and can easy modify it ). What about using http://ci.opensuse.org/ instance? where then must live workers? I see that cloud guys use it, so maybe we can discuss it with them (mvidner do you have closer information?)?
+1 from me Lukas -- Lukas Ocilka, Cloud & Systems Management Department SUSE LINUX s.r.o., Praha -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Wed, Sep 11, Lukas Ocilka wrote:
autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV?
Yast:Head is nice, Factory was a bit problematic as Coolo was against that - there were too many changes in Yast and Factory was rebuilding over and over. On the other hand - how to to submit to factory then? Who and when?
There was a suggestion a while ago to only do automatic Factory checkins when package version number changed (at least I increase version number only once in a while or when really user visible faixes are in, not for every small commit). Apparently this was not implemented. Otherwise there would not be too many changes in Factory from YaST packages. Tschuess, Thomas Fehr -- Thomas Fehr, SuSE Linux Products GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Tel: +49-911-74053-0, Fax: +49-911-74053-482, Email: fehr@suse.de GPG public key available. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Wed, 11 Sep 2013 15:28:05 +0200 Thomas Fehr <fehr@suse.de> wrote:
On Wed, Sep 11, Lukas Ocilka wrote:
autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV?
Yast:Head is nice, Factory was a bit problematic as Coolo was against that - there were too many changes in Yast and Factory was rebuilding over and over. On the other hand - how to to submit to factory then? Who and when?
There was a suggestion a while ago to only do automatic Factory checkins when package version number changed (at least I increase version number only once in a while or when really user visible faixes are in, not for every small commit).
Apparently this was not implemented. Otherwise there would not be too many changes in Factory from YaST packages.
Interesting idea, thanks for sharing it. I think it is interesting approach, so lets collect if anyone else have ideas, prepare plan what need to work and then we can do it. I really like to have clear state of yast features (including ecosystem around like CI ) - dropped or done, but no partially done features, as it just leads to confusion and bugs. Josef
Tschuess, Thomas Fehr
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 11.9.2013 15:10, Lukas Ocilka napsal(a): [...]
All maintained branches would make sense. That's the only way how we can make sure nothing breaks by our change somewhere. Additionally, once a week (for instance), all the checks could be rerun even if no Yast packages were changed - it's because of the possible environment changes.
I do not think we need to run the test "just in case" every week. The tests are run during package build so if there is detected any change outside Yast the build service will trigger rebuild (via package dependencies) and the tests will be re-run. The only difference is that Jenkins sends mails on failures while OBS does not. -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, 12 Sep 2013 10:39:17 +0200 Ladislav Slezak <lslezak@suse.cz> wrote:
Dne 11.9.2013 15:10, Lukas Ocilka napsal(a): [...]
All maintained branches would make sense. That's the only way how we can make sure nothing breaks by our change somewhere. Additionally, once a week (for instance), all the checks could be rerun even if no Yast packages were changed - it's because of the possible environment changes.
I do not think we need to run the test "just in case" every week. The tests are run during package build so if there is detected any change outside Yast the build service will trigger rebuild (via package dependencies) and the tests will be re-run.
The only difference is that Jenkins sends mails on failures while OBS does not.
We can setup hermes to send mails to yast-devel or yast-commit in case of failures so even obs can send mails. Josef
--
Best Regards
Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Wed, Sep 11, 2013 at 02:58:32PM +0200, Josef Reidinger wrote:
testing: current it test only changes in master branch. How important for you if we automatic test also pull requests and maintenance branches? Related question is how important for us is to have generated <package>.spec file as it quite block more generic solutions.
Automatic pull requests for branches would be nice. Packaging for old distros usually requires a development system for that distro. Generating the spec file looks like a must to me. Manual editing VERSION and LIBVERSION changes is error-prone.
autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV?
I never know whether the autosubmit to factory works. Right now I'm expecting submitrequests but there aren't any. I have no idea where to look for a status or problem. The build node (for YaST:Head) should use packages from openSUSE:Factory. E.g. libbtrfs-devel is not available in 12.3 but required for yast2-snapper now. Regards, Arvin -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Wed, 11 Sep 2013 15:17:24 +0200 Arvin Schnell <aschnell@suse.de> wrote:
On Wed, Sep 11, 2013 at 02:58:32PM +0200, Josef Reidinger wrote:
testing: current it test only changes in master branch. How important for you if we automatic test also pull requests and maintenance branches? Related question is how important for us is to have generated <package>.spec file as it quite block more generic solutions.
Automatic pull requests for branches would be nice. Packaging for old distros usually requires a development system for that distro.
Yes, problem is that we need it even now for each branch and I would like to reduce need of such environment or better do it automatic. And one idea is to use osc which can build in chroot. But osc need already spec file.
Generating the spec file looks like a must to me. Manual editing VERSION and LIBVERSION changes is error-prone.
Generating spec file is slightly against previous idea as you need proper version of devtools to generate proper spec file. If problem is really only VERSION and LIBVERSION update, then we can solve it by some kind of seding during making package ( so spec is automatic update when you call make package ). Or is there more things we need to generate?
autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV?
I never know whether the autosubmit to factory works. Right now I'm expecting submitrequests but there aren't any. I have no idea where to look for a status or problem.
Me neither, I try to write email to thomas so we know where it is located and can manage it.
The build node (for YaST:Head) should use packages from openSUSE:Factory. E.g. libbtrfs-devel is not available in 12.3 but required for yast2-snapper now.
If we use idea above ( original author is lslezak ), then osc in its chroot install everything it need with proper target ( so in your case factory ). At first I would like collect requirements and then we can discuss how to make it on workshop. Josef
Regards, Arvin
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 11.9.2013 15:31, Josef Reidinger napsal(a):
And one idea is to use osc which can build in chroot. But osc need already spec file.
Using osc means we need - *.spec file - currently created by create-spec script in devtools, during "make package-local", see https://github.com/yast/yast-devtools/blob/master/devtools/bin/create-spec The history shows that there just few commits in the last several years, see https://github.com/yast/yast-devtools/commits/master/devtools/bin/create-spe... so it's pretty stable. Maybe we could simply switch from *.spec.in to *.spec files? What do you think about it? Pros/cons? Martin? - *.tar.bz2 archive - currently handled by autotools in "make package-local", Maybe we could use something like "git archive <branch>" to simply export the Git content without need to install/run anything? (This is actually the research for the CI integration workshop project...)
Generating spec file is slightly against previous idea as you need proper version of devtools to generate proper spec file. If problem is really only VERSION and LIBVERSION update, then we can solve it by some kind of seding during making package ( so spec is automatic update when you call make package ). Or is there more things we need to generate?
Yes, some simple sed could be possible, but I'd avoid anything more complex than that. For the reasons above. [...]
If we use idea above ( original author is lslezak ), then osc in its chroot install everything it need with proper target ( so in your case factory ).
Yes, the idea is to create *.spec and source tarball and run "osc build". The advantage is that the build runs in a chroot, the package is compiled and tested against the latest Factory versions. Also it handles package dependencies, if you add a new dependency to yast package you need to manually install it into the Jenkins node (in the appropriate version, which might be difficult in some cases) and manually update it if needed. The most problematic part is that obtaining the *.spec and tar ball requires the latest yast2-devtools installed. Moreover "make package-local" requires all needed packages to be installed (because of the checks in "./configure" step). -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 11.9.2013 14:58, Josef Reidinger napsal(a):
Hi developers, we agreed that current CI in jenkins have some problems and I would like to face it. But at first I would like to hear your opinions on current setup, what is weak points and what is nice features we like.
Few questions I have in my mind:
testing: current it test only changes in master branch. How important for you if we automatic test also pull requests and maintenance branches? Related question is how important for us is to have generated <package>.spec file as it quite block more generic solutions.
As Lukas already mentioned all maintained branches should have CI support. But my requirement would be to allow easily adding support for basically any branch. That would be nice for developing some big features in a topic branches. Now you do not know the current status and you realize just after merging to master that some tests are broken. That's little bit late.
autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV?
Autosubmit to YaST:Head is a nice feature, it basically means that I can just click "merge" button in a pull request at GitHub and that's it, I do not need to take care of OBS. So YaST:Head really means "head", there are no missing commits. And there are also checks in Factory which scans for unsubmitted changes in devel projects so also Factory can find possible missing updates.
involvement: Do you want actively help with CI? ( current status is that usually I fix it myself when I see fail ) Do you have own idea what can be automatic or what can help with removing common task?
I would help, but currently there is no documentation where/what can be changed or how does it work actually. For me it looks like a black box which I do not want to touch at all to not break something... AFAIK there is also no clear ownership, i.e. who is supposed to install updates, missing packages, who fixes the scripts etc... This also needs to be clarified. Ideally the code should be at GitHub so we could easily know who/what/why was changed. And also code review would help to make it maintainable.
openess: Problem of current status is that yast CI lives in suse network ( benefit is that we have direct access to our test machines and can easy modify it ). What about using http://ci.opensuse.org/ instance? where then must live workers? I see that cloud guys use it, so maybe we can discuss it with them (mvidner do you have closer information?)?
What would be the benefits? What would be the drawbacks? It also depends how big the ci.opensuse.org infrastructure is. Do they have enough resources for Yast CI? (Keep in mind that in the future we might check more branches, use more runners for parallel runs... etc. so we might need more power than right now so the possibility to extend the infrastructure is also important IMHO.) -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, 12 Sep 2013 10:36:42 +0200 Ladislav Slezak <lslezak@suse.cz> wrote:
Dne 11.9.2013 14:58, Josef Reidinger napsal(a):
Hi developers, we agreed that current CI in jenkins have some problems and I would like to face it. But at first I would like to hear your opinions on current setup, what is weak points and what is nice features we like.
Few questions I have in my mind:
testing: current it test only changes in master branch. How important for you if we automatic test also pull requests and maintenance branches? Related question is how important for us is to have generated <package>.spec file as it quite block more generic solutions.
As Lukas already mentioned all maintained branches should have CI support.
But my requirement would be to allow easily adding support for basically any branch. That would be nice for developing some big features in a topic branches. Now you do not know the current status and you realize just after merging to master that some tests are broken. That's little bit late.
Also integration with pull requests can be nice as we see imediatelly if it pass tests like travis have it.
autosubmit: Are you satisfied with current autosubmit to Yast:Head? And what about automatic submit to factory? What is criteria to submit to Yast:Head and to factory? Do you want it automatic or some someautomatic or manual step is better from your POV?
Autosubmit to YaST:Head is a nice feature, it basically means that I can just click "merge" button in a pull request at GitHub and that's it, I do not need to take care of OBS.
So YaST:Head really means "head", there are no missing commits. And there are also checks in Factory which scans for unsubmitted changes in devel projects so also Factory can find possible missing updates.
involvement: Do you want actively help with CI? ( current status is that usually I fix it myself when I see fail ) Do you have own idea what can be automatic or what can help with removing common task?
I would help, but currently there is no documentation where/what can be changed or how does it work actually. For me it looks like a black box which I do not want to touch at all to not break something...
AFAIK there is also no clear ownership, i.e. who is supposed to install updates, missing packages, who fixes the scripts etc... This also needs to be clarified.
Ideally the code should be at GitHub so we could easily know who/what/why was changed. And also code review would help to make it maintainable.
I agreed, it would be nice.
openess: Problem of current status is that yast CI lives in suse network ( benefit is that we have direct access to our test machines and can easy modify it ). What about using http://ci.opensuse.org/ instance? where then must live workers? I see that cloud guys use it, so maybe we can discuss it with them (mvidner do you have closer information?)?
What would be the benefits? What would be the drawbacks?
It also depends how big the ci.opensuse.org infrastructure is. Do they have enough resources for Yast CI? (Keep in mind that in the future we might check more branches, use more runners for parallel runs... etc. so we might need more power than right now so the possibility to extend the infrastructure is also important IMHO.)
It should run in virtual machine like we already do for river. In such case it scale really nice and even hundreds of project is fine when it just login to target node and run there bunch of commands. Josef
--
Best Regards
Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (5)
-
Arvin Schnell -
Josef Reidinger -
Ladislav Slezak -
Lukas Ocilka -
Thomas Fehr