[opensuse-buildservice] Thoughts on OBS as a git bisect build cluster
Hello- I want to create a web based tool where users could do git bisections themselves by plugging in starting hashes (provided by a developer most likely) and get an RPM out. Then as they test they come back to the website to report their test results, good or bad, to get the next rpm to test. Now, the most intensive part would be doing all of the builds and checkouts. Something that I think OBS could handle nicely :D First, is this use of OBS is acceptable? If so the next challenge is avoiding lengthy and expensive uploads to the OBS. I have thought up a few strategies for doing such and would appreciate feedback: 1. Create an OBS project with a base version of the project tarball e.g. Kernel v2.6.28, then generate a patch with: $ git diff v2.6.28...<$BISECTHASH> Then branch this OBS project to create a project that will build the RPM for the user with this git diff patch applied. 2. Create an OBS project with a tar snapshot of the external git repo and submit it to an OBS project. Then create an OBS branch from this project for each git bisection step and submit some control file that tells the spec file what hash to build. 3. Have a spec file that does a clone from some internal mirror of the external git tree and does the bisect locally. (Can you get to the internet when building inside a VM?) For #1 and #2 I would need a way to submit a patch to a branched project _without_ doing a full checkout of the tarball again. How would I change one file in an OBS branch without downloading the whole repo? What do people think? Any other strategies to create something like this that uses OBS on the backend? Cheers, Brandon -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Hello Brandon, Am Samstag, 28. Februar 2009 02:56:20 schrieb Brandon Philips:
Hello-
I want to create a web based tool where users could do git bisections themselves by plugging in starting hashes (provided by a developer most likely) and get an RPM out. Then as they test they come back to the website to report their test results, good or bad, to get the next rpm to test.
Now, the most intensive part would be doing all of the builds and checkouts. Something that I think OBS could handle nicely :D
First, is this use of OBS is acceptable?
I see two sides on this: 1. It is good to have such a support. The idea is good and it can indeed help in various cases. If not for build.opensuse.org it is at least very good to have for other instances. 2. OBS at build.opensuse.org is already very high loaded most of the time, it still has it's free cycles. So I would propose that we invent a system to mark such builds somehow as low priority by default. This support might use the temprary build support, fate #305931, instead of creating porjects/packages btw.
If so the next challenge is avoiding lengthy and expensive uploads to the OBS. I have thought up a few strategies for doing such and would appreciate feedback:
1. Create an OBS project with a base version of the project tarball e.g. Kernel v2.6.28, then generate a patch with: $ git diff v2.6.28...<$BISECTHASH> Then branch this OBS project to create a project that will build the RPM for the user with this git diff patch applied.
I would like to see also support to rebuild a submitted source changes, based on the history of our source server.
2. Create an OBS project with a tar snapshot of the external git repo and submit it to an OBS project. Then create an OBS branch from this project for each git bisection step and submit some control file that tells the spec file what hash to build.
3. Have a spec file that does a clone from some internal mirror of the external git tree and does the bisect locally. (Can you get to the internet when building inside a VM?)
No, by intention.
For #1 and #2 I would need a way to submit a patch to a branched project _without_ doing a full checkout of the tarball again. How would I change one file in an OBS branch without downloading the whole repo?
What do people think? Any other strategies to create something like this that uses OBS on the backend?
From my point of view, we have way more often the case that we need to findout regressions about submitted packages. Just because this is what our users are testing. So from my personal perspective it is more important to rebuild all submitted sources than working on external cvs/svn/git repos. bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Montag, 2. März 2009 10:24:38 schrieb Adrian Schröter:
Hello Brandon,
Am Samstag, 28. Februar 2009 02:56:20 schrieb Brandon Philips: ...
3. Have a spec file that does a clone from some internal mirror of the external git tree and does the bisect locally. (Can you get to the internet when building inside a VM?)
No, by intention.
But you can run your own build service instance with chroot envirment only. In that case you have network. But of course also all security/privacy problems also ;) bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Brandon, I like the idea. You could greatly reduce the network traffic by just adding small incremental patches with each upload: foofoo.spec foofoo-$BASE.tbz foofoo-$BASE-$hash1.patch foofoo-$hash1-$hash2.patch foofoo-$hash2-$hash3.patch Then apply the patches with some quilt control file, for example. Once the testing is successfull, you go to the $BASE_NEW revision tar ball (which now has all the incremental patches applied): Would that work for your use case? S. -- Susanne Oberhauser +49-911-74053-574 SUSE -- a Novell Business OPS Engineering Maxfeldstraße 5 Processes and Infrastructure Nürnberg SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Susanne Oberhauser napsal(a):
Brandon,
I like the idea.
You could greatly reduce the network traffic by just adding small incremental patches with each upload:
foofoo.spec foofoo-$BASE.tbz foofoo-$BASE-$hash1.patch foofoo-$hash1-$hash2.patch foofoo-$hash2-$hash3.patch
linux-2.6.x.tar.bz2 and _small_ patches? ;-) Why not upload a clone of the git repository once and then just checkout individual commits? Michal -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Michal Marek <mmarek@suse.cz> writes:
Susanne Oberhauser napsal(a):
Brandon,
I like the idea.
You could greatly reduce the network traffic by just adding small incremental patches with each upload:
foofoo.spec foofoo-$BASE.tbz foofoo-$BASE-$hash1.patch foofoo-$hash1-$hash2.patch foofoo-$hash2-$hash3.patch
linux-2.6.x.tar.bz2 and _small_ patches? ;-) Why not upload a clone of the git repository once and then just checkout individual commits?
If it's for isolating which patch broke the whole thing, that's another nice approach. For ongoing testing of incremental updates, uploading the (new,not preexisting) patches. Looking at $SUBJECT I guess "git bisection" is more about figuring what introduced some regression? which is case a)? that said, relative to the linux.tbz, the patches still are small ;) S. -- Susanne Oberhauser +49-911-74053-574 SUSE -- a Novell Business OPS Engineering Maxfeldstraße 5 Processes and Infrastructure Nürnberg SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (4)
-
Adrian Schröter
-
Brandon Philips
-
Michal Marek
-
Susanne Oberhauser