[opensuse-buildservice] Build synchronization bug?
Hey, I've noticed this a few times now: I'll commit an update to a package (via osc if that matters) and then go to the website to watch the build progress for that package. If it happens to be a point where the buildservice is relatively idle, the package will usually have started rebuilding against all of the repositories I've assigned. On multiple occasions I've gone to view a build log for the package, catching it in progress. I'll watch it continue building to completion, only to discover at the end that it just rebuilt the *previous* version of the package, If I then go back to the package status page, the package will have had a new build kicked off for the same repository target; returning to the log, it will indeed be a build for the newly committed version of this package. Is this known behavior? Expected behavior? Please let me know if there's anything in the above that I need to clarify. Thanks. -- Steve Beattie SUSE Labs, Novell Inc. <sbeattie@suse.de> http://NxNW.org/~steve/
On Sunday 05 November 2006 08:41, Steve Beattie wrote:
catching it in progress. I'll watch it continue building to completion, only to discover at the end that it just rebuilt the *previous* version of the package, If I then go back to the package status page, the package
I also watches this regularly. :-( Bye, Steve --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Sunday 05 November 2006 08:41 schrieb Steve Beattie:
I've noticed this a few times now: I'll commit an update to a package (via osc if that matters) and then go to the website to watch the build progress for that package. If it happens to be a point where the buildservice is relatively idle, the package will usually have started rebuilding against all of the repositories I've assigned.
On multiple occasions I've gone to view a build log for the package, catching it in progress. I'll watch it continue building to completion, only to discover at the end that it just rebuilt the *previous* version of the package, If I then go back to the package status page, the package will have had a new build kicked off for the same repository target; returning to the log, it will indeed be a build for the newly committed version of this package.
Is this known behavior? Expected behavior? Please let me know if there's anything in the above that I need to clarify.
This can happen, when the client is changing something (lets say some unimportant file) and commits right afterwards, instead of submitting all other changes before. However, I think that osc does handle this in the right way already, so it should not happen using it. Peter, is this true ? bye adrian -- Adrian Schroeter SUSE Linux Products GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany email: adrian@suse.de --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Mon, Nov 06, 2006 at 10:16:01AM +0100, Adrian Schröter wrote:
This can happen, when the client is changing something (lets say some unimportant file) and commits right afterwards, instead of submitting all other changes before.
However, I think that osc does handle this in the right way already, so it should not happen using it.
If it helps for debugging, all I'm doing is adding a tarball, removing an old tarball[1], and updating a specfile, and then committing. In terms of osc operations, all I'm doing is 'osc addremove; osc ci'. [1] The tarballs have the svn repo version number as part of their name so that it's obvious to me what it corresponds to in our svn server. -- Steve Beattie SUSE Labs, Novell Inc. <sbeattie@suse.de> http://NxNW.org/~steve/
On Mon, Nov 06, 2006 at 10:16:01AM +0100, Adrian Schröter wrote:
Am Sunday 05 November 2006 08:41 schrieb Steve Beattie:
I've noticed this a few times now: I'll commit an update to a package (via osc if that matters) and then go to the website to watch the build progress for that package. If it happens to be a point where the buildservice is relatively idle, the package will usually have started rebuilding against all of the repositories I've assigned.
On multiple occasions I've gone to view a build log for the package, catching it in progress. I'll watch it continue building to completion, only to discover at the end that it just rebuilt the *previous* version of the package, If I then go back to the package status page, the package will have had a new build kicked off for the same repository target; returning to the log, it will indeed be a build for the newly committed version of this package.
Is this known behavior? Expected behavior? Please let me know if there's anything in the above that I need to clarify.
This can happen, when the client is changing something (lets say some unimportant file) and commits right afterwards, instead of submitting all other changes before.
However, I think that osc does handle this in the right way already, so it should not happen using it.
Peter, is this true ?
In former times, each commit would trigger a rebuild, which would invalidate (kill) ongoing build jobs. Wouldn't it? Has this changed? If new builds are scheduled behing ongoing builds, that would explain the behaviour (which I have been seeing myself a lot as well). Although I (so far) claimed that build jobs aren't just done twice, but even more often. If "obsolete" build jobs are not (no longer?) killed: what's the rationale? Regards, Peter -- SUSE LINUX Products GmbH Bug, bogey, bugbear, bugaboo: Research & Development A malevolent monster (not true?); Some mischief microbic; What makes someone phobic; The work one does not want to do. From: Chris Young (The Omnificent English Dictionary In Limerick Form)
Hi,
On Mon, Nov 06, 2006 at 10:16:01AM +0100, Adrian Schröter wrote:
Am Sunday 05 November 2006 08:41 schrieb Steve Beattie:
I've noticed this a few times now: I'll commit an update to a package (via osc if that matters) and then go to the website to watch the build progress for that package. If it happens to be a point where the buildservice is relatively idle, the package will usually have started rebuilding against all of the repositories I've assigned.
On multiple occasions I've gone to view a build log for the package, catching it in progress. I'll watch it continue building to completion, only to discover at the end that it just rebuilt the *previous* version of the package, If I then go back to the package status page, the package will have had a new build kicked off for the same repository target; returning to the log, it will indeed be a build for the newly committed version of this package.
Is this known behavior? Expected behavior? Please let me know if there's anything in the above that I need to clarify.
This can happen, when the client is changing something (lets say some unimportant file) and commits right afterwards, instead of submitting all other changes before.
However, I think that osc does handle this in the right way already, so it should not happen using it.
Peter, is this true ?
In former times, each commit would trigger a rebuild, which would invalidate (kill) ongoing build jobs. Wouldn't it?
Has this changed?
If new builds are scheduled behing ongoing builds, that would explain the behaviour (which I have been seeing myself a lot as well).
Although I (so far) claimed that build jobs aren't just done twice, but even more often.
If "obsolete" build jobs are not (no longer?) killed: what's the rationale?
I have seen the same. If you change for example the spec file twice times with "osc ci" commands in between I can see how the build is scheduled -> building -> scheduled -> building. But each build runs until it's finished. Regards Marcus -- Public Key available ------------------------------------------------------- Marcus Schäfer (Res. & Dev.) SUSE LINUX Products GmbH Tel: 07562-905437 Unterösch 9 FAX: 07562-905438 D-88316 Isny / Rohrdorf http://www.suse.de Germany [ HomeOffice ] ------------------------------------------------------- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Tuesday 07 November 2006 17:30 schrieb Marcus Schäfer:
Hi,
On Mon, Nov 06, 2006 at 10:16:01AM +0100, Adrian Schröter wrote:
Am Sunday 05 November 2006 08:41 schrieb Steve Beattie:
I've noticed this a few times now: I'll commit an update to a package (via osc if that matters) and then go to the website to watch the build progress for that package. If it happens to be a point where the buildservice is relatively idle, the package will usually have started rebuilding against all of the repositories I've assigned.
On multiple occasions I've gone to view a build log for the package, catching it in progress. I'll watch it continue building to completion, only to discover at the end that it just rebuilt the *previous* version of the package, If I then go back to the package status page, the package will have had a new build kicked off for the same repository target; returning to the log, it will indeed be a build for the newly committed version of this package.
Is this known behavior? Expected behavior? Please let me know if there's anything in the above that I need to clarify.
This can happen, when the client is changing something (lets say some unimportant file) and commits right afterwards, instead of submitting all other changes before.
However, I think that osc does handle this in the right way already, so it should not happen using it.
Peter, is this true ?
In former times, each commit would trigger a rebuild, which would invalidate (kill) ongoing build jobs. Wouldn't it?
Has this changed?
If new builds are scheduled behing ongoing builds, that would explain the behaviour (which I have been seeing myself a lot as well).
Although I (so far) claimed that build jobs aren't just done twice, but even more often.
If "obsolete" build jobs are not (no longer?) killed: what's the rationale?
I have seen the same. If you change for example the spec file twice times with "osc ci" commands in between I can see how the build is scheduled -> building -> scheduled -> building. But each build runs until it's finished.
right, build job killing is not yet implemented. Independend from that, I think the commit needs anyway to be atomar for all changes. Because you want to see later which files/changes did happen with one commit. bye adrian -- Adrian Schroeter SUSE Linux Products GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany email: adrian@suse.de --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Nov 07, 2006 at 05:41:43PM +0100, Adrian Schröter wrote:
I have seen the same. If you change for example the spec file twice times with "osc ci" commands in between I can see how the build is scheduled -> building -> scheduled -> building. But each build runs until it's finished.
right, build job killing is not yet implemented.
I thought it was -- okay.
Independend from that, I think the commit needs anyway to be atomar for all changes. Because you want to see later which files/changes did happen with one commit.
It is unrelated though. Even with osc doing atomar commits, I (as a developer) will find myself editing another file (changelog), and committing again. Forgetting something, and so on. It is not reasonable to queue up the build jobs. Regards, Peter -- SUSE LINUX Products GmbH Bug, bogey, bugbear, bugaboo: Research & Development A malevolent monster (not true?); Some mischief microbic; What makes someone phobic; The work one does not want to do. From: Chris Young (The Omnificent English Dictionary In Limerick Form)
On Tue, Nov 07, 2006 at 05:30:51PM +0100, Marcus Schäfer wrote:
I have seen the same. If you change for example the spec file twice times with "osc ci" commands in between I can see how the build is scheduled -> building -> scheduled -> building. But each build runs until it's finished.
While I can understand the above behavior occurring, this is *not* the situation I'm talking about. I make all my change to my checked out repo and do *one commit* (one invocation of 'osc ci') and then see *two* builds; one of the precommit version of the package, and then a followup build of the newly committed version. -- Steve Beattie SUSE Labs, Novell Inc. <sbeattie@suse.de> http://NxNW.org/~steve/
On 2006-11-07 09:23:24 -0800, Steve Beattie wrote:
Date: Tue, 7 Nov 2006 09:23:24 -0800 From: Steve Beattie <sbeattie@suse.de> Subject: Re: [opensuse-buildservice] Build synchronization bug? To: opensuse-buildservice@opensuse.org
On Tue, Nov 07, 2006 at 05:30:51PM +0100, Marcus Schäfer wrote:
I have seen the same. If you change for example the spec file twice times with "osc ci" commands in between I can see how the build is scheduled -> building -> scheduled -> building. But each build runs until it's finished.
While I can understand the above behavior occurring, this is *not* the situation I'm talking about. I make all my change to my checked out repo and do *one commit* (one invocation of 'osc ci') and then see *two* builds; one of the precommit version of the package, and then a followup build of the newly committed version.
anyway. if you changed multiple files and run "osc ci" it will be uploaded as multiple requests atm. so this might be the explaination for the unneeded rebuilds. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Nov 07, 2006 at 06:37:24PM +0100, Marcus Rueckert wrote:
While I can understand the above behavior occurring, this is *not* the situation I'm talking about. I make all my change to my checked out repo and do *one commit* (one invocation of 'osc ci') and then see *two* builds; one of the precommit version of the package, and then a followup build of the newly committed version.
anyway. if you changed multiple files and run "osc ci" it will be uploaded as multiple requests atm. so this might be the explaination for the unneeded rebuilds.
Ah really? Yes, indeed, one file gets deleted, one file gets added, and the specfile gets modified. I would have thought a commit operation would have been atomic. If true, that explains it. -- Steve Beattie SUSE Labs, Novell Inc. <sbeattie@suse.de> http://NxNW.org/~steve/
On 2006-11-07 09:44:58 -0800, Steve Beattie wrote:
On Tue, Nov 07, 2006 at 06:37:24PM +0100, Marcus Rueckert wrote:
While I can understand the above behavior occurring, this is *not* the situation I'm talking about. I make all my change to my checked out repo and do *one commit* (one invocation of 'osc ci') and then see *two* builds; one of the precommit version of the package, and then a followup build of the newly committed version.
anyway. if you changed multiple files and run "osc ci" it will be uploaded as multiple requests atm. so this might be the explaination for the unneeded rebuilds.
Ah really? Yes, indeed, one file gets deleted, one file gets added, and the specfile gets modified. I would have thought a commit operation would have been atomic. If true, that explains it.
to be done. at the moment i wish we had svn as transport and could just use all the transaction magic from it. :) a simple post-commit for the "trigger-rebuild" hah... but that sounds to easy and i am sure mls will kill me for that idea. :) darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Nov 07, 2006 at 06:53:46PM +0100, Marcus Rueckert wrote:
On 2006-11-07 09:44:58 -0800, Steve Beattie wrote:
On Tue, Nov 07, 2006 at 06:37:24PM +0100, Marcus Rueckert wrote:
While I can understand the above behavior occurring, this is *not* the situation I'm talking about. I make all my change to my checked out repo and do *one commit* (one invocation of 'osc ci') and then see *two* builds; one of the precommit version of the package, and then a followup build of the newly committed version.
anyway. if you changed multiple files and run "osc ci" it will be uploaded as multiple requests atm. so this might be the explaination for the unneeded rebuilds.
Ah really? Yes, indeed, one file gets deleted, one file gets added, and the specfile gets modified. I would have thought a commit operation would have been atomic. If true, that explains it.
to be done. at the moment i wish we had svn as transport and could just use all the transaction magic from it. :) a simple post-commit for the "trigger-rebuild" hah... but that sounds to easy and i am sure mls will kill me for that idea. :)
explicit rebuilding? Yes, people have been arguing for it since day one ;) _I_ would love if we had a WebDAV capable server (with delta extensions, like subversion uses), so I (we) wouldn't need to invent anything anew, in some (then) limited form. Or mayby Amazon's S3 API. (Or Google FS? But I have only a vague idea about the latter.) It would instantaneously allow to use a number of clients to mount the filesystem, edit files, etc. (konqueror, davfs, libraries, ...) On the other hand, if we actually used an svn client on the client side, I think we would have a hard time to extend it with the features we need. Not only commands like rebuild triggering and such. The entire protocol that we use to access the API would need to be implemented. And what osc does to supply the functionality of the svn client which we need, is much more simple than it might look, it is done in like 1000 or 2000 lines of code. (Admittedly, osc is still very basic. I'm looking forward to have time to work on it soon.) Another reason against subversion which I have in mind is that I see a long-term advantage in a slender and lightweight and specific implementation -- which just suits our needs. In the spirit of Amazon S3 web services, or Google's stuff. Using a full-blown solution as starting base will likely scale worse. Provided that we get the design right, we are on the right way I hope... And while I'm putting together my list of Christmas wishes: I _wish_ we would use Digest Authentication instead of SSL+Basic, which sucks for development because it makes debugging harder, and otherwise slows things unnecessarily down. Digest auth is secure without SSL. Peter -- SUSE LINUX Products GmbH Bug, bogey, bugbear, bugaboo: Research & Development A malevolent monster (not true?); Some mischief microbic; What makes someone phobic; The work one does not want to do. From: Chris Young (The Omnificent English Dictionary In Limerick Form)
On 2006-11-07 22:55:32 +0100, Dr. Peter Poeml wrote:
explicit rebuilding? Yes, people have been arguing for it since day one ;)
no. still automatic rebuilds. just use a post-commit hook to notify the scheduler of the new files. about the other stuff i have to think a bit more. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Sat, Nov 04, 2006 at 11:41:20PM -0800, Steve Beattie wrote:
On multiple occasions I've gone to view a build log for the package, catching it in progress. I'll watch it continue building to completion, only to discover at the end that it just rebuilt the *previous* version of the package, If I then go back to the package status page, the package will have had a new build kicked off for the same repository target; returning to the log, it will indeed be a build for the newly committed version of this package.
Steve, thanks for debugging this! Regards, Peter -- SUSE LINUX Products GmbH Bug, bogey, bugbear, bugaboo: Research & Development A malevolent monster (not true?); Some mischief microbic; What makes someone phobic; The work one does not want to do. From: Chris Young (The Omnificent English Dictionary In Limerick Form)
participants (6)
-
Adrian Schröter -
Dr. Peter Poeml -
Marcus Rueckert -
Marcus Schäfer -
Stephan Binner -
Steve Beattie