[opensuse-buildservice] Build hanging forever when worker dies/is shut down during build
Hi, is there a way to prevent obs builds to hang forever in case of a failing or stopped worker that is currently building? The only way I found is to cd to /srv/obs/jobs/XXX and remove the job file. Is there a timeout or something similar which can be set? -- With best regards, Carsten Hoeger
I've had similar issues in that OBS status still displays workers that have been shutdown either with /etc/init.d/obsworker stop or by shutting down the worker machine itself. Shouldn't /etc/init.d/obsworker stop send a message to the server telling it to remove the worker? On Fri, 2009-03-27 at 15:53 +0100, Carsten Hoeger wrote:
Hi,
is there a way to prevent obs builds to hang forever in case of a failing or stopped worker that is currently building?
The only way I found is to cd to /srv/obs/jobs/XXX and remove the job file.
Is there a timeout or something similar which can be set?
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Mar 27, Luke Imhoff wrote:
I've had similar issues in that OBS status still displays workers that have been shutdown either with
/etc/init.d/obsworker stop
or by shutting down the worker machine itself. Shouldn't /etc/init.d/obsworker stop send a message to the server telling it to remove the worker?
Yes, I'd also expect that. Should I create a bug report or is that just a matter of a borked installation?
On Fri, 2009-03-27 at 15:53 +0100, Carsten Hoeger wrote:
Hi,
is there a way to prevent obs builds to hang forever in case of a failing or stopped worker that is currently building?
The only way I found is to cd to /srv/obs/jobs/XXX and remove the job file.
Is there a timeout or something similar which can be set?
-- With best regards, Carsten Hoeger
I am running a 1.0 + some patches, so it may be fixed in 1.5. We haven't had time to merge 1.5 in with our changes. On Fri, 2009-03-27 at 17:52 +0100, Carsten Hoeger wrote:
On Fri, Mar 27, Luke Imhoff wrote:
I've had similar issues in that OBS status still displays workers that have been shutdown either with
/etc/init.d/obsworker stop
or by shutting down the worker machine itself. Shouldn't /etc/init.d/obsworker stop send a message to the server telling it to remove the worker?
Yes, I'd also expect that. Should I create a bug report or is that just a matter of a borked installation?
On Fri, 2009-03-27 at 15:53 +0100, Carsten Hoeger wrote:
Hi,
is there a way to prevent obs builds to hang forever in case of a failing or stopped worker that is currently building?
The only way I found is to cd to /srv/obs/jobs/XXX and remove the job file.
Is there a timeout or something similar which can be set?
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
I'm running the last RC before 1.5 was released, and I see the same issues. It is at least somewhat annoying. I have found that if you reboot a stopped worker, it picks up where it left off, but if a worker stops and never restarts, it leaves jobs hanging out. This is definitely a problem, not huge or catastrophic on the big picture, but annoying nonetheless. -- Michael -----Original Message----- From: Luke Imhoff [mailto:luke@cray.com] Sent: Fri 3/27/2009 12:00 PM To: Carsten Hoeger Cc: opensuse-buildservice@opensuse.org Subject: Re: [opensuse-buildservice] Build hanging forever when worker dies/is shut down during build I am running a 1.0 + some patches, so it may be fixed in 1.5. We haven't had time to merge 1.5 in with our changes. On Fri, 2009-03-27 at 17:52 +0100, Carsten Hoeger wrote:
On Fri, Mar 27, Luke Imhoff wrote:
I've had similar issues in that OBS status still displays workers that have been shutdown either with
/etc/init.d/obsworker stop
or by shutting down the worker machine itself. Shouldn't /etc/init.d/obsworker stop send a message to the server telling it to remove the worker?
Yes, I'd also expect that. Should I create a bug report or is that just a matter of a borked installation?
On Fri, 2009-03-27 at 15:53 +0100, Carsten Hoeger wrote:
Hi,
is there a way to prevent obs builds to hang forever in case of a failing or stopped worker that is currently building?
The only way I found is to cd to /srv/obs/jobs/XXX and remove the job file.
Is there a timeout or something similar which can be set?
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Mar 27, Michael_E_Brown@Dell.com wrote:
I'm running the last RC before 1.5 was released, and I see the same issues.
It is at least somewhat annoying. I have found that if you reboot a stopped worker, it picks up where it left off, but if a worker stops and never restarts, it leaves jobs hanging out.
This is definitely a problem, not huge or catastrophic on the big picture, but annoying nonetheless.
Just created a bug report for this issue: https://bugzilla.novell.com/show_bug.cgi?id=490604
-- Michael
-----Original Message----- From: Luke Imhoff [mailto:luke@cray.com] Sent: Fri 3/27/2009 12:00 PM To: Carsten Hoeger Cc: opensuse-buildservice@opensuse.org Subject: Re: [opensuse-buildservice] Build hanging forever when worker dies/is shut down during build
I am running a 1.0 + some patches, so it may be fixed in 1.5. We haven't had time to merge 1.5 in with our changes.
On Fri, 2009-03-27 at 17:52 +0100, Carsten Hoeger wrote:
On Fri, Mar 27, Luke Imhoff wrote:
I've had similar issues in that OBS status still displays workers that have been shutdown either with
/etc/init.d/obsworker stop
or by shutting down the worker machine itself. Shouldn't /etc/init.d/obsworker stop send a message to the server telling it to remove the worker?
Yes, I'd also expect that. Should I create a bug report or is that just a matter of a borked installation?
On Fri, 2009-03-27 at 15:53 +0100, Carsten Hoeger wrote:
Hi,
is there a way to prevent obs builds to hang forever in case of a failing or stopped worker that is currently building?
The only way I found is to cd to /srv/obs/jobs/XXX and remove the job file.
Is there a timeout or something similar which can be set?
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- With best regards, Carsten Hoeger
participants (3)
-
Carsten Hoeger -
Luke Imhoff -
Michael_E_Brown＠Dell.com