[opensuse-buildservice] OBS DoS caused by a broken package
Hello, right now the OBS is mostly idle, trying to repeatedly build home:prusnak:scout/scout. The full log I managed to capture is at http://pastebin.ca/1773632, the interesting parts seem to be: ===== ... preinstalling bash... ... preinstalling libreadline6... ... /bin/bash: error while loading shared libraries: libreadline.so.5: cannot open shared object file: No such file or directory Kernel panic - not syncing: Attempted to kill init! ===== I'm not sure why exactly this got broken, but home:prusnak:scout includes also a bash package, so I assume the preinstall somehow used packages also from there, and it got somehow messed up. The real problem however is that the OBS was repeatedly trying to dispatch this build, it failed almost immediately and the OBS retried again, and again, and again. It managed to dispatch about 1 other build per minute. I think this is a consequence of the recent fix that retries builds that failed because of the build host itself instead of the package. Would it be possible to adjust it so that such failed builds are put at the end of the queue instead of being handled again right away? -- Lubos Lunak openSUSE Boosters team, KDE developer l.lunak@suse.cz , l.lunak@kde.org -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 01/31/2010 11:53 PM, Lubos Lunak wrote:
I'm not sure why exactly this got broken, but home:prusnak:scout includes also a bash package, so I assume the preinstall somehow used packages also from there, and it got somehow messed up.
I removed the bash package from the project (it was needed only for 11.0) and called 'osc abortbuild'. I think it should be OK now ... -- Best Regards / S pozdravom, Pavol RUSNAK SUSE LINUX, s.r.o openSUSE Boosters Team Lihovarska 1060/12 PGP 0xA6917144 19000 Praha 9, CR prusnak[at]suse.cz http://www.suse.cz -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Sonntag, 31. Januar 2010 23:53:59 schrieb Lubos Lunak:
Hello,
right now the OBS is mostly idle, trying to repeatedly build home:prusnak:scout/scout. The full log I managed to capture is at http://pastebin.ca/1773632, the interesting parts seem to be: ===== ... preinstalling bash... ... preinstalling libreadline6... ... /bin/bash: error while loading shared libraries: libreadline.so.5: cannot open shared object file: No such file or directory Kernel panic - not syncing: Attempted to kill init! =====
I'm not sure why exactly this got broken, but home:prusnak:scout includes also a bash package, so I assume the preinstall somehow used packages also from there, and it got somehow messed up.
The real problem however is that the OBS was repeatedly trying to dispatch this build, it failed almost immediately and the OBS retried again, and again, and again. It managed to dispatch about 1 other build per minute.
Yes, that is a new behaviour. OBS is handling this as "Bad build host", because even the bash inside of the VM is not able to start and to write any status. (it was a change from me 1-2 weeks ago).
I think this is a consequence of the recent fix that retries builds that failed because of the build host itself instead of the package. Would it be possible to adjust it so that such failed builds are put at the end of the queue instead of being handled again right away?
No, that makes not really sense, because it either is a bad host, than it should not affect the priority. Or it is one and the server should give up. But I will add some support that the dispatcher is giving up after 3 tries with "bad host". bye adrian
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Monday 01 of February 2010, Adrian Schröter wrote:
Am Sonntag, 31. Januar 2010 23:53:59 schrieb Lubos Lunak:
I think this is a consequence of the recent fix that retries builds that failed because of the build host itself instead of the package. Would it be possible to adjust it so that such failed builds are put at the end of the queue instead of being handled again right away?
No, that makes not really sense, because it either is a bad host, than it should not affect the priority. Or it is one and the server should give up.
But I will add some support that the dispatcher is giving up after 3 tries with "bad host".
There appears to be a problem with this. Currently the OBS is again spinning a few packages in devel:languages:perl. It randomly switches between broken1.txt and broken2.txt . Notice in the second log the following: No buildstatus set, packager broke either the base system (glibc/bash) or the build host has a problem, server will retry... acheSession.spec ... init_buildsystem --repository zypp:// /.build-srcdir/perl-MasonX-Request-WithApacheSession.spec build rpmlint-Factory ... -- Lubos Lunak openSUSE Boosters team, KDE developer l.lunak@suse.cz , l.lunak@kde.org
Am Mittwoch, 10. Februar 2010 09:03:46 schrieb Lubos Lunak:
On Monday 01 of February 2010, Adrian Schröter wrote:
Am Sonntag, 31. Januar 2010 23:53:59 schrieb Lubos Lunak:
I think this is a consequence of the recent fix that retries builds that failed because of the build host itself instead of the package. Would it be possible to adjust it so that such failed builds are put at the end of the queue instead of being handled again right away?
No, that makes not really sense, because it either is a bad host, than it should not affect the priority. Or it is one and the server should give up.
But I will add some support that the dispatcher is giving up after 3 tries with "bad host".
There appears to be a problem with this. Currently the OBS is again spinning a few packages in devel:languages:perl. It randomly switches between broken1.txt and broken2.txt . Notice in the second log the following:
there was a broken perl in your project. built, but build disabled afterwards. This is one of the ways to destroy the build root. coolo and me called "osc wipebinaries --build-disabled" for this project and this should solve this issue.
No buildstatus set, packager broke either the base system (glibc/bash) or the build host has a problem, server will retry... acheSession.spec ... init_buildsystem --repository zypp:// /.build-srcdir/perl-MasonX-Request-WithApacheSession.spec build rpmlint-Factory ...
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (3)
-
Adrian Schröter -
Lubos Lunak -
Pavol Rusnak