[opensuse-buildservice] Remote OBS: build fails with 400 remote error: remote error Bad request
Hi all, I have a small issue on my private OBS instance, which remote links to build.o.o. I see some of my packages in status failed (even though they used to work / build just fine without code changes). osc bl shows me: 400 remote error: remote error Bad request A little bit of digging in src_server.log, this error is apparently an error from a too long list of remote sources supposed to be downloaded. The matching line in src_server.log: 2013-01-28 06:36:25 [9698]: GET /getbinaries?project=openSUSE.org:SUSE:SLE-11:SP2&repository=standard&arch=x86_64&binaries=aaa_base,acl,*many more packages listed*,perl-TimeDate,rpmlint-mini (AJAX) 400 remote error: Bad request A very quick check shows that this get is somewhere between 4 and 5K in size.. so not really surprising that this causes issues.. When using osc build, there is the option --disable-cpio-bulk-download to work around the exact same issue. Is there a way to set an OBS instance to do the same? It clearly can not handle such a large chunk and as such should have some logic to not send such huge requests (maybe split it in multiple requests of only 1k each?) My OBS version: obs-api-2.3.91_736_gfc3358d-1.1.x86_64 obs-server-2.3.91_736_gfc3358d-1.1.x86_64 Best regards Dominique -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Mon, Jan 28, 2013 at 10:34:30AM +0100, Dominique Leuenberger a.k.a. Dimstar wrote:
I have a small issue on my private OBS instance, which remote links to build.o.o.
I see some of my packages in status failed (even though they used to work / build just fine without code changes). osc bl shows me:
400 remote error: remote error Bad request
A little bit of digging in src_server.log, this error is apparently an error from a too long list of remote sources supposed to be downloaded.
The matching line in src_server.log: 2013-01-28 06:36:25 [9698]: GET /getbinaries?project=openSUSE.org:SUSE:SLE-11:SP2&repository=standard&arch=x86_64&binaries=aaa_base,acl,*many more packages listed*,perl-TimeDate,rpmlint-mini (AJAX) 400 remote error: Bad request
A very quick check shows that this get is somewhere between 4 and 5K in size.. so not really surprising that this causes issues..
When using osc build, there is the option --disable-cpio-bulk-download to work around the exact same issue. Is there a way to set an OBS instance to do the same? It clearly can not handle such a large chunk and as such should have some logic to not send such huge requests (maybe split it in multiple requests of only 1k each?)
My OBS version: obs-api-2.3.91_736_gfc3358d-1.1.x86_64 obs-server-2.3.91_736_gfc3358d-1.1.x86_64
It already splits the download into chunks of 50 packages, but there's a request to get the state of the binaries done at the beginning of the worker_getbinaries function that may result in a too bug URL. There's an easy way to fix this: we can simply ask for all the binaries if the list is too big (untested diff): --- bs_srcserver.orig 2013-01-21 12:12:58.000000000 +0100 +++ bs_srcserver 2013-01-28 11:31:38.049503000 +0100 @@ -6718,7 +6718,10 @@ 'uri' => "$remoteurl/build/$remoteprojid/$repoid/$arch/_repository", 'proxy' => $remoteproxy, }; - my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaries); + my @binaryfilter = @binaries; + # don't filter if too many binaries are used + @binaryfilter = () if length(join(',binary=', @binaries)) > 1024; + my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaryfilter); return undef if $BSStdServer::isajax && !$binarylistcpio; $binarylist = {}; for my $b (@{$binarylistcpio->{'binary'} || []}) { Does that fix your problem? Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Mon, Jan 28, 2013 at 11:34:24AM +0100, Michael Schroeder wrote:
It already splits the download into chunks of 50 packages, but there's a request to get the state of the binaries done at the beginning of the worker_getbinaries function that may result in a too bug URL. There's an easy way to fix this: we can simply ask for all the binaries if the list is too big (untested diff):
--- bs_srcserver.orig 2013-01-21 12:12:58.000000000 +0100 +++ bs_srcserver 2013-01-28 11:31:38.049503000 +0100 @@ -6718,7 +6718,10 @@ 'uri' => "$remoteurl/build/$remoteprojid/$repoid/$arch/_repository", 'proxy' => $remoteproxy, }; - my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaries); + my @binaryfilter = @binaries; + # don't filter if too many binaries are used + @binaryfilter = () if length(join(',binary=', @binaries)) > 1024; + my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaryfilter); return undef if $BSStdServer::isajax && !$binarylistcpio; $binarylist = {}; for my $b (@{$binarylistcpio->{'binary'} || []}) {
Does that fix your problem?
Darix just mentioned to me that we could also change the method to a POST to fix it, which may be the cleaner fix. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Quoting Michael Schroeder
On Mon, Jan 28, 2013 at 11:34:24AM +0100, Michael Schroeder wrote:
Darix just mentioned to me that we could also change the method to a POST to fix it, which may be the cleaner fix.
Indeed, a POST would probably be cleaner for a 10k request (Darix said it was that big). for now I reverted the patch, as darix was also looking into some other things being filtered away, and it gave him so far some good 'test machine'. So for now, I'll just switch between patched and unpatched based on darix' needs :) Dominique -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Quoting Michael Schroeder
On Mon, Jan 28, 2013 at 10:34:30AM +0100, Dominique Leuenberger
It already splits the download into chunks of 50 packages, but there's a request to get the state of the binaries done at the beginning of the worker_getbinaries function that may result in a too bug URL. There's an easy way to fix this: we can simply ask for all the binaries if the list is too big (untested diff):
--- bs_srcserver.orig 2013-01-21 12:12:58.000000000 +0100 +++ bs_srcserver 2013-01-28 11:31:38.049503000 +0100 @@ -6718,7 +6718,10 @@ 'uri' => "$remoteurl/build/$remoteprojid/$repoid/$arch/_repository", 'proxy' => $remoteproxy, }; - my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaries); + my @binaryfilter = @binaries; + # don't filter if too many binaries are used + @binaryfilter = () if length(join(',binary=', @binaries)) > 1024; + my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaryfilter); return undef if $BSStdServer::isajax && !$binarylistcpio; $binarylist = {}; for my $b (@{$binarylistcpio->{'binary'} || []}) {
Does that fix your problem?
Michael, I applied this diff to my bs_srcserver (had a few lines offset.. but otherwise clean). Restarted src server and triggered a rebuild of the 'failed' packages... so far it looks better (build did not fail within 15 seconds)... Dominique -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (2)
-
Dominique Leuenberger a.k.a. Dimstar
-
Michael Schroeder