On Mon, Jan 28, 2013 at 10:34:30AM +0100, Dominique Leuenberger a.k.a. Dimstar wrote:

I have a small issue on my private OBS instance, which remote links to build.o.o.

I see some of my packages in status failed (even though they used to work / build just fine without code changes). osc bl shows me:

400 remote error: remote error Bad request

A little bit of digging in src_server.log, this error is apparently an error from a too long list of remote sources supposed to be downloaded.

The matching line in src_server.log: 2013-01-28 06:36:25 [9698]: GET /getbinaries?project=openSUSE.org:SUSE:SLE-11:SP2&repository=standard&arch=x86_64&binaries=aaa_base,acl,*many more packages listed*,perl-TimeDate,rpmlint-mini (AJAX) 400 remote error: Bad request

A very quick check shows that this get is somewhere between 4 and 5K in size.. so not really surprising that this causes issues..

When using osc build, there is the option --disable-cpio-bulk-download to work around the exact same issue. Is there a way to set an OBS instance to do the same? It clearly can not handle such a large chunk and as such should have some logic to not send such huge requests (maybe split it in multiple requests of only 1k each?)

My OBS version: obs-api-2.3.91_736_gfc3358d-1.1.x86_64 obs-server-2.3.91_736_gfc3358d-1.1.x86_64

It already splits the download into chunks of 50 packages, but there's a request to get the state of the binaries done at the beginning of the worker_getbinaries function that may result in a too bug URL. There's an easy way to fix this: we can simply ask for all the binaries if the list is too big (untested diff): --- bs_srcserver.orig 2013-01-21 12:12:58.000000000 +0100 +++ bs_srcserver 2013-01-28 11:31:38.049503000 +0100 @@ -6718,7 +6718,10 @@ 'uri' => "$remoteurl/build/$remoteprojid/$repoid/$arch/_repository", 'proxy' => $remoteproxy, }; - my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaries); + my @binaryfilter = @binaries; + # don't filter if too many binaries are used + @binaryfilter = () if length(join(',binary=', @binaries)) > 1024; + my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaryfilter); return undef if $BSStdServer::isajax && !$binarylistcpio; $binarylist = {}; for my $b (@{$binarylistcpio->{'binary'} || []}) { Does that fix your problem? Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Quoting Michael Schroeder :

On Mon, Jan 28, 2013 at 10:34:30AM +0100, Dominique Leuenberger

It already splits the download into chunks of 50 packages, but there's a request to get the state of the binaries done at the beginning of the worker_getbinaries function that may result in a too bug URL. There's an easy way to fix this: we can simply ask for all the binaries if the list is too big (untested diff):

--- bs_srcserver.orig 2013-01-21 12:12:58.000000000 +0100 +++ bs_srcserver 2013-01-28 11:31:38.049503000 +0100 @@ -6718,7 +6718,10 @@ 'uri' => "$remoteurl/build/$remoteprojid/$repoid/$arch/_repository", 'proxy' => $remoteproxy, }; - my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaries); + my @binaryfilter = @binaries; + # don't filter if too many binaries are used + @binaryfilter = () if length(join(',binary=', @binaries)) > 1024; + my $binarylistcpio = BSWatcher::rpc($param, $BSXML::binarylist, "view=names", map {"binary=$_"} @binaryfilter); return undef if $BSStdServer::isajax && !$binarylistcpio; $binarylist = {}; for my $b (@{$binarylistcpio->{'binary'} || []}) {

Does that fix your problem?

Michael, I applied this diff to my bs_srcserver (had a few lines offset.. but otherwise clean). Restarted src server and triggered a rebuild of the 'failed' packages... so far it looks better (build did not fail within 15 seconds)... Dominique -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org