[opensuse-buildservice] Getting binaries through linked project
Hi, In our local OBS instance we have two linked projects. One of them produces error: "400 remote error: unexpected EOF" for any build attempt. I digged a bit into the problem and here is what I've found: With working link I can get rpms in cpio format from the repository: wget "http://my.local.obs:81/public/build/link.to.good.obs:project/distro/arch/_repository?binary=foo&view=cpio" -O- -q |cpio -t foo.rpm 121 blocks With not working link the same attempt ends with empty ouput: wget "http://my.local.obs:81/public/build/link.to.bad.obs:project/distro/arch/_repository?binary=foo&view=cpio" -O- -q On the server side I see this error in src_server.log: 2012-02-23 17:15:17 [8319]: GET /build/link.to.bad.obs:project/distro/arch/_repository?binary=foo&view=cpio 2012-02-23 17:15:17 [8221]: GET /build/ link.to.bad.obs:project/distro/arch/_repository?view=cpio&binary=foo (AJAX) rpc_recv_stream_handler: illegal chunk size: 16175 In the code of BSWatcher.pm:rpc_recv_stream_handler I can see this: if ($cl < 0 || $cl >= 16000) { print "rpc_recv_stream_handler: illegal chunk size: $cl\n"; BSServerEvents::stream_close($rev, $ev); return; } Where this limit came from? How it at all can be possible that some obs server sends chunks bigger than this limit? How would you suggest to fix this? I'm using obs server 2.1.16 if it matters. (not related to the topic) What's the point of having two almost similar requests? GET /build/link.to.bad.obs:project/distro/arch/_repository?binary=foo&view=cpio and GET /build/ link.to.bad.obs:project/distro/arch/_repository?view=cpio&binary=foo (AJAX) Regards, Ed -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Thu, Feb 23, 2012 at 06:27:50PM +0200, Ed Bartosh wrote:
On the server side I see this error in src_server.log: 2012-02-23 17:15:17 [8319]: GET /build/link.to.bad.obs:project/distro/arch/_repository?binary=foo&view=cpio 2012-02-23 17:15:17 [8221]: GET /build/ link.to.bad.obs:project/distro/arch/_repository?view=cpio&binary=foo (AJAX) rpc_recv_stream_handler: illegal chunk size: 16175
In the code of BSWatcher.pm:rpc_recv_stream_handler I can see this: if ($cl < 0 || $cl >= 16000) { print "rpc_recv_stream_handler: illegal chunk size: $cl\n"; BSServerEvents::stream_close($rev, $ev); return; }
Where this limit came from?
It was added to find some problems with the SSL code. I'm not sure if it's still needed, you might want to change the 16000 to some higher value. (Some limit is still good to have to detect really broken streams instead of using up all memory.)
How it at all can be possible that some obs server sends chunks bigger than this limit? How would you suggest to fix this?
I don't know. The code makes sure that the chunks are not much bigger than 4k. Maybe there's some proxy inbetween that reassembles chunks.
I'm using obs server 2.1.16 if it matters.
(not related to the topic) What's the point of having two almost similar requests? GET /build/link.to.bad.obs:project/distro/arch/_repository?binary=foo&view=cpio and GET /build/ link.to.bad.obs:project/distro/arch/_repository?view=cpio&binary=foo (AJAX)
That's actually the same request, it's handed over from the normal server to the server for "long running" requests (AJAX requests). Thus you see one line from the normal server and one line from the ajax server. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hi,
Where this limit came from?
It was added to find some problems with the SSL code. I'm not sure if it's still needed, you might want to change the 16000 to some higher value. (Some limit is still good to have to detect really broken streams instead of using up all memory.)
Increasing the limit doesn't help. That was the first thing I tried. It leads to long wait and result is just a couple of chunks. The rest is lost somewhere. BTW, wget handles direct request to the same url (not through the link) just fine: wget --no-check-certificate "https://api.bad.obs/public/build/:project/distro/arch/_repository?binary=fo o&view=cpio" -O- -q |cpio -t foo.rpm 121 blocks Correct me if I'm wrong, but it looks like a bug somewhere in obs code. --- Ed -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hi,
Where this limit came from?
It was added to find some problems with the SSL code. I'm not sure if it's still needed, you might want to change the 16000 to some higher value. (Some limit is still good to have to detect really broken streams instead of using up all memory.)
Increasing the limit doesn't help. That was the first thing I tried. It leads to long wait and result is just a couple of chunks. The rest is lost somewhere.
BTW, wget handles direct request to the same url (not through the link) just fine: wget --no-check-certificate "https://api.bad.obs/public/build/:project/distro/arch/_repository?bina ry=fo o&view=cpio" -O- -q |cpio -t foo.rpm 121 blocks
Correct me if I'm wrong, but it looks like a bug somewhere in obs code.
After some more digging I realized that current obs code just can't process HTTP chunks bigger than 16384 bytes. It expects the whole chunk to fit into the buffer(recvbuf) which is used to read data from a socket. However, there is no limit for HTTP chunk size in RFC 2616 and my case confirms that in reality chunks can be much bigger. Maximum I've seen is more than 200K. Here is my fix: https://github.com/openSUSE/open-build-service/pull/30 I tested it on obs 2.1.16 and it solves the problem explained in this thread. Sorry, for the ugly code. I'm not a Perl programmer by any means. Regards, Ed -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (2)
-
Ed Bartosh -
Michael Schroeder