[Bug 466997] New: "osc ci" fails uploading large files to Provo BuildService
https://bugzilla.novell.com/show_bug.cgi?id=466997 Summary: "osc ci" fails uploading large files to Provo BuildService Classification: openSUSE Product: openSUSE.org Version: unspecified Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: BuildService AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: cgardner@novell.com QAContact: adrian@novell.com CC: ajorgensen@novell.com, sshaw@novell.com, bwhiteley@novell.com Found By: --- osc clients on opensuse 11.1 can't upload certain files to Provo BuildService (PBS). The same client can successfully upload to OBS, no problems. The problem seems to be isolated to PBS. PBS is running buildservice svn revision 5959+ (1.5+), on SLES10, i586. Uploads of specfiles don't fail. Uploads of things a bit larger fail. The problem only occurs using osc. If I upload the same file using the webclient, there is no problem. I discussed this with darix and Peter Poeml back on 12 Dec and 17 Dec 2008, but no more discussion and debugging has happened since then. To properly track this and work on a fix, now filing the bug report. Back on 17 Dec, Peter seemed to think that the problem was with the variable way that osc uploads files. For smaller files it uses one method, but for larger files it uses a different method. Maybe something to do with python 2.6. Darix looked at PBS's lighttpd configuration, and he reported that the problem does not appear to be related to lighttpd. # ls -l foo-0.2.0.tar.gz -rw-r--r-- 1 root root 53819 2009-01-16 11:35 foo-0.2.0.tar.gz # osc -A http://autobuild.provo.novell.com:8081 ci foo-0.2.0.tar.gz Sending foo-0.2.0.tar.gz Transmitting file data . Committed revision 4. # ls -l bar-0.9.39.tar.bz2 -rw-r--r-- 1 root root 12161652 2009-01-16 11:27 bar-0.9.39.tar.bz2 # osc -A http://autobuild.provo.novell.com:8081 ci bar-0.9.39.tar.bz2 Sending bar-0.9.39.tar.bz2 Transmitting file data .Traceback (most recent call last): File "/usr/bin/osc", line 12, in <module> r = babysitter.run(osccli) File "/usr/lib/python2.6/site-packages/osc/babysitter.py", line 40, in run return prg.main() File "/usr/lib/python2.6/site-packages/osc/cmdln.py", line 256, in main return self.cmd(args) File "/usr/lib/python2.6/site-packages/osc/cmdln.py", line 279, in cmd retval = self.onecmd(argv) File "/usr/lib/python2.6/site-packages/osc/cmdln.py", line 395, in onecmd return self._dispatch_cmd(handler, argv) File "/usr/lib/python2.6/site-packages/osc/cmdln.py", line 1070, in _dispatch_cmd return handler(argv[0], opts, *args) File "/usr/lib/python2.6/site-packages/osc/commandline.py", line 1337, in do_commit p.commit(msg) File "/usr/lib/python2.6/site-packages/osc/core.py", line 751, in commit self.put_source_file(filename) File "/usr/lib/python2.6/site-packages/osc/core.py", line 705, in put_source_file http_PUT(u, file = os.path.join(self.dir, n)) File "/usr/lib/python2.6/site-packages/osc/core.py", line 1491, in http_PUT def http_PUT(*args, **kwargs): return http_request('PUT', *args, **kwargs) File "/usr/lib/python2.6/site-packages/osc/core.py", line 1475, in http_request fd = urllib2.urlopen(req, data=data) File "/usr/lib64/python2.6/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/usr/lib64/python2.6/urllib2.py", line 395, in open response = meth(req, response) File "/usr/lib64/python2.6/urllib2.py", line 508, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib64/python2.6/urllib2.py", line 427, in error result = self._call_chain(*args) File "/usr/lib64/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py", line 847, in http_error_401 url, req, headers) File "/usr/lib64/python2.6/urllib2.py", line 825, in http_error_auth_reqed return self.retry_http_basic_auth(host, req, realm) File "/usr/lib64/python2.6/urllib2.py", line 835, in retry_http_basic_auth return self.parent.open(req) File "/usr/lib64/python2.6/urllib2.py", line 389, in open response = self._open(req, data) File "/usr/lib64/python2.6/urllib2.py", line 407, in _open '_open', req) File "/usr/lib64/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py", line 1139, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib64/python2.6/urllib2.py", line 1112, in do_open r = h.getresponse() File "/usr/lib64/python2.6/httplib.py", line 970, in getresponse response.begin() File "/usr/lib64/python2.6/httplib.py", line 390, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.6/httplib.py", line 354, in _read_status raise BadStatusLine(line) httplib.BadStatusLine -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=466997
Cyril Hrubis
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c1
Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User cgardner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c2
craig gardner
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c3
--- Comment #3 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c4
--- Comment #4 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c5
Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User cgardner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c6
craig gardner
https://bugzilla.novell.com/show_bug.cgi?id=466997
User cgardner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c7
--- Comment #7 from craig gardner
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c8
Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c9
Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c10
--- Comment #10 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c11
--- Comment #11 from Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c12
Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c14
--- Comment #14 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c15
--- Comment #15 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c16
--- Comment #16 from Jan Matejek
Maybe the authentication works different with the Provo bs, and osc gets a "Unauthorized" after the PUT. It could be possible that osc retries the request then, and the offset in the mmap'ed file is at the end then. The next attempt to read() would give empty data.
that makes a lot of sense. so, where is the retry-after-401 logic, that should seek to the beginnig of the file? Peter, is it in osc, or should i dig into urllib to find it? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c17
--- Comment #17 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c18
--- Comment #18 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User cgardner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c19
--- Comment #19 from craig gardner
https://bugzilla.novell.com/show_bug.cgi?id=466997
User cgardner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c20
--- Comment #20 from craig gardner
https://bugzilla.novell.com/show_bug.cgi?id=466997
User cgardner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c21
--- Comment #21 from craig gardner
do you happen to get a timeout?
Yes, it sure feels like it. There's no timeout message, but it takes a lot longer to respond than expected, suggesting a timeout somewhere along the line.
And could you please reproduce with ....
I made the modifications to the python code, then created a small text file.
# cat foobar
foo
Then I tried to upload the small file. I got the same failure (as expected),
and I see the additional 'print' from httplib.py. In summary, here's what it
printed:
-- PUT
http://autobuild.provo.novell.com:8081/source/home:cgardner/wine/foobar?rev=...
PUT
http://autobuild.provo.novell.com:8081/source/home:cgardner/wine/foobar?rev=...
send: 'PUT /source/home:cgardner/wine/foobar?rev=upload
HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 4\r\nHost:
autobuild.provo.novell.com:8081\r\nContent-Type:
application/octet-stream\r\nConnection: close\r\nUser-Agent: osc/0.111\r\n\r\n'
send:
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c22
--- Comment #22 from Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c23
Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c24
--- Comment #24 from Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c25
--- Comment #25 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jnelson-suse@jamponi.net added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c26
Jon Nelson
https://bugzilla.novell.com/show_bug.cgi?id=466997
User poeml@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c27
--- Comment #27 from Peter Poeml
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jnelson-suse@jamponi.net added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c28
--- Comment #28 from Jon Nelson
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c29
--- Comment #29 from Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c30
--- Comment #30 from Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jnelson-suse@jamponi.net added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c31
--- Comment #31 from Jon Nelson
https://bugzilla.novell.com/show_bug.cgi?id=466997
User srinidhi@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c32
Srinidhi B S
Created an attachment (id=268856) --> (https://bugzilla.novell.com/attachment.cgi?id=268856) [details] workaround inside osc
there. the class stringonly behaves like a string on the outside, so the "sendIng a read()able" won't happen and old behavior is restored.
in other news, all quiet on the python front :e/
This patch doesn't work for me on openSUSE 11.1 64bit: $ python --version Python 2.6 $ uname -a Linux blrsrinidhi 2.6.27.19-3.2-default #1 SMP 2009-02-25 15:40:44 +0100 x86_64 x86_64 x86_64 GNU/Linux I get this when I run "osc ci": Traceback (most recent call last): File "/usr/bin/osc", line 12, in <module> r = babysitter.run(osccli) File "/usr/lib64/python2.6/site-packages/osc/babysitter.py", line 40, in run return prg.main() File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 256, in main return self.cmd(args) File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 279, in cmd retval = self.onecmd(argv) File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 395, in onecmd return self._dispatch_cmd(handler, argv) File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 1070, in _dispatch_cmd return handler(argv[0], opts, *args) File "/usr/lib64/python2.6/site-packages/osc/commandline.py", line 1355, in do_commit p.commit(msg) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 751, in commit self.put_source_file(filename) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 705, in put_source_file http_PUT(u, file = os.path.join(self.dir, n)) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 1503, in http_PUT def http_PUT(*args, **kwargs): return http_request('PUT', *args, **kwargs) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 1491, in http_request fd = urllib2.urlopen(req, data=data) File "/usr/lib64/python2.6/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/usr/lib64/python2.6/urllib2.py", line 389, in open response = self._open(req, data) File "/usr/lib64/python2.6/urllib2.py", line 407, in _open '_open', req) File "/usr/lib64/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py", line 1139, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib64/python2.6/urllib2.py", line 1111, in do_open h.request(req.get_method(), req.get_selector(), req.data, headers) File "/usr/lib64/python2.6/httplib.py", line 894, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.6/httplib.py", line 934, in _send_request self.send(body) File "/usr/lib64/python2.6/httplib.py", line 739, in send self.sock.sendall(str) File "<string>", line 1, in sendall TypeError: sendall() argument 1 must be string or buffer, not instance -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=466997
User suse-tux@gmx.de added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c33
--- Comment #33 from Marcus Hüwe
there. the class stringonly behaves like a string on the outside, so the "sendIng a read()able" won't happen and old behavior is restored.
Wouldn't it be easier to get the the sequence representation of the mmap object via the buffer() built-in function? Instead of data = stringonly(data) we would use data = buffer(data) which creates a reference to the mmap object's sequence representation. If we pass "data" to urllib2/httplib it'll behave like a buffer type and everything will be fine. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=466997
User jmatejek@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c34
--- Comment #34 from Jan Matejek
https://bugzilla.novell.com/show_bug.cgi?id=466997
User srinidhi@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c35
--- Comment #35 from Srinidhi B S
(In reply to comment #30)
there. the class stringonly behaves like a string on the outside, so the "sendIng a read()able" won't happen and old behavior is restored.
Wouldn't it be easier to get the the sequence representation of the mmap object via the buffer() built-in function? Instead of
data = stringonly(data)
we would use
data = buffer(data)
which creates a reference to the mmap object's sequence representation. If we pass "data" to urllib2/httplib it'll behave like a buffer type and everything will be fine.
This worked perfectly for me! Thanks a lot Marcus! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=466997
User suse-tux@gmx.de added comment
https://bugzilla.novell.com/show_bug.cgi?id=466997#c36
--- Comment #36 from Marcus Hüwe
participants (1)
-
bugzilla_noreply@novell.com