https://bugzilla.novell.com/show_bug.cgi?id=466997 Cyril Hrubis changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|bnc-team-screening@forge.pr |poeml@novell.com |ovo.novell.com | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User cgardner@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c2 craig gardner changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Info Provider|cgardner@novell.com | --- Comment #2 from craig gardner 2009-01-20 11:31:04 MST --- -- GET http://autobuild.provo.novell.com:8081/source/home:cgardner/bar send: 'GET /source/home:cgardner/bar HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: autobuild.provo.novell.com:8081\r\nConnection: close\r\nUser-Agent: osc/0.111\r\n\r\n' reply: 'HTTP/1.1 401 Unauthorized\r\n' header: Connection: close^M header: Content-Type: application/xml; charset=utf-8^M header: WWW-Authenticate: basic realm="Frontend login"^M header: X-Runtime: 0.02469^M header: X-Opensuse-Errorcode: unknown^M header: Cache-Control: no-cache^M header: Content-Length: 140^M header: Date: Tue, 20 Jan 2009 18:19:06 GMT^M header: Server: lighttpd (1.4.13/SuSE)^M send: 'GET /source/home:cgardner/bar HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: autobuild.provo.novell.com:8081\r\nConnection: close\r\nAuthorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxx==\r\nUser-Agent: osc/0.111\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: Connection: close^M header: Content-Type: text/xml^M header: X-Opensuse-APIVersion: 0.1.1^M header: X-Runtime: 0.81264^M header: ETag: "b7d968f69cc9d79e73b100c9f4185a70"^M header: Content-Transfer-Encoding: binary^M header: Cache-Control: private^M header: Content-Disposition: inline^M header: Content-Length: 528^M header: Date: Tue, 20 Jan 2009 18:19:06 GMT^M header: Server: lighttpd (1.4.13/SuSE)^M Sending bar-0.9.39.tar.bz2 Transmitting file data . -- PUT http://autobuild.provo.novell.com:8081/source/home:cgardner/bar/bar-0.9.39.t... send: 'PUT /source/home:cgardner/bar/bar-0.9.39.tar.bz2?rev=upload HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 12161652\r\nHost: autobuild.provo.novell.com:8081\r\nContent-Type: application/octet-stream\r\nConnection: close\r\nUser-Agent: osc/0.111\r\n\r\n' send: sendIng a read()able reply: 'HTTP/1.1 401 Unauthorized\r\n' header: Connection: close^M header: Content-Type: application/xml; charset=utf-8^M header: WWW-Authenticate: basic realm="Frontend login"^M header: X-Opensuse-APIVersion: 0.1.1^M header: X-Runtime: 0.34814^M header: X-Opensuse-Errorcode: unknown^M header: Cache-Control: no-cache^M header: Content-Length: 140^M header: Date: Tue, 20 Jan 2009 18:19:08 GMT^M header: Server: lighttpd (1.4.13/SuSE)^M send: 'PUT /source/home:cgardner/bar/bar-0.9.39.tar.bz2?rev=upload HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 12161652\r\nHost: autobuild.provo.novell.com:8081\r\nUser-Agent: osc/0.111\r\nConnection: close\r\nContent-Type: application/octet-stream\r\nAuthorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxx==\r\n\r\n' send: sendIng a read()able reply: '' -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c4 --- Comment #4 from Peter Poeml 2009-01-21 11:00:04 MST --- Craig, is it correct that you see this problem only happening for clients on 11.1, and only when they commit to your build service instance (not OBS)? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User cgardner@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c6 craig gardner changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Info Provider|cgardner@novell.com | --- Comment #6 from craig gardner 2009-01-21 11:13:11 MST --- Re: comment#4 Yes, you're exactly right. When using 11.1, osc 0.111, and committing to Provo BuildService (PBS), we see this problem every time. 3 separate users have verified. And with those same 3 users, on 11.1, osc 0.111, but committing to OBS, there is no problem. Re: comment#5 The output collected for comment#2 came from the following command: osc -H ci wine-0.9.39.tar.bz2 | tee /tmp/osc_output.txt Then I edited osc_output.txt in the following way: * trimmed the svn comment data the from the top * edited AuthBasic field Yes, your question on comment#5 is exactly right: The "httplib.BadStatusLine" is reported from python -- as noted -- without using -H. The "sendIng a read()able reply" output is seen only when using -H. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c8 Peter Poeml changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jmatejek@novell.com --- Comment #8 from Peter Poeml 2009-01-21 11:37:50 MST --- Thanks for confirming on all this! Very strange that mmap'ing fails on 11.1. And that it only does so when uploading to your build service. Adding Jan, the Python maintainer to Cc. I guess we need to destillate a test case. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c10 --- Comment #10 from Peter Poeml 2009-01-21 11:59:33 MST --- what osc does is the following: <snip> if file and not data: size = os.path.getsize(file) if size < 1024*512: data = open(file).read() else: import mmap filefd = open(file, 'r') try: data = mmap.mmap(filefd.fileno(), os.path.getsize(file), mmap.MAP_SHARED, mmap.PROT_READ) except EnvironmentError, e: if e.errno == 19: sys.exit('\n\n%s\nThe file \'%s\' could not be memory mapped. It is ' \ '\non a filesystem which does not support this.' % (e, file)) else: raise if conf.config['debug']: print method, url try: fd = urllib2.urlopen(req, data=data) </snip> -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User jmatejek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c12 Jan Matejek changed: What |Removed |Added ---------------------------------------------------------------------------- Info Provider|jmatejek@novell.com |cgardner@novell.com --- Comment #12 from Jan Matejek 2009-01-22 12:47:45 MST --- the conclusion so far goes like this: a) On my 11.1-based system, mmap'ed files are read properly, as good as non-mmap'ed files. So i see no bug there. That doesn't say much, perhaps your files are different in some regard. b) You say that submitting to OBS works but PBS is broken. That means that your system -can- properly send data from mmap'ed files, at least to some URLs. c) But if it is like this exclusively for mmap + PBS + 11.1 combination, it's hard to believe that it would be only PBS's fault. Can you do a network traffic dump and see what is actually sent and received for both OBS and PBS ? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c17 --- Comment #17 from Peter Poeml 2009-01-23 06:48:52 MST --- The retry-after-401 logic is completely in urllib2.py, possibly also urllib.py (or even httplib.py). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User cgardner@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c19 --- Comment #19 from craig gardner 2009-01-23 08:14:07 MST --- Re: comment#15 , I'll try your changes, Peter. Report back as soon as I can. But more important is this: I just remembered one more detail that is likely to be involved in this problem. OBS uses iChain authentication, and runs in "production" mode. So the HTTP headers will be slightly different. OBS's http transactions may be slightly different than that of PBS, such as the inclusion of the X-username and X-HTTP_AUTHORIZATION headers. PBS runs in "development" mode, which normally would use authentication through a local db (mysql). However, I have also modified the local authentication to use eDirectory authentication using an ssl certificate, instead of authentication through the local db. The local modification to enable edir authentication will be attached to the bug in a moment. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User cgardner@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c21 --- Comment #21 from craig gardner 2009-01-23 08:46:32 MST --- Re: comment#15:

do you happen to get a timeout?

Yes, it sure feels like it. There's no timeout message, but it takes a lot longer to respond than expected, suggesting a timeout somewhere along the line.

And could you please reproduce with ....

I made the modifications to the python code, then created a small text file. # cat foobar foo Then I tried to upload the small file. I got the same failure (as expected), and I see the additional 'print' from httplib.py. In summary, here's what it printed: -- PUT http://autobuild.provo.novell.com:8081/source/home:cgardner/wine/foobar?rev=... PUT http://autobuild.provo.novell.com:8081/source/home:cgardner/wine/foobar?rev=... send: 'PUT /source/home:cgardner/wine/foobar?rev=upload HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 4\r\nHost: autobuild.provo.novell.com:8081\r\nContent-Type: application/octet-stream\r\nConnection: close\r\nUser-Agent: osc/0.111\r\n\r\n' send: sendIng a read()able 'foo\n' reply: 'HTTP/1.1 401 Unauthorized\r\n' header: Connection: close header: Content-Type: application/xml; charset=utf-8 header: WWW-Authenticate: basic realm="Frontend login" header: X-Opensuse-APIVersion: 0.1.1 header: X-Runtime: 0.01378 header: X-Opensuse-Errorcode: unknown header: Cache-Control: no-cache header: Content-Length: 140 header: Date: Fri, 23 Jan 2009 15:36:42 GMT header: Server: lighttpd (1.4.13/SuSE) send: 'PUT /source/home:cgardner/wine/foobar?rev=upload HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 4\r\nHost: autobuild.provo.novell.com:8081\r\nUser-Agent: osc/0.111\r\nConnection: close\r\nContent-Type: application/octet-stream\r\nAuthorization: Basic xxxx==\r\n\r\n' send: sendIng a read()able reply: '' -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User jmatejek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c23 Jan Matejek changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED AssignedTo|poeml@novell.com |jmatejek@novell.com --- Comment #23 from Jan Matejek 2009-01-23 09:13:24 MST --- okay, found it - it was never there in the first place py2.6 introduces support for "file-like" objects, http://svn.python.org/view/python/trunk/Lib/httplib.py?rev=52736&r1=50961&r2=52736 mmap'ed files behave like files and strings at the same time. before this patch, httplib just assumed that it's a string and all was well. now it's taken as a file, and the necessary seek(0) is completely missing. i'll file an upstream bug and try to come up with a sensible solution -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c25 --- Comment #25 from Peter Poeml 2009-01-23 16:30:34 MST --- Thanks Jan!! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c27 --- Comment #27 from Peter Poeml 2009-01-26 07:13:07 MST --- Well, your suggestion is not reallty a "fix". The problem is a regression in Python, by all that we know so far. The suggestion for a workaround notwithstanding - it is not the root of the problem. The larger implication is that the regression in Python will have the same impact on similar applications, and this is what needs to be fixed. Of course, a workaround in osc to change behaviour when running on Python could be a worthwhile thought. However, you are probably not aware of the fact that the mmap is there for a reason. It avoids the need to read the entire file to be sent into memory. It is not a good idea to read huge files into memory, because memory on most computer systems is quite limited. Thus, a better workaround is needed. I wondered if passing a file object might be a better idea than passing a mmap object. But I think it would fail in the same way. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User jmatejek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c29 --- Comment #29 from Jan Matejek 2009-01-26 13:36:47 MST --- well it's a regression by definition, but ... an unclear one ;e) anyway, while python is "broken" in this, i propose a workaround: wrap the mmap instance in a class that doesn't have read() method. i'll send you the code tomorrow or something. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User jmatejek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c30 --- Comment #30 from Jan Matejek 2009-01-30 03:29:49 MST --- Created an attachment (id=268856) --> (https://bugzilla.novell.com/attachment.cgi?id=268856) workaround inside osc there. the class stringonly behaves like a string on the outside, so the "sendIng a read()able" won't happen and old behavior is restored. in other news, all quiet on the python front :e/ -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User srinidhi@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c32 Srinidhi B S changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |srinidhi@novell.com --- Comment #32 from Srinidhi B S 2009-03-09 10:34:56 MST --- (In reply to comment #30)

Created an attachment (id=268856) --> (https://bugzilla.novell.com/attachment.cgi?id=268856) [details] workaround inside osc

there. the class stringonly behaves like a string on the outside, so the "sendIng a read()able" won't happen and old behavior is restored.

in other news, all quiet on the python front :e/

This patch doesn't work for me on openSUSE 11.1 64bit: $ python --version Python 2.6 $ uname -a Linux blrsrinidhi 2.6.27.19-3.2-default #1 SMP 2009-02-25 15:40:44 +0100 x86_64 x86_64 x86_64 GNU/Linux I get this when I run "osc ci": Traceback (most recent call last): File "/usr/bin/osc", line 12, in <module> r = babysitter.run(osccli) File "/usr/lib64/python2.6/site-packages/osc/babysitter.py", line 40, in run return prg.main() File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 256, in main return self.cmd(args) File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 279, in cmd retval = self.onecmd(argv) File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 395, in onecmd return self._dispatch_cmd(handler, argv) File "/usr/lib64/python2.6/site-packages/osc/cmdln.py", line 1070, in _dispatch_cmd return handler(argv[0], opts, *args) File "/usr/lib64/python2.6/site-packages/osc/commandline.py", line 1355, in do_commit p.commit(msg) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 751, in commit self.put_source_file(filename) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 705, in put_source_file http_PUT(u, file = os.path.join(self.dir, n)) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 1503, in http_PUT def http_PUT(*args, **kwargs): return http_request('PUT', *args, **kwargs) File "/usr/lib64/python2.6/site-packages/osc/core.py", line 1491, in http_request fd = urllib2.urlopen(req, data=data) File "/usr/lib64/python2.6/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/usr/lib64/python2.6/urllib2.py", line 389, in open response = self._open(req, data) File "/usr/lib64/python2.6/urllib2.py", line 407, in _open '_open', req) File "/usr/lib64/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py", line 1139, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib64/python2.6/urllib2.py", line 1111, in do_open h.request(req.get_method(), req.get_selector(), req.data, headers) File "/usr/lib64/python2.6/httplib.py", line 894, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.6/httplib.py", line 934, in _send_request self.send(body) File "/usr/lib64/python2.6/httplib.py", line 739, in send self.sock.sendall(str) File "<string>", line 1, in sendall TypeError: sendall() argument 1 must be string or buffer, not instance -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User jmatejek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c34 --- Comment #34 from Jan Matejek 2009-03-10 11:51:41 MST --- you are right, i didn't realize that you can construct a buffer over mmap()ed file that's obviously a better solution. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=466997 User suse-tux@gmx.de added comment https://bugzilla.novell.com/show_bug.cgi?id=466997#c36 --- Comment #36 from Marcus Hüwe 2009-03-10 12:38:03 MST --- Ok. I just committed the fix (r6783). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.