[Bug 298378] New: osc fails when $https_proxy is set
https://bugzilla.novell.com/show_bug.cgi?id=298378 Summary: osc fails when $https_proxy is set Product: openSUSE.org Version: unspecified Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: BuildService AssignedTo: poeml@novell.com ReportedBy: adrian@novell.com QAContact: adrian@novell.com CC: werner@novell.com Found By: --- osc failed with a python trace (no nice error output) when the $https_proxy variable is set. unsetting it solved the problem for werner. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c1 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO Info Provider| |werner@novell.com --- Comment #1 from Peter Poeml <poeml@novell.com> 2007-08-08 06:28:43 MST --- Which version of openSUSE was this observed on? This is a bug in python, I think, which has already been fixed (bug 214983). It should not be reproducible on Factory or 10.3 Alphas. root@cherry ~ # python -c "import urllib2; urllib2.urlopen('https://www.sf.net').read()" root@cherry ~ # https_proxy=http://localhost:3128 python -c "import urllib2; urllib2.urlopen('https://www.sf.net').read()" /usr/lib/python2.5/urllib2.py:662: RuntimeWarning: urllib can't handle https proxies, your https_proxy setting will not work proxies = getproxies() root@cherry ~ # Can you confirm that? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c2 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jmatejek@novell.com --- Comment #2 from Peter Poeml <poeml@novell.com> 2007-08-08 06:31:33 MST --- I could add a workaround anyway, which always ignores https_proxy, since it doesn't work anyway, respectively isn't used. So, the error could be avoided on installations older than 10.3, and the warning message avoided on 10.3 and newer since the user has no use in the warning anyway (only the programmer would have). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c3 --- Comment #3 from Peter Poeml <poeml@novell.com> 2007-08-08 07:23:36 MST --- I'll add the following patch to work around, so the error will be gone: Index: osc/conf.py =================================================================== --- osc/conf.py (revision 1941) +++ osc/conf.py (working copy) @@ -122,6 +122,15 @@ global cookiejar + # HTTPS proxy is not supported by urllib2. It only leads to an error + # or, at best, a warning. + # https://bugzilla.novell.com/show_bug.cgi?id=214983 + # https://bugzilla.novell.com/show_bug.cgi?id=298378 + if 'https_proxy' in os.environ: + del os.environ['https_proxy'] + if 'HTTPS_PROXY' in os.environ: + del os.environ['HTTPS_PROXY'] + if config['http_debug']: # brute force def urllib2_debug_init(self, debuglevel=0): -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c4 --- Comment #4 from Jan Matejek <jmatejek@novell.com> 2007-08-08 07:39:06 MST --- (In reply to comment #2 from Peter Poeml)
I could add a workaround anyway, which always ignores https_proxy, since it doesn't work anyway, respectively isn't used. So, the error could be avoided on installations older than 10.3, and the warning message avoided on 10.3 and newer since the user has no use in the warning anyway (only the programmer would have).
actually, i think the warning is there for the user; suppose that somebody can't reach buildservice without a proxy, the warning tells him "sorry, that will not work", while the workaround will fail silently just my 2c -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c5 --- Comment #5 from Peter Poeml <poeml@novell.com> 2007-08-08 07:50:27 MST --- The problem is, the user has no way of fixing it, if he really needs to use a HTTPS proxy. Note that even though python 2.5 issues a warning, it still tries to access the network without proxy. And there may be situations where https_proxy is set, but not really needed at all (like in the case reported here -- we don't restrict access to port 443 at all, you see). I believe the latter case is the majority of cases. That's why. If port 443 is really not reachable directly, does it really make a big difference in your opinion, if python tells me that "urllib can't handle https proxies" and gives a timeout (or network unreachable), or only gives a timeout (or network unreachable)? Most people use older python versions anyway, so far. Note that I did never get a report that using osc via HTTPS proxy does not work. Only this report about the error message! I doubt that people want to see the warning message all the time. Although it may eventually speed up fixing urllib2 to get this missing feature ;) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c6 --- Comment #6 from Jan Matejek <jmatejek@novell.com> 2007-08-08 08:01:24 MST --- i do think that it makes a difference when port 443 is inaccessible. but you are right, the warning appears even when https proxy is not needed, and if that's the majority of cases, it is probably better to not show it. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c7 Dr. Werner Fink <werner@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Info Provider|werner@novell.com | --- Comment #7 from Dr. Werner Fink <werner@novell.com> 2007-08-08 08:04:15 MST --- This was observed on openSuSE 10.2 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378#c8 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED --- Comment #8 from Peter Poeml <poeml@novell.com> 2007-08-08 10:40:44 MST --- Okay, thanks. The fix is in svn, and will end up in the package shortly. Thus, after upate from http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_10.2/ the error should be gone. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User froh@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c9 Susanne Oberhauser <froh@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.wilck@fujitsu-siemens.com, | |froh@novell.com Severity|Normal |Critical Status|RESOLVED |REOPENED Resolution|FIXED | --- Comment #9 from Susanne Oberhauser <froh@novell.com> 2008-02-05 03:41:58 MST --- Martin from Fujitsu Siemens computers is testing the build server and has a proxy setup to access the internet. Corporate policy. Don't ask. Now the lack of https proxy support in urllib2 is biting him hard. There seem to be workarounds for this, even with redirection support, so even if it's not fixed in urllib2, it could be fixed in osc: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/456195 This is a critical issue for some corporate environments to reasonably use osc. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c10 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Critical |Major --- Comment #10 from Peter Poeml <poeml@novell.com> 2008-02-06 08:44:21 MST --- The issue can probably be worked around by creating a TCP tunnel which goes over HTTP CONNECT. However that is going to be tricky because of name-based virtual hosting ('localhost' will not mean anything to the origin server). Here's a report of somebody who went fully through it, and thus worked around the issue for osc with success: http://pymachine.blogspot.com/2007/06/opensuse-build-service-access-through.... which refers to a posting on opensuse-buildservice here: http://lists.opensuse.org/opensuse-buildservice/2007-04/msg00018.html Regarding a fix, the ASPN snippet might be possible to re-use, if it plays together with the other opener parts we already use (cookies and auth, plus the standard redirect followers) According to http://bugs.python.org/issue?%40search_text=https+urllib2+proxy&title=&%40columns=title&id=&%40columns=id&creation=&creator=&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&severity=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&%40pagesize=50&%40startwith=0&%40action=search which Susanne dug out, it doesn't look as if there's enough interest in a fix for anyone to work on it, or rather it seems to be pretty difficult to cook something up which is "good enough" for inclusion by upstream. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User martin.wilck@fujitsu-siemens.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c11 --- Comment #11 from Martin Wilck <martin.wilck@fujitsu-siemens.com> 2008-02-06 10:50:18 MST --- (In reply to comment #10 from Peter Poeml)
Here's a report of somebody who went fully through it, and thus worked around the issue for osc with success:
Sorry, ssh tunnel doesn't work here, either.
According to http://bugs.python.org/issue?%40search_text=https+urllib2+proxy&title=&%40columns=title&id=&%40columns=id&creation=&creator=&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&severity=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&%40pagesize=50&%40startwith=0&%40action=search which Susanne dug out, it doesn't look as if there's enough interest in a fix for anyone to work on it, or rather it seems to be pretty difficult to cook something up which is "good enough" for inclusion by upstream.
Hard for me to grasp that such an important funtionality is simply dismissed as a wishlist feature request. perhaps the "recipe" of http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/456195 (referenced in http://bugs.python.org/issue1424152) might work? Could that be intergrated somehow? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User martin.wilck@fujitsu-siemens.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c12 --- Comment #12 from Martin Wilck <martin.wilck@fujitsu-siemens.com> 2008-02-06 11:31:26 MST --- (In reply to comment #11 from Martin Wilck)
perhaps the "recipe" of http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/456195 (referenced in http://bugs.python.org/issue1424152) might work? Could that be intergrated somehow?
Just tried - it doesn't work with python 2.5.1. Supposed to work with 2.4, though. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User martin.wilck@fujitsu-siemens.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c13 --- Comment #13 from Martin Wilck <martin.wilck@fujitsu-siemens.com> 2008-02-06 11:36:19 MST --- The reason it fails is that the HTTPConnection.request() method isn't passed the request itself (with the proxy host and port info). No easy solution to be seen. If you ask me, urllib2 is massively overdesigned OO code - so overdesigned that it seems to be impossible to add something as boring and common as a HTTP connect proxy with simple means. I am unimpressed. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c16 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |NEEDINFO Info Provider| |martin.wilck@fujitsu-siemens.com --- Comment #16 from Peter Poeml <poeml@novell.com> 2008-02-19 05:59:55 MST --- So the only way I see is to implement a workaround in osc, which works for different python versions. I have no idea when I'll have time to do that... realistically, not anytime soon. Or provide instructions to work around... Martin, have you figured something out for yourself? How do you currently deal with the situation? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User martin.wilck@fujitsu-siemens.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c17 Martin Wilck <martin.wilck@fujitsu-siemens.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |REOPENED Info Provider|martin.wilck@fujitsu-siemens.com | --- Comment #17 from Martin Wilck <martin.wilck@fujitsu-siemens.com> 2008-02-19 06:45:18 MST --- (In reply to comment #16 from Peter Poeml)
How do you currently deal with the situation?
I don't use the build service :-/ -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c18 --- Comment #18 from Peter Poeml <poeml@novell.com> 2008-03-05 16:35:53 MST --- libcurl might actually be an option. But so far I wanted to avoid it, because it is quite low-level and different to use. And it would be an added dependency. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User martin.wilck@fujitsu-siemens.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c19 --- Comment #19 from Martin Wilck <martin.wilck@fujitsu-siemens.com> 2008-03-06 03:55:02 MST --- I think we should try to get urllib2 fixed. I have just added some comments in http://bugs.python.org/issue1424152. This is the first time I have used the python issue system, so I don't know what's going to happen. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User mmarek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c20 --- Comment #20 from Michal Marek <mmarek@novell.com> 2008-03-06 11:01:55 MST --- (In reply to comment #18 from Peter Poeml)
libcurl might actually be an option. But so far I wanted to avoid it, because it is quite low-level and different to use. And it would be an added dependency.
AFAIR the pycurl binding requires the newest libcurl version, so it would require users to install pycurl and update libcurl. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c21 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |ASSIGNED --- Comment #21 from Peter Poeml <poeml@novell.com> 2008-03-31 04:15:38 MST --- Yes, thanks for reminding about that. In fact, that was one of the reasons why I didn't use cURL for osc. Otherwise it could have been a good choice, however additional dependencies can be quite painful, especially in the libcurl case, because it seems to require an update all too often. Therefore the implementation in Python. I'll add myself to the python issue and watch the outcome. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c22 --- Comment #22 from Peter Poeml <poeml@novell.com> 2008-07-17 11:20:23 MDT --- There is some progress on a patch in the python bug tracker, and some people talk about integration. That's about it for now. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User jmatejek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c23 --- Comment #23 from Jan Matejek <jmatejek@novell.com> 2008-09-19 12:57:15 MDT --- [prof. Farnsworth's voice] "Good news everyone!" the patch from upstream's bugtracker seems to work just fine, i'm including it into our python. please test! .. one-click install (Factory): http://software.opensuse.org/ymp/devel:languages:python:Factory/openSUSE_Fac... files: http://download.opensuse.org/repositories/devel:/languages:/python:/Factory/ buildservice project: https://build.opensuse.org/project/show?project=devel:languages:python:Facto... -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=298378#c24 --- Comment #24 from Peter Poeml <poeml@novell.com> 2008-09-19 13:07:40 MDT --- That's great news! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=298378 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 - None |P2 - High -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com