Mailinglist Archive: opensuse-buildservice (233 mails)

< Previous Next >
[opensuse-buildservice] Problematic mirrors causing hangs while downloading packages with osc build
  • From: "Dr. Peter Poeml" <poeml@xxxxxxx>
  • Date: Wed, 16 May 2007 18:49:09 +0200
  • Message-id: <20070516164909.GH14243@xxxxxxx>
Hi,

recently, we see hangs when osc downlaods packages during osc build. You
might have seen it as well. Ctrl-C and restarting helps in those cases.
I just debugged this a bit, and it seems that some mirrors are faulty.
-H will show which mirrors osc contacts, but it doesn't show the
individual following of redirects, so I had to dig a little deeper.

I saw a hang with this server for example:
python  1645 poeml   28u  IPv4 43107270              TCP batavia510.suse.de:45944->babbage.hrz.tu-chemnitz.de:http (ESTABLISHED)


With more debug output (in /usr/lib*/python2.4/urllib2.py), showing the
redirects, I could immediately reproduce a hang with another server:


 % ftp http://ftp.tu-chemnitz.de/pub/linux/opensuse/distribution/10.2/repo/oss/suse/i586/util-linux-2.12r-61.i586.rpm
Requesting http://ftp.tu-chemnitz.de/pub/linux/opensuse/distribution/10.2/repo/oss/suse/i586/util-linux-2.12r-61.i586.rpm
 76% |*************************************************************************************************************************************************                                               |  1194 KB   79.63 KB/s  - stalled -

Stalled since several minutes now.

(No web cache in between in my case.)

I fear, that we will need to do routine random download tests on
mirrors in the future, to test if they are really functional.......

Although it's not even clear if that would help, because once I tried
the above another time, it worked right away :(

We definitely need good debug output for this, to spot failing mirrors
better. A hack is

--- /usr/lib64/python2.4/urllib2.py     2007-05-16 18:47:58.000000000 +0200
+++ /dev/shm/urllib2.py 2007-05-16 18:47:44.987975358 +0200
@@ -526,6 +526,7 @@
         else:
             return
         newurl = urlparse.urljoin(req.get_full_url(), newurl)
+        print newurl
 
         # XXX Probably want to forget about the state of the current
         # request, although that might interact poorly with other


Peter
-- 
Allen Gewalten zum Trutz sich erhalten.
 
SUSE LINUX Products GmbH
Research & Development
< Previous Next >
Follow Ups