Feature changed by: Ján Kupec (jkupec) Feature #305320, revision 28 Title: Show accessed mirror(s) openSUSE-11.1: Rejected by Stanislav Visnovsky (visnov) reject date: 2008-10-01 15:49:39 reject reason: Too late, but let's consider for the future ones. Priority Requester: Mandatory openSUSE-11.2: Candidate Priority Requester: Mandatory Projectmanager: Mandatory Requested by: Peter Poeml (poeml) Description: This concerns both YaST and zypper. It would help a lot if they would show the mirror which is accessed when downloading a package. (Technically, the download client is following redirects and the actually URL needs to be shown, not just the first URL that was contacted but that redirected to another URL) It is a long-standing problem that this information is lacking and it makes debugging and optimizing much harder than necessary, because this information is only available deeply buried in some logfile. We need this information for two reasons: 1) Debugging - to see which URL was accessed when something goes wrong, like a mirror timing out. For example, Whenever a mirror returns garbage, it takes a lot of additional time to find out which one, because all reports are lacking the critical detail, and it is usually hard to find it out later. 2) Optimizing the load sharing between different mirrors - to distinguish slow mirrors from fast ones. For example, I am very much in need of information about the situation in the US, but even though people in the US run installations for me and are willing to send me logs, I can't get this information. In reply to a question from Stano:
Do you want the information in the log or to be shown in e.g. zypper -vvv ? Shown, in normal output. It is already in some log I believe, I don't know which one exactly anymore though. Thing is, users won't be able to tell us anything useful if they weren't shown the information. And later it is not reproducible. At least in a failure situation the URLs must be shown. That alone wouldn't give us the chance to find out about slow mirrors, but at least it would allow to debug problems. Real life example #1: If a mirror is behind a firewall that lets every 10th connection stall (so the transfer hangs or the connection is reset), it usually takes weeks (!) to find out about it. But it hits virtually every user from mirrors country during the time. We have had this 4 times during the last 18 months. Real life example #2: I got feedback from someone who did a network install of openSUSE 11.0 from download.opensuse.org, located in the US. He had to let it running overnight because it took a long time. Next morning, the install hadn't even completed. YaST was hanging and waiting for somebody to hit retry. Problem is, it took us a lot of work to find out which mirror was at fault. We actually had to repeat the install, I had to follow the log and live watching his machine accessing download.opensuse.org, to see where it was redirected, and where it spent most time (while downloading some package). We found a mirror which was really slow, and I could disable it. I have a suspicion though, that we have more such mirrors, because I have got some similar reports from recently. But how to find out? People only complain about slowness, without the ability to give any useful detail. See http://news.opensuse.org/2008/09/23/upcoming-factory-changes/#comments (http://news.opensuse.org/2008/09/23/upcoming-factory-changes/#comments) for examples of this.
Discussion: #1: Susanne Oberhauser (froh) (2008-10-01 15:16:30) What if we focussed on something like torrent for distributing packages instead? AFAICT these p2p network technologies have mechanisms to recover from missing fragments, i.e. failing servers. In that world, todays mirrors would be strong and almost always present peers in the network. #2: Peter Poeml (poeml) (2008-10-01 15:30:56) (reply to #1) Torrents are not efficient for files of this small size. Another concern is that, on large scale, the reasonability of P2P traffic with regard to network topology is questionable; confer to what's called the Application-Layer Traffic Optimization (ALTO) problem. See http://alto.tilab.com/ (http://alto.tilab.com/) . There are working examples where e.g. university have rolled out a P2P update distribution campus-wide; in a controlled limited environment this is feasible, but not across the Internet. Further, note that libzypp in 11.1 has preliminary support for using metalinks for downlaods. Metalink clients recover nicely from missing fragments. See http://en.opensuse.org/Libzypp/Failover (http://en.opensuse.org/Libzypp/Failover) . #3: Mark Goldstein (markgo) (2008-10-04 17:20:42) Other usage of this feature would be possibility of "blacklisting" servers with persistent problems. (See thread in opensuse list "Is it possible to "blacklist" specific download server in Yast?" ). As Carlos E. R. suggested other useful option could be for YaST/zypper to fail gracefully requesting another server to the redirector on failure on the one it gave use previously. #4: Peter Poeml (poeml) (2008-10-04 17:55:14) (reply to #3) Gracefully falling back to other mirrors is already being implemented (currently in prototype stage) by way of http://en.opensuse.org/Libzypp/Failover (http://en.opensuse.org/Libzypp/Failover) . The feature of blacklisting a specific mirrors is included in that proposal, and I have added it explicitely. #5: Gerald Pfeifer (geraldpfeifer) (2008-10-30 17:33:25) Looking at the discussion on opensuse-factory and my own experience, I also see this as high priority for 11.2. #6: Peter Poeml (poeml) (2009-01-27 20:39:59) Just for the record, the way to enable logging that makes mirrors show up in the log seems to be: $ ZYPP_MEDIA_CURL_DEBUG=1 zypper ... Cf. http://en.opensuse.org/Debugging_YaST #7: Ján Kupec (jkupec) (2009-01-28 12:25:39) (reply to #6) Yes, it is, but that's quite excessive log that we don't want to be enabled by default. Instead we need to pick up the redirects and log them one line per each. Michal Marek already gave me a hint how to do this: "Use CURLOPT_HEADERFUNCTION callback, which gets each http header and then test for 'Location:' and log that." I'll try to do it ASAP. #8: Jiri Srain (jsrain) (2009-06-02 11:02:23) (reply to #7) Let's try to display this information at least with zypper. However, we will need some investigation, since we switched to aria2c. #9: Peter Poeml (poeml) (2009-06-24 15:04:21) Another example of the implications. What users get right now is this (http://www.3111skyline.com/download/openSUSE_bugs/kde4/screenshots/YastsZypp...) (screenshot that shows a rather meaningless URL, but not the accessed mirror). See this thread (http://lists.opensuse.org/opensuse/2009-06/msg01206.html) on the English opensuse mailing list. #10: Ján Kupec (jkupec) (2009-07-14 17:54:07) (reply to #9) OK, this is all about the CURL backend, so we're developing an improvment to be released as an online update for code 10 products (openSUSE 11.1 and SLE11), right? Peter, is this needed also for the aria2c backend? If so, can you give me some hints how to do it with aria (i have zero knowlege of aria)? #11: Ján Kupec (jkupec) (2009-07-14 18:53:11) (reply to #10) above, i meant code 11, of course + #12: Ján Kupec (jkupec) (2009-07-15 14:16:05) + Alright, logging is easy. If a Location: line will be found in reponse + header, it will be logged like: "MediaCurl.cc(log_redirects_curl):120 + redirecting to Location: URL" in zypper.log. I have the patch ready and + it is pretty simple and can be released as update for openSUSE 11.x and + SLE11. + With showing the redirects in the UIs it is quite complicated since it + requires MediaException API changes and adding the list of redirects to + each MediaException thrown. Nothing for the released products i'd say. + Can we agree on this? Let me empasize that this is for the cURL backend + used in code11 products. For openSUSE 11.2 and later (using aria2c + backend) i'm still waiting for info asked in c#10. -- openSUSE Feature: https://features.opensuse.org/305320