Feature changed by: Mark Goldstein
Feature #305320, revision 6
Title: Show accessed mirror(s)
openSUSE-11.1: Rejected by Stanislav Visnovsky
reject date: 2008-10-01 15:49:39
reject reason: Too late, but let's consider for the future ones.
Priority
Requester: Mandatory
openSUSE-11.2: New
Priority
Requester: Mandatory
Requested by: Peter Poeml
Partner organization: openSUSE.org
Description:
This concerns both YaST and zypper.
It would help a lot if they would show the mirror which is accessed
when downloading a package.
(Technically, the download client is following redirects and the
actually URL needs to be shown, not just the first URL that was
contacted but that redirected to another URL)
It is a long-standing problem that this information is lacking and it
makes debugging and optimizing much harder than necessary, because this
information is only available deeply buried in some logfile.
We need this information for two reasons:
1) Debugging - to see which URL was accessed when something goes wrong,
like a mirror timing out. For example, Whenever a mirror returns
garbage, it takes a lot of additional time to find out which one,
because all reports are lacking the critical detail, and it is usually
hard to find it out later.
2) Optimizing the load sharing between different mirrors - to
distinguish slow mirrors from fast ones. For example, I am very much in
need of information about the situation in the US, but even though
people in the US run installations for me and are willing to send me
logs, I can't get this information.
In reply to a question from Stano:
Do you want the information in the log or to be shown in e.g. zypper
-vvv ?
Shown, in normal output. It is already in some log I believe, I don't
know which one exactly anymore though.
Thing is, users won't be able to tell us anything useful if they
weren't shown the information. And later it is not reproducible.
At least in a failure situation the URLs must be shown. That alone
wouldn't give us the chance to find out about slow mirrors, but at
least it would allow to debug problems.
Real life example #1:
If a mirror is behind a firewall that lets every 10th connection stall
(so the transfer hangs or the connection is reset), it usually takes
weeks (!) to find out about it. But it hits virtually every user from
mirrors country during the time. We have had this 4 times during the
last 18 months.
Real life example #2:
I got feedback from someone who did a network install of openSUSE 11.0
from download.opensuse.org, located in the US. He had to let it running
overnight because it took a long time. Next morning, the install hadn't
even completed. YaST was hanging and waiting for somebody to hit retry.
Problem is, it took us a lot of work to find out which mirror was at
fault. We actually had to repeat the install, I had to follow the log
and live watching his machine accessing download.opensuse.org, to see
where it was redirected, and where it spent most time (while
downloading some package). We found a mirror which was really slow, and
I could disable it. I have a suspicion though, that we have more such
mirrors, because I have got some similar reports from recently. But how
to find out? People only complain about slowness, without the ability
to give any useful detail. See
http://news.opensuse.org/2008/09/23/upcoming-factory-changes/#comments (http://news.opensuse.org/2008/09/23/upcoming-factory-changes/#comments)
for examples of this.
Discussion:
#1: Susanne Oberhauser (2008-10-01 15:16:30)
What if we focussed on something like torrent for distributing packages
instead?
AFAICT these p2p network technologies have mechanisms to recover from
missing fragments, i.e. failing servers.
In that world, todays mirrors would be strong and almost always present
peers in the network.
#2: Peter Poeml (2008-10-01 15:30:56) (reply to #1)
Torrents are not efficient for files of this small size.
Another concern is that, on large scale, the reasonability of P2P
traffic with regard to network topology is questionable; confer to
what's called the Application-Layer Traffic Optimization (ALTO)
problem. See http://alto.tilab.com/ (http://alto.tilab.com/) .
There are working examples where e.g. university have rolled out a P2P
update distribution campus-wide; in a controlled limited environment
this is feasible, but not across the Internet.
Further, note that libzypp in 11.1 has preliminary support for using
metalinks for downlaods. Metalink clients recover nicely from missing
fragments. See http://en.opensuse.org/Libzypp/Failover
(http://en.opensuse.org/Libzypp/Failover) .
+ #3: Mark Goldstein (2008-10-04 17:20:42)
+ Other usage of this feature would be possibility of "blacklisting"
+ servers with persistent problems. (See thread in opensuse list "Is it
+ possible to "blacklist" specific download server in Yast?" ).
+ As Carlos E. R. suggested other useful option could be for YaST/zypper
+ to fail gracefully requesting another server to the redirector on
+ failure on the one it gave use previously.
--
openSUSE Feature:
https://features.opensuse.org/?rm=feature_show&id=305320
