Hi, Am 17.03.2010 um 15:08 schrieb Peter Pöml:
Am 12.03.2010 um 08:30 schrieb Roger Oberholtzer:
On Wed, 2010-03-10 at 14:18 +0100, Peter Pöml wrote:
Hi,
So IMO what we see is a violation of RFC2616 by the Ironport appliance. Might be worthwhile to open a bug with its manufacturer.
I got the IT guys to let me out without going through the proxy. And magically all worked. That does not mean the IronPort is the source of the problem. Do we know that aria2c does the right thing when range requests are disabled? The reason the IronPort disables range requests is that when they are on it cannot check the downloads for malware. It needs the whole thing to do so.
As mentioned before, disabling byte ranges is fine, however then the proxy must also remove the header line that claims that byte ranges are supported. This is a very simple header manipulation that the manufacturer of the appliance just overlooked. Maybe your IT guys could even work around by configuring the removal of the header manually -- if that's possible.
I'd suggest asking them whether they can completely suppress that header line by filtering it.
When the range issue is sorted, another issue could be that arias2c would have to wait for the proxy to download the whole item and check it before aria2c sees the first byte. For large/slow items, I guess that could be an issue. Or not.
That would be a principal issue with any intercepting intermediary that scans for malware.
But I will start by seeing about the range request issue. Anyone have any ideas on how I can tell that aria2c is even trying to detect if range requests are allowed?
I should note that I don't claim that fixing the problem in IronPort will necessarily make aria2c work in your situation. There could of course also be a problem with aria2c, in addition to the evident problem in IronPort. I tried to download a file http://opensuse.mirrors.proxad.net/opensuse/, which happens to have byte ranges disabled. Just pick a file from there and run aria2c with the URL. aria2c reports some warnings on the way, but downloads correctly in the end (I verified with a cryptographic hash). I interrupt the download, I can't restart it later - which makes sense in a way, because without byte range support there is no way to complete a download other than starting from the beginning.
So I'd say, aria2c does the best it can and works correctly here (despite some warnings that it expresses).
Peter--
Recap (& further analysis after a little bit of testing): - aria2c can download fine from behind the intercepting intermediary or from a mirror that doesn't support byte ranges at all - aria2c also copes with the situation of the broken intermediary in your case - libzypp leaves last attempts downloads around (I strongly suppose), so when _retrying_ a download, aria2c tries to complete the existing file; in order to do that, it _has_ to use byte range requests (or resume in FTP). - that fails if aria2c is behind a proxy like yours (which leaves byterange headers in, but doesn't allow them), or when no mirror supports byte ranges (nearly all do) - aria2c tries all mirrors in that situation, which of course takes a long time. - (quoting Tatsuhiro here) it would be an easy solution for libzypp to delete partial file before download, but if the file size is big and the connection is slow, resuming file will help users. Apparently, the folks over at Mandriva ran into the same situation. I have good news though: Tatsuhiro Tsujikawa (author of aria2c) has added an interesting feature that allows dealing with the situation:
Here is changelog:
Added --always-resume and --max-resume-failure-tries option. If --always-resume=false is given, when all given URIs do not support resume or aria2 encounters N URIs which does not support resume(N is the value specified using --max-resume-failure-tries option), aria2 downloads file from scratch. The default behavior is --always-resume=true, which means if all URIs do not support resume, download fails. I think this is OK because user normally don't like to see that partially downloaded file is overwritten(this is particularly true if file size is big). This option is useful when aria2 is used as download backend and graceful falling back to overwritten behavior is preferable. Added exit status value 8, which means download failed
That should help. Jano, I expect the change to turn up with the next aria2c release. So the new feature could be used in 11.3! Peter-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org