[opensuse-factory] Can't combine ftp and torrent

older
[opensuse-factory] Problem with...

Rajko M.

8 Nov 2008 8 Nov '08

04:21

I can't find way to use aria2c to download from torrent and ftp server. aria2-0.16.2-1.1.i586.rpm from http://download.opensuse.org/repositories/network:/utilities/openSUSE_11.0/ Command: aria2c -p -s2 -d./ \ ftp://ftp5.gwdg.de/pub/opensuse/distribution/11.1-Beta4/iso/dvd/openSUSE-11.1-Beta4-DVD-i586.iso\ openSUSE-11.1-Beta4-DVD-i586.iso.torrent It downloads 1 iso from ftp server to ./ as specified with '-d' option and 1 from torrent in ./openSUSE-11.1-Beta4-DVD-i586-iso/ as specified in torrent file. Giving '-d ./openSUSE-11.1-Beta4-DVD-i586-iso' doesn't help. It just changes base directory from ./ to ./openSUSE-11.1-Beta4-DVD-i586-iso and runs both downloads separately again. It would be fine to be able to use ftp to speed up download and leave it as seeder for torrent. -- Regards, Rajko -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Show replies by date

Patrick Shanahan

8 Nov 8 Nov

04:49

* Rajko M. [11-07-08 23:23]:

...

I can't find way to use aria2c to download from torrent and ftp server.

...

Command: aria2c -p -s2 -d./ \ ftp://ftp5.gwdg.de/pub/opensuse/distribution/11.1-Beta4/iso/dvd/openSUSE-11.1-Beta4-DVD-i586.iso\ openSUSE-11.1-Beta4-DVD-i586.iso.torrent

... aria2c -s2 ./openSUSE-11.1-Beta4-DVD-i586.iso.torrent it will figure the direct http/ftp download part from the torrent. here is an example: 23:46 wahoo:/mnt/pictures/test > aria2c -s2 ./openSUSE-11.1-Beta4-KDE4-LiveCD-x86_64.iso.torrent *** Download Progress Summary as of Fri Nov 7 23:47:55 2008 *** ===================================================================================== [#1 SIZE:17.1MiB/698.8MiB(2%) CN:11 SPD:407.67KiB/s UP:1.24KiB/s(37.6KiB) ETA:28m32s] FILE: ./openSUSE-11.1-Beta4-KDE-LiveCD-x86_64-iso -- Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 @ http://counter.li.org -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Patrick Shanahan

05:12

* Patrick Shanahan [11-07-08 23:51]:

...

* Rajko M. [11-07-08 23:23]:

...
I can't find way to use aria2c to download from torrent and ftp server.

...

...
Command: aria2c -p -s2 -d./ \ ftp://ftp5.gwdg.de/pub/opensuse/distribution/11.1-Beta4/iso/dvd/openSUSE-11.1-Beta4-DVD-i586.iso\ openSUSE-11.1-Beta4-DVD-i586.iso.torrent

...

aria2c -s2 ./openSUSE-11.1-Beta4-DVD-i586.iso.torrent

it will figure the direct http/ftp download part from the torrent.

here is an example:

23:46 wahoo:/mnt/pictures/test > aria2c -s2 ./openSUSE-11.1-Beta4-KDE4-LiveCD-x86_64.iso.torrent *** Download Progress Summary as of Fri Nov 7 23:47:55 2008 *** ===================================================================================== [#1 SIZE:17.1MiB/698.8MiB(2%) CN:11 SPD:407.67KiB/s UP:1.24KiB/s(37.6KiB) ETA:28m32s] FILE: ./openSUSE-11.1-Beta4-KDE-LiveCD-x86_64-iso

from the man file, you could also try: aria2c -Ttest.torrent http://host1/file ftp://host2/file -d -- Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 @ http://counter.li.org -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Rajko M.

08:54

On Friday 07 November 2008 11:12:30 pm Patrick Shanahan wrote: Problem is that metalink file exists for CDs, but not for DVD. The DVD iso is special service from Eberhard, not present on other mirrors, so I had to edit metalink file, to include special /dvd directory, or torrent file not to create subdirectory. I planned to edit torrent, and point to the same directory, but it seems that only few torrent clients, like Azureus, or utorrent, have ability to change them, so I'm back to metalink file as first option.

...

from the man file, you could also try:

aria2c -Ttest.torrent http://host1/file ftp://host2/file -d

Tried it just as described in man page, but with '-d' at the beginning, and for some reason I wasn't happy. Either it used torrent alone, or doubled download as other combinations. Man page tells that -T is not necessary, but I tried both, with and without. I'll try it again later on some other file, but now I'll check each combination with the netstat. Problem is that 'netstat -tup' was very slow and 'netstat -tupn' not easy to decipher with torrent in place. Then another problem is that aria2c will open 5 connections to each source by default. I need max 2 to use my bandwidth. -- Regards, Rajko -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

josef reidinger

11:16

Rajko M. napsal(a):

...

On Friday 07 November 2008 11:12:30 pm Patrick Shanahan wrote:

Problem is that metalink file exists for CDs, but not for DVD. The DVD iso is special service from Eberhard, not present on other mirrors, so I had to edit metalink file, to include special /dvd directory, or torrent file not to create subdirectory.

I planned to edit torrent, and point to the same directory, but it seems that only few torrent clients, like Azureus, or utorrent, have ability to change them, so I'm back to metalink file as first option.

...
from the man file, you could also try:

aria2c -Ttest.torrent http://host1/file ftp://host2/file -d

Tried it just as described in man page, but with '-d' at the beginning, and for some reason I wasn't happy. Either it used torrent alone, or doubled download as other combinations. Man page tells that -T is not necessary, but I tried both, with and without.

I'll try it again later on some other file, but now I'll check each combination with the netstat. Problem is that 'netstat -tup' was very slow and 'netstat -tupn' not easy to decipher with torrent in place.

Then another problem is that aria2c will open 5 connections to each source by default. I need max 2 to use my bandwidth.

-j, --max-concurrent-downloads=N Set maximum number of parallel downloads for every static (HTTP/FTP) URL, torrent and metalink. Default: 5

...

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Peter Poeml

10 Nov 10 Nov

12:26

On Sat, Nov 08, 2008 at 12:16:09PM +0100, josef reidinger wrote:

...

...
Then another problem is that aria2c will open 5 connections to each source by default. I need max 2 to use my bandwidth.

-j, --max-concurrent-downloads=N Set maximum number of parallel downloads for every static (HTTP/FTP) URL, torrent and metalink. Default: 5

Note, the -j option is used to set the number of simultaneous downloads, in the sense of multiple files being downloaded. This is possible because metalinks can contain URLs to several files. But because in this case just 1 download takes place, -j2 has no effect. The option that limits parallel (HTTP/FTP) connections per download is -s, --split=N Download a file using N connections. If more than N URLs are given, first N URLs are used and remaining URLs are used for backup. If less than N URLs are given, those URLs are used more than once so that N connections total are made simultaneously. N must be between 1 and 16. Please see -j option too. Default: 5 Peter -- Contact: admin@opensuse.org (a.k.a. ftpadmin@suse.com) #opensuse-mirrors on freenode.net Info: http://en.opensuse.org/Mirror_Infrastructure SUSE LINUX Products GmbH Research & Development

Peter Poeml

13:30

On Mon, Nov 10, 2008 at 01:26:24PM +0100, Peter Poeml wrote:

...

On Sat, Nov 08, 2008 at 12:16:09PM +0100, josef reidinger wrote:

...
...
Then another problem is that aria2c will open 5 connections to each source by default. I need max 2 to use my bandwidth.

-j, --max-concurrent-downloads=N Set maximum number of parallel downloads for every static (HTTP/FTP) URL, torrent and metalink. Default: 5

Note, the -j option is used to set the number of simultaneous downloads, in the sense of multiple files being downloaded. This is possible because metalinks can contain URLs to several files. But because in this case just 1 download takes place, -j2 has no effect.

The option that limits parallel (HTTP/FTP) connections per download is

-s, --split=N Download a file using N connections. If more than N URLs are given, first N URLs are used and remaining URLs are used for backup. If less than N URLs are given, those URLs are used more than once so that N connections total are made simultaneously. N must be between 1 and 16. Please see -j option too. Default: 5

I have to correct myself: -s is not effective in this scenario. It is ignored for metalink downloads. The -C option needs to be used instead. This is a bit complicated and confusing, and Tatsuhiro Tsujikawa (author of aria2c) is aware of this and plans to drop the -C option and use -s option for metalink as well in 1.0.0 release. Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development

Peter Poeml

10:25

On Sat, Nov 08, 2008 at 02:54:48AM -0600, Rajko M. wrote:

...

On Friday 07 November 2008 11:12:30 pm Patrick Shanahan wrote:

Problem is that metalink file exists for CDs, but not for DVD.

Yes - so far, the metalinks don't exist for beta DVDs. But this will change with the next beta. (As I mentioned in my first mail, http://lists.opensuse.org/opensuse-factory/2008-11/msg00251.html which I'm sure you have read, but let me just reference it for newcomers in the thread.)

...

The DVD iso is special service from Eberhard, not present on other mirrors, so I had to edit metalink file, to include special /dvd directory, or torrent file not to create subdirectory.

Exactly, so far the DVDs are "special Eberhard service" (there are a few other mirrors btw), but beginning with the next beta, the isos will be in-tree, so they can become "normal Eberhard service" :-)

...

Then another problem is that aria2c will open 5 connections to each source by default. I need max 2 to use my bandwidth.

The upstream default was recently increased from 2 to 5 connections, which may be too much under some circumstances. So be sure to adjust it, see the -s/--split option. We might need to tune that default a bit, so that it is as adequate as possible for mirrors and users. Peter -- Contact: admin@opensuse.org (a.k.a. ftpadmin@suse.com) #opensuse-mirrors on freenode.net Info: http://en.opensuse.org/Mirror_Infrastructure SUSE LINUX Products GmbH Research & Development

josef reidinger

8 Nov 8 Nov

11:17

Rajko M. napsal(a):

...

I can't find way to use aria2c to download from torrent and ftp server.

aria2-0.16.2-1.1.i586.rpm from http://download.opensuse.org/repositories/network:/utilities/openSUSE_11.0/

Command: aria2c -p -s2 -d./ \ ftp://ftp5.gwdg.de/pub/opensuse/distribution/11.1-Beta4/iso/dvd/openSUSE-11.1-Beta4-DVD-i586.iso\ openSUSE-11.1-Beta4-DVD-i586.iso.torrent

It downloads 1 iso from ftp server to ./ as specified with '-d' option and 1 from torrent in ./openSUSE-11.1-Beta4-DVD-i586-iso/ as specified in torrent file.

Giving '-d ./openSUSE-11.1-Beta4-DVD-i586-iso' doesn't help. It just changes base directory from ./ to ./openSUSE-11.1-Beta4-DVD-i586-iso and runs both downloads separately again.

It would be fine to be able to use ftp to speed up download and leave it as seeder for torrent.

This would help: -i, --input-file=FILE Downloads URIs found in FILE. You can specify multiple URIs for a single entity: separate URIs on a single line using the TAB character. Reads input from stdin when - is specified. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

5653

Age (days ago)

5655

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

josef reidinger
Patrick Shanahan
Peter Poeml
Rajko M.