On 11/06/2019 21.21, Patrick Shanahan wrote:

* Andrei Borzenkov <> [06-11-19 14:02]:

...

...

lynx -listonly -dump http://download.opensuse.org/update/leap/15.1/oss/x86_64

...

Somebody has done this before?

lynx -listonly -dump -nonumbers \ http://download.opensuse.org/update/leap/15.1/oss/x86_64/

Thanks! :-) I'm impressed you two came with that so fast :-o :-) It produced a list in less than a second, I'm also impressed at that. 171755 bytes. Ah, ok, the update repository is still small, that's why it goes so fast. still, wget thinks longer. This concoction generates the wanted download list: grep -e "kernel.*rpm\|nfs-kernel.*rpm\|virtualbox.*rpm " list | \ grep -v "\.mirrorlist\|.drpm" But aria2c downloads duplicates, which it renames. :-? -rw-r--r-- 1 cer users 67263444 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.1.rpm -rw-r--r-- 1 cer users 67263444 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.rpm -rw-r--r-- 1 cer users 27761 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.rpm.1.meta4 -rw-r--r-- 1 cer users 27761 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.rpm.meta4 -rw-r--r-- 1 cer users 26482500 Jun 11 22:30 kernel-debug-base-4.12.14-lp151.28.4.1.x86_64.1.rpm -rw-r--r-- 1 cer users 26482500 Jun 11 22:29 kernel-debug-base-4.12.14-lp151.28.4.1.x86_64.rpm ... Indeed, the list file contains duplicates, if file seems to be listed twice: http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-4.12.1... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-4.12.1... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-4.12.1... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-base-4... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-base-4... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-base-4... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-devel-... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-devel-... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-devel-... So I will have to use "uniq". ... There is a remaining problem: a second run downloads a second copy, renamed *.1.rpm. I fail to see the adequate option in the aria2c man :-? Maybe... --allow-overwrite [true|false] Restart download from scratch if the corresponding control file doesn’t exist. See also --auto-file-renaming option. Default: false --auto-file-renaming [true|false] Rename file name if the same file already exists. This option works only in HTTP(S)/FTP download. The new file name has a dot and a number(1..9999) appended after the name, but before the file extension, if any. Default: true Ok, got something that works: lynx -listonly -dump -nonumbers "http://download.opensuse.org/update/leap/15.1/oss/x86_64/" > 0list_1 uniq 0list_1 0list_2 grep -e "kernel.*rpm\|nfs-kernel.*rpm\|virtualbox.*rpm" 0list_2 | grep -v "\.mirrorlist\|.drpm" > 0list_3 aria2c --continue --allow-overwrite=false --auto-file-renaming=false --input-file=0list_3 lynx -listonly -dump -nonumbers http://download.opensuse.org/update/leap/15.1/oss/noarch/ > 0list_1 uniq 0list_1 0list_2 grep -e "kernel.*rpm\|nfs-kernel.*rpm\|virtualbox.*rpm" 0list_2 | grep -v "\.mirrorlist\|.drpm" > 0list_3 aria2c --continue --allow-overwrite=false --auto-file-renaming=false --input-file=0list_3 rm *meta4 *.aria2 -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)

* Patrick Shanahan [06-11-19 17:20]:

* Carlos E. R. [06-11-19 16:59]:

...

On 11/06/2019 21.21, Patrick Shanahan wrote:

...

* Andrei Borzenkov <> [06-11-19 14:02]:

...

...

...

lynx -listonly -dump http://download.opensuse.org/update/leap/15.1/oss/x86_64

...

Somebody has done this before?

lynx -listonly -dump -nonumbers \ http://download.opensuse.org/update/leap/15.1/oss/x86_64/

Thanks! :-)

I'm impressed you two came with that so fast :-o :-)

It produced a list in less than a second, I'm also impressed at that. 171755 bytes. Ah, ok, the update repository is still small, that's why it goes so fast. still, wget thinks longer.

This concoction generates the wanted download list:

grep -e "kernel.*rpm\|nfs-kernel.*rpm\|virtualbox.*rpm " list | \ grep -v "\.mirrorlist\|.drpm"

But aria2c downloads duplicates, which it renames. :-?

-rw-r--r-- 1 cer users 67263444 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.1.rpm -rw-r--r-- 1 cer users 67263444 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.rpm -rw-r--r-- 1 cer users 27761 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.rpm.1.meta4 -rw-r--r-- 1 cer users 27761 Jun 11 22:29 kernel-debug-4.12.14-lp151.28.4.1.x86_64.rpm.meta4 -rw-r--r-- 1 cer users 26482500 Jun 11 22:30 kernel-debug-base-4.12.14-lp151.28.4.1.x86_64.1.rpm -rw-r--r-- 1 cer users 26482500 Jun 11 22:29 kernel-debug-base-4.12.14-lp151.28.4.1.x86_64.rpm ...

Indeed, the list file contains duplicates, if file seems to be listed twice:

http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-4.12.1... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-4.12.1... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-4.12.1... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-base-4... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-base-4... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-base-4... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-devel-... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-devel-... http://download.opensuse.org/update/leap/15.1/oss/x86_64/kernel-debug-devel-...

So I will have to use "uniq".

...

There is a remaining problem: a second run downloads a second copy, renamed *.1.rpm. I fail to see the adequate option in the aria2c man :-?

Maybe...

--allow-overwrite [true|false] Restart download from scratch if the corresponding control file doesn’t exist. See also --auto-file-renaming option. Default: false

--auto-file-renaming [true|false] Rename file name if the same file already exists. This option works only in HTTP(S)/FTP download. The new file name has a dot and a number(1..9999) appended after the name, but before the file extension, if any. Default: true

Ok, got something that works:

lynx -listonly -dump -nonumbers "http://download.opensuse.org/update/leap/15.1/oss/x86_64/" > 0list_1 uniq 0list_1 0list_2

grep -e "kernel.*rpm\|nfs-kernel.*rpm\|virtualbox.*rpm" 0list_2 | grep -v "\.mirrorlist\|.drpm" > 0list_3 aria2c --continue --allow-overwrite=false --auto-file-renaming=false --input-file=0list_3

lynx -listonly -dump -nonumbers http://download.opensuse.org/update/leap/15.1/oss/noarch/ > 0list_1 uniq 0list_1 0list_2 grep -e "kernel.*rpm\|nfs-kernel.*rpm\|virtualbox.*rpm" 0list_2 | grep -v "\.mirrorlist\|.drpm" > 0list_3 aria2c --continue --allow-overwrite=false --auto-file-renaming=false --input-file=0list_3

rm *meta4 *.aria2

or maybe the man page for aria2c --continue=true and/or --auto-file-renaming=false

you test and you choose

doh! too much cruft! -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Registered Linux User #207535 @ http://linuxcounter.net Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org