On 10/20/2013 03:42 PM, Frans de Boer wrote:

...on changed or new files on HTTP/HTML only servers.

I like to track some directories to see if there are new source tarballs added and subsequently download them automatically. I can do that with FTP sites using my own scripts, but processing HTML pages is different for every page/site and would require custom processing for each different page and directory.

I know it is hard to do while using linux basic tools (lynx, curl, wget) and bash scripting, but if someone has a pointer, then please let me know. By the way, they say that perl and LWP might do, but I do a lot of programming in various languages but not with perl or python.

Regards, Frans. What about using rsync ?

-- Duaine Hechler Piano, Player Piano, Pump Organ - Tuning, Servicing & Rebuilding (314) 838-5587 / dahechler@att.net / www.hechlerpianoandorgan.com Home & Business user of Linux - 13 years -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

 On October 20, 2013 at 10:42 PM Frans de Boer  wrote:  ...on changed or new files on HTTP/HTML only servers.    I like to track some directories to see if there are new source tarballs   added and subsequently download them automatically.

I'd start with   wget --mirror and play with options like -X -np to limit the download. Have fun, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org