On Saturday 02 December 2006 13:10, Francis Giannaros wrote:

Are there any extra measures in place to prevent opensuse.org going down before the official 10.2 announcement, because of the hype?

What hype? Do you mean rush? Bye, Steve --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

...

...

Are there any extra measures in place to prevent opensuse.org going down before the official 10.2 announcement, because of the hype?

What hype? Do you mean rush?

I think he means, are you prepared for a possible DoS ahead of the release announcement, on account of all the noise that was generated in response to the deal with The Company From Redmond�.

This article at http://www.linux.com/article.pl?sid=06/11/01/1641247 deals with how Metalink users were still able to easily download Fedora Core 6 at release. With the release of Fedora Core 6, their main website and yum update servers were unusable for days. I would guess this was because the main site was serving ISOs as well. I think it would be good to encourage people to download with methods that will lessen the strain on the main servers, at least for the first few days. p2p is great for this, but we have to remember that p2p isn't available to everyone. It's blocked by some universities, businesses, and ISPs. Besides regular p2p/bittorrent, you can continue using Metalink to spread out the downloads between mirrors, either removing the listing for the main server or lowering its priority rating. Metalinks can list mirrors, along with torrents, ed2k, and magnet links. People can then download from local servers if they set their location, since the mirrors have location and priority information. <url type="ftp" location="jp" preference="10"> ftp://ftp.jaist.ac.jp/pub/Linux/openSUSE/distribution/openSUSE-10.2-RC1/iso/openSUSE-10.2-RC1-i386-CD1.iso </url> <url type="http" location="jp" preference="10"> http://ftp.kddilabs.jp/Linux/packages/opensuse/distribution/openSUSE-10.2-RC... </url> The Metalinks for RC1 are available at http://download.packages.ro/metalink/opensuse/ . Ones for the final release will be automatically generated once the ISOs are synced to the mirrors. (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

...

p2p is great for this, but we have to remember that p2p isn't available to everyone. It's blocked by some universities, businesses, and ISPs. Besides regular p2p/bittorrent, you can continue using Metalink to spread out

On Saturday 02 December 2006 20:40, Anthony Bryan wrote: the

...

downloads between mirrors, either removing the listing for the main server or lowering its priority rating. Metalinks can list mirrors, along with torrents, ed2k, and magnet links. People can then download from local servers if they set their location, since the mirrors have location and priority information.

<url type="ftp" location="jp" preference="10">

ftp://ftp.jaist.ac.jp/pub/Linux/openSUSE/distribution/openSUSE-10.2- RC1/iso /openSUSE-10.2-RC1-i386-CD1.iso </url> <url type="http" location="jp" preference="10">

http://ftp.kddilabs.jp/Linux/packages/opensuse/distribution/openSUSE- 10.2-R C1/iso/openSUSE-10.2-RC1-i386-CD1.iso </url>

The Metalinks for RC1 are available at http://download.packages.ro/metalink/opensuse/ . Ones for the final release will be automatically generated once the ISOs are synced to the mirrors.

That seems like a very nice idea; I investigated by downloading a few ISOs and they maxed out my connection (24mbit, but still).

Are there any plans to advertise these metalinks specifically, at all? I guess part of the problem would be that people really don't know about the procedure or applications (despite being so easy to use), unlike torrents. Still, looks like something very good to explore.

I've only been here for two previous releases, but the site was down both times and things weren't going so well at all. We get many new users in the channels asking questions, and while we'd like to give them the links to the very easy howtos on the wiki, it's not available :/. Try to improvise with Google cache, most of the time. Not ideal though.

Hi Francis, Thanks for taking the time to test metalink out. 24 mbit, nice! About 3 MB/sec? Unfortunately my home cable only gets about a third of that. I don't know if metalinks will be specifically advertised. They are listed on http://en.opensuse.org/Released_Version#Metalink, but like you said when the sites down - it's down (kinda frustrating and). One nice thing about metalink, in these busy situations you don't have to manually try all sorts of mirrors to see if you can get through. The clients do that automatically. I always hated that. As I mentioned, it allowed people to get FC6 during the mad onslaught when it seemed impossible otherwise. What I was proposing on the list (maybe not explicit enough) was that the site goes into crisis mode. Only list metalinks and torrents, and rely on p2p/mirrors to distribute the ISOs. Either stop the main site from ISO downloads, or severely limit it so the site can stay up. Only list the most important info. Make sure people can get aria2 or wxDownload Fast easily, have em packaged for a few versions back of suse. Make it as simple as possible. 1) Install RPM 2) Open terminal and type 'aria2c URLtoMetalink' 3) Wait curl and wget are adding support for metalink, but I think someone could whip something up quick which would allow either of them to use it now. It would just be a quick front end that reads the XML, and gets a chunk from a few servers, then runs md5sum on it when done. (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

opensuse users, read ahead :-) thanks Anthony Bryan a écrit :

What I was proposing on the list (maybe not explicit enough) was that the site goes into crisis mode. Only list metalinks and torrents, and rely on p2p/mirrors to distribute the ISOs. Either stop the main site from ISO downloads, or severely limit it so the site can stay up. Only list the most important info.

to be practical, I propose that you spread this very post. I already crosspost to opensuse list. Here are the instruction, feel freee to edit is I made any mistake. If you have problems joining the opensuse.org site or any mirror during the first days of the 10.2 release, there is a solution. The solution is to use metalink. This utility, in very short summary, make a balance between all the available ftp sources and allows you to download from the better one. as long as the site is available you can look at http://en.opensuse.org/Released_Version#Metalink The .metalink file is approximately like a torrent file, very small and easy to catch. you need a metalink client, there are many for any system (for example aria2 or wxDownload). then look at ftp://ftp.jaist.ac.jp/pub/Linux/openSUSE/distribution/ with your browser. This ftp is _not_ a opensuse mirror, so should not be down test your system with the 10.2 rc1, for example. GM and 10.2 are not yet functional, but will be asap (the dec 7) enjoy :-) -- http://www.dodin.net http://dodin.org/mediawiki/index.php/GPS_Lowrance_GO --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

sorry, bad link in previous mail :-( opensuse users, read ahead :-) thanks Anthony Bryan a écrit :

What I was proposing on the list (maybe not explicit enough) was that the site goes into crisis mode. Only list metalinks and torrents, and rely on p2p/mirrors to distribute the ISOs. Either stop the main site from ISO downloads, or severely limit it so the site can stay up. Only list the most important info.

to be practical, I propose that you spread this very post. I already crosspost to opensuse list. Here are the instruction, feel freee to edit is I made any mistake. If you have problems joining the opensuse.org site or any mirror during the first days of the 10.2 release, there is a solution. The solution is to use metalink. This utility, in very short summary, make a balance between all the available ftp sources and allows you to download from the better one. as long as the site is available you can look at http://en.opensuse.org/Released_Version#Metalink The .metalink file is approximately like a torrent file, very small and easy to catch. you need a metalink client, there are many for any system (for example aria2 or wxDownload). then look at http://download.packages.ro/metalink/opensuse/ with your browser. This ftp is _not_ a opensuse mirror, so should not be down test your system with the 10.2 rc1, for example. GM and 10.2 are not yet functional, but will be asap (the dec 7) enjoy :-) -- http://www.dodin.net http://dodin.org/mediawiki/index.php/GPS_Lowrance_GO --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2006-12-04 at 19:12 +0100, jdd sur free wrote: ...

The solution is to use metalink. This utility, in very short summary, make a balance between all the available ftp sources and allows you to download from the better one.

as long as the site is available you can look at http://en.opensuse.org/Released_Version#Metalink ... you need a metalink client, there are many for any system (for example aria2 or wxDownload).

A warning: there are three "aria" projects at sourceforge. The "aria" one is: ARIA (Accounting, Receiving and Inventory Administration) is a web based accounting and inventory management package based off the NOLA project. ARIA aim's to be platform and browser independent with multi-language support. which obviously is not the one we want - I found out while trying to install it O:-) -. Another one is "Aria download manager RPM Project", which also isn't, but not so obvious (it is a "download manager", after all). The correct one is "aria2", which is not found searching for "aria" in sourceforge: you have to search for "aria2" specifically. It would be nice if the wiki (http://en.opensuse.org/Released_Version#Metalink) did not only mention the Metalink existence, but also provided a link to a readily compiled rpm for all currently SuSE versions in use (10.1, 10.1, 9.3, SLES, etc), if you want people to use metalink to download 10.2. That means today ;-) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFFdugdtTMYHG2NR9URAsLDAJ4jzveE9o8bSXyxqHvFI8vp90P18wCeNHWF e5jsZZWvpqv7Tg1yBE19qPY= =zedx -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On the whole it was all quite successful -- the site stayed up throughout, and users got very fast and reliable downloads thanks to metalink. The torrents were originally very fast but the pace dropped down after the initial flood in. Metalinks went well throughout the day, so good stuff. Regards, Francis.

On Fri, Dec 08, 2006 at 05:00:52PM +0000, Francis Giannaros wrote:

On the whole it was all quite successful -- the site stayed up throughout, and users got very fast and reliable downloads thanks to metalink. The torrents were originally very fast but the pace dropped down after the initial flood in. Metalinks went well throughout the day, so good stuff.

It all depends on how many downloads there were compared to last times. It could b that people run away from Novell, due to the deal, or instead became curious and downloaded even more. It would be interesting to see the numbers compared to other versions if those are available. I understand that you can't put a total on it, but Novell could compare the number of people who 'registerd'. In perecntage compared with the real downloads, that should give a good indication. Or just some webstats comparing the different versions in another way. I suspect that the download is less, because e.g. /. still has not mentioned it and my submited story is still pending and I doubt that I am the only one. houghi -- To have a nice mailinglist experience, follow the guidelines below:

Please do not toppost. Please turn off HTML Read http://en.opensuse.org/Opensuse_mailing_list_netiquette Read http://www.catb.org/~esr/faqs/smart-questions.html

To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

Francis Giannaros a écrit :

The torrents were originally very fast but the pace dropped down after the initial flood

no drop here, on the contrary the number of clients/servers increase. the more are for the i386 dvd, of course. I have now 730 sources and 3303 clients for it (326/1144 for the x86_64 dvd) and of course all the people installing the distro can't seed :-) jdd -- http://www.dodin.net http://dodin.org/mediawiki/index.php/GPS_Lowrance_GO --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On Sat, 9 Dec 2006, Carlos E. R. wrote:

The Friday 2006-12-08 at 22:17 +0100, jdd sur free wrote:

...

...

The torrents were originally very fast but the pace dropped down after the initial flood

no drop here, on the contrary the number of clients/servers increase. the more are for the i386 dvd, of course. I have now 730 sources and 3303 clients for it (326/1144 for the x86_64 dvd)

No, it is slow: my torrent download is going slow: 58% in 22 hours... ie, less than 30 KB/S avg, a third of my bandwidth. This matchs what you say, that you see more clients. And remember, adsl is asymetric, we can not seed as fast as we download.

You can use metalink and use the stuff I added to the page. 30 people have verified that it works with rsync to correct problems. It is extremely fast repaiging them. -- Boyd Gerber ZENEZ 1042 East Fort Union #135, Midvale Utah 84047 --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

Carlos E. R. a écrit :

No, it is slow: my torrent download is going slow: 58% in 22 hours... ie, less than 30 KB/S avg, a third of my bandwidth. This matchs what you say, that you see more clients. And remember, adsl is asymetric, we can not seed as fast as we download.

I know (I have 10/1Mb), I mean only the numbers are encreasing, so there is more interest, but the ration dl/seed is not enough :-(, may be because one needs to install on his only computer (not anybody have 4 computers at home as I do :-() jdd -- http://www.dodin.net http://dodin.org/mediawiki/index.php/GPS_Lowrance_GO --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On Sat, 9 Dec 2006, Carlos E. R. wrote:

...

The Friday 2006-12-08 at 22:17 +0100, jdd sur free wrote:

...

...

The torrents were originally very fast but the pace dropped down after the initial flood

no drop here, on the contrary the number of clients/servers increase. the more are for the i386 dvd, of course. I have now 730 sources and 3303 clients for it (326/1144 for the x86_64 dvd)

No, it is slow: my torrent download is going slow: 58% in 22 hours... ie, less than 30 KB/S avg, a third of my bandwidth. This matchs what you say, that you see more clients. And remember, adsl is asymetric, we can not seed as fast as we download.

You can use metalink and use the stuff I added to the page. 30 people have verified that it works with rsync to correct problems. It is extremely fast repaiging them.

From the point of view of metalink, this release seems to have gone pretty well. I spent some time in #suse & would like to thank apokryphos, yaloki, bgerber, benjiman, and the others helping everyone out that was trying to get the ISOs. And Manuel Subredu, at the RoEduNet Iasi mirror who wrote the software generating the .metalinks.

It looks like there were twenty thousand .metalink downloads in the first day. Obviously, most mirrors were stressed from the sudden onslaught and people were having trouble getting the ISOs or getting them very slowly. Some mirrors crawl, while others do better. I saw quite a few people learning about metalink for the first time. Everyone reported good download speeds (usually their max). A few reported checksum errors, which could not be avoided due to the situation, but were repaired with rsync. (Except for Carlos' problem). The chunk checksums should soon correct this, when they've been added to clients. I always thought manually gathering URLs was annoying and I'm glad I could finally talk some people into trying metalink. Having the multiple redundant URLs and automating it is all much simpler. It would be nice if there were official RPMs for aria2, hopefully enough people were exposed to it and had a good experience. KDE4's KGet should also support metalink so that should be much easier for average people. (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On Sunday 10 December 2006 13:33, Anthony Bryan wrote: .....

I always thought manually gathering URLs was annoying and I'm glad I could finally talk some people into trying metalink. Having the multiple redundant URLs and automating it is all much simpler.

Hi Anthony, The idea to automate mirror (download source) selection is great, but there is so many details on and around your web page that doesn't help me. Let me explain. 1) Project web page http://www.metalinker.org gives promisses in the same marketing style as menu commercial sites. All is so great, no problems, only benefits, the life is good. 2) All rights reserved at the bottom, sounds so proprietary. No word about license. 3) Downloads are verified for enhanced reliability. -Nothing new for YaST users. 4) No Single Point of Failure (SPOF) like FTP or HTTP URLs. -It showed that implementation is not that perfect and single point of failure ruined some people experience. Anything that comes with a lot of declarative sentences and some inserted jokes, can't give me confidence. Jokes alone are not the problem. 5) More fault tolerant. -I don't need rsync to repair anything with classic methods like wget. 6) It's a neutral standard that doesn't favor any one program, Operating system, or group, and is easy to implement. -Who made it standard? I can't find references on web site. 7) OpenOffice.org uses Metalinks. http://distribution.openoffice.org/p2p/magnet.html I'm removing listed magnet clients from any computer and explain users that they should not install them again. Do you think that anything listed on the same page will gain my trust? As I said, the idea is very attractive, but marketing isn't. The download speed and convenient automation of process are not the only factors when someone uses Internet as a software source. -- Regards, Rajko. --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On 12/11/06, Anthony Bryan wrote:

...

2) All rights reserved at the bottom, sounds so proprietary. No word about license.

That's for the text of the site. What do you mean by license? I offer no software. Some of the software that uses Metalink is under the GPL, some is commercial.

The fact that there is no official implementation of Metalink software is actually the main issue I have with Metalink. It feels too much like there is snake oil involved, with claims and promises and what feels like marketing, and the delivery of actual results is left to other parties. Take Bittorrent and its rise to fame, it had an official and basic client, which was proven to work, and was then implemented by countless clients. Similarly I would like to have seen Metalinks pushed by an implementor who wanted to share their creation.

I'm sorry you see it as marketing, I know I'd be a pretty bad salesman. (it's just what I wrote to get the basic idea across, I don't think it's particularly good but it seemed ok to me). I think Metalink solves some problems and of course that's fine if you disagree :) I only hope that people try it out and decide for themselves.

Most of us are used to two worlds, the opensource - where projects are generally started to scratch your own itch and the successful ones take on a life - and the commercial - where things are made for profit. Metalink.org and your motivations for pushing metalinks are not clear. Don't get me wrong, I am not in any way trying to discourage you in your quest or others from trying clients with Metalinks, I am just sharing my thoughts how I and maybe others see it. Pflodo Peter Flodin.

(( Anthony Bryan )) Metalink [ http://www.metalinker.org ]

To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On 12/11/06, Anthony Bryan wrote:

...

...

2) All rights reserved at the bottom, sounds so proprietary. No word about license.

That's for the text of the site. What do you mean by license? I offer no software. Some of the software that uses Metalink is under the GPL, some is commercial.

The fact that there is no official implementation of Metalink software is actually the main issue I have with Metalink. It feels too much like there is snake oil involved, with claims and promises and what feels like marketing, and the delivery of actual results is left to other parties.

Take Bittorrent and its rise to fame, it had an official and basic client, which was proven to work, and was then implemented by countless clients. Similarly I would like to have seen Metalinks pushed by an implementor who wanted to share their creation.

I don't have the resources of BitTorrent Inc ($28 million and deals with the MPAA). I am sharing my creation, there are multiple parties using it and I have been trying my best to get others to use it. I've been working on Metalink on and off for over a decade, and more than full time for over a year. I fund it all myself, with absolutely no income from it.

...

I'm sorry you see it as marketing, I know I'd be a pretty bad salesman. (it's just what I wrote to get the basic idea across, I don't think it's particularly good but it seemed ok to me). I think Metalink solves some problems and of course that's fine if you disagree :) I only hope that people try it out and decide for themselves.

Most of us are used to two worlds, the opensource - where projects are generally started to scratch your own itch and the successful ones take on a life - and the commercial - where things are made for profit. Metalink.org and your motivations for pushing metalinks are not clear.

Don't get me wrong, I am not in any way trying to discourage you in your quest or others from trying clients with Metalinks, I am just sharing my thoughts how I and maybe others see it.

I don't get it, it is pretty discouraging. Where are you seeing anything commercial with what I am doing? (Besides GetRight and Speed Download, which have pay versions, but that has nothing to do w/ me). There aren't even ads on my site (which is down now). BitTorrent which you mentioned is commercial with a bit of opensource, now with closed source clients and patented extensions. This is my itch. I'm scratching it. I'd like others to join in, and some have, but more would be nice. just having a rotten day, (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On Monday 11 December 2006 01:17, Anthony Bryan wrote: Sorry for late answer, I've seen message yesterday, but it was too late to answer. I'm trying to be helpful and that takes time.

Hi Rajko,

...

The idea to automate mirror (download source) selection is great, but there is so many details on and around your web page that doesn't help me.

Let me explain.

1) Project web page http://www.metalinker.org gives promisses in the same marketing style as menu commercial sites. All is so great, no problems, only benefits, the life is good.

The point of the site/text is to talk people into trying it out and giving it a chance. I've done my best to give info to most of the people interested. I'm not familiar with menu commercial sites.

It should be "many" sorry for typo. I mentioned the style where many words are used to praise product and no word what are the limitations. More technical details about methods to achieve download will be more than welcome. Open talk about underlaying ideas will give everybody different feeling about your project. The way you did it is OK for commercial software, where ideas how it works are hidden from public.

...

2) All rights reserved at the bottom, sounds so proprietary. No word about license.

That's for the text of the site. What do you mean by license? I offer no software. Some of the software that uses Metalink is under the GPL, some is commercial.

OK. The problem is that with previously mentioned ,missing talk about ideas, it gives feeling that whatever you offer is proprietary. If you would mention anywhere that described method is patent free and content of web site is protected with some kind of opensource license than there will be no reasons for questions.

...

3) Downloads are verified for enhanced reliability. -Nothing new for YaST users.

Cool. Not everyone uses or can use YaST in all situations :)

I agree. That is actually trivial and missing feature in many download products.

...

4) No Single Point of Failure (SPOF) like FTP or HTTP URLs. -It showed that implementation is not that perfect and single point of failure ruined some people experience. Anything that comes with a lot of declarative sentences and some inserted jokes, can't give me confidence. Jokes alone are not the problem.

This was explained above. More technical details will give me and many others more confidence that metalink is worth to try.

That is a reference to having unorganized single links to an ISO for instance. If the server goes down or is hammered, it will be hard to download. A lot of people experienced this for the 10.2 release. Metalinks list many URLs, so the clients can automate the process of connecting and using them, if some mirrors are down it recovers gracefully, and for instance not using some mirrors if the download rate is below a certain limit, etc.

...

5) More fault tolerant. -I don't need rsync to repair anything with classic methods like wget.

? Of course you have to (or do something else), or you have to start over from scratch if there's an error in transfer. As far as I know, there is no difference in the basic transfer methods of Metalink clients and wget. Someone could quickly write a Metalink interface for wget.

The problem is that TCP has error correction and it will request packages that are bad. The only thing that can brake download is timeout in transmission and that can happen in rush hours on new release. In that case wget -c will not start from scratch.

...

6) It's a neutral standard that doesn't favor any one program, Operating system, or group, and is easy to implement. -Who made it standard? I can't find references on web site.

The clients that use it. No standards body has endorsed it. If you know of any that you think would be interested, let me know.

This is new method and it is far from being even "de facto" standard because everybody is using it. If you would advertised it as a "new method that doesn't favor any one program, Operating system, or group, and is easy to implement." it will be exactly what it is and no one will complain. Well, I'm the only one that said this loud, but be sure that many think this way.

...

7) OpenOffice.org uses Metalinks. http://distribution.openoffice.org/p2p/magnet.html I'm removing listed magnet clients from any computer and explain users that they should not install them again. Do you think that anything listed on the same page will gain my trust?

I'm not sure what this means. Can you explain it more? Is there something untrustworthy about magnet clients? I wasn't aware of that. Metalink just happens to be listed on the same page by OpenOffice.org. If there is something wrong with them, I will try to get removed from that page.

Google returns links like this: http://www.wired.com/news/privacy/0,1848,49430,00.html http://news.com.com/2100-1038_3-5215028.html http://www.pestpatrol.com/spywarecenter/pest.aspx?id=453088059 (note versions) Could be also something like this: http://www.gnutellaforums.com/showthread.php?t=40161 So after some time spent in research I can understand why OP didn't understand my comment about magnet clients. It can be that it was just old version or hacked copies that gave me a lot of problems on few occasions.

...

As I said, the idea is very attractive, but marketing isn't. The download speed and convenient automation of process are not the only factors when someone uses Internet as a software source.

I'm sorry you see it as marketing, I know I'd be a pretty bad salesman. (it's just what I wrote to get the basic idea across, I don't think it's particularly good but it seemed ok to me). I think Metalink solves some problems and of course that's fine if you disagree :) I only hope that people try it out and decide for themselves.

I actually do not disagree with idea and I stated that more than one time, but your presentation of metalinks is missing details. I would like that you give some technical background how it works and to clear legal status. Is it proprietary technology or opensource? I have no problem with any technology as long as it is explained, software works as described and the price is right :-) but I have problem with any black boxes that do something, but even author can't explain details. That is, looking to some other posts, not only my impression. Present one (or few) clients in more details. You didn't make them, but they will make whole idea fly or fail, so you have to take time and present them with much more than one screenshot and few links. It will give to visitors feeling that whole idea is a serious technical work and that is what many are looking for before give a test drive. -- Regards, Rajko. --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

Anthony Bryan a écrit :

I don't have the resources of BitTorrent Inc

the real problem, here is in my opinions to know if metalink is a real progress. I mean a complete system, with bittorrent and eventually rsync addition (may be a simple batch to do that). and is it a real progress for Novell and openSUSE in the way of spreading they distribution. If it is not, let alone... if it is openSUSE have the man power to get a client uptodate (aeria2 or any other)... it's as simple as that. We all know that openSUSE (and probably others distributions as well) have mirror problems, so is this a solution? obviously not for 10.2, but for 10.3 may be Anthony didn't know sufficiently well advertise it's product. If this product is really usefull, he may appreciate any help, I'm sure... jdd -- http://www.dodin.net http://dodin.org/mediawiki/index.php/GPS_Lowrance_GO --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On Wednesday 13 December 2006 05:14, Rajko M. wrote:

Commercial applications that offer empty shell wave with features and lack details. One (not so good) screenshot and few links to clients is not enough to present one technical idea.

The proposition that it offers empty shell wave features is completely unjustified. It quite clearly claims *what* it offers and how. Whether it delivers on all of the points is perhaps questionable for you, but it's emphatically clear that it does deliver on some of them.

The problem that you don't want to see is that the number of successes doesn't prove that implementation is good, and one failure proves opposite. It works the same as with software bugs.

Currently this is down to the incomplete implementation in aria2, granted. But your conclusion doesn't follow. If it works for hundreds of people then it is most certainly a decent implementation, but not a perfect one. So currently, we make do, and use rsync if there are any errors. You might not like it, hundreds of other people don't mind, if it's the way to go for the time-being.

The wget works fine in normal traffic. It can fail if server (any on the road from source to destination) fails, and that should be covered with metalinks.

That's the job of the client, and it's being implemented into aria2 as we speak I believe.

The rsync should not be necessary. It is proof that either metalink concept or implementation is not good in all details.

I completely agree, and it's the implementation isn't perfect yet. That doesn't mean it's not decent.

BTW, I an use rsync alone and skip any repair.

Which leaves you with (i) looking through some 51 mirrors to find a fast one, or (ii) dealing with an incredibly low download speed. Not a solution in my opinion, for release time.

The Fedora, #suse and any size of positive comments will not help to convince any carefull reader. I can see only black box that sometimes fails on one of it's important features and no one can tell why.

I'm not convinced that it's a black box: everything that's done is quite open. aria2 is free software, and metalinks is an idea quite freely available. There's also a difference between a "careful reader" and a "cautious reader". A careful reader will notice that the description doesn't provide enough information (if it doesn't), but it's the cautious reader that goes the extra step. Anyhow, at the moment I think, considering all the flaws of the current implementation, it's still hands-down the top method I'd use. I explained why. Fastest ever download speeds, and if you use rsync it's always guaranteed. Again, not perfect, but still very very good for me. Considering I'd perhaps only ever have to do rsync 1 in 100 times.

I would like that metalinks works perfectly and to be what web site is promising, fine complement to present file transfer options that we have right now. The speed improvement is fine, but now developers have to see why it failed and how to skip rsync phase in the future.

Agreed. Regards, Francis.

Hi again Rajko,

It should be "many" sorry for typo. I mentioned the style where many words are used to praise product and no word what are the limitations. More technical details about methods to achieve download will be more than welcome. Open talk about underlaying ideas will give everybody different feeling about your project.

The way you did it is OK for commercial software, where ideas how it works are hidden from public.

Nothing at all is hidden from the public.

From the site: Metalink is a simple XML (text) format. Download one and open it with a text editor, & most of it should be self explanatory. If you want to read the long, boring Metalink 3.0 specification, here you go:

http://www.metalinker.org/Metalink_3.0_Spec.odt http://www.metalinker.org/Metalink_3.0_Spec.pdf (I think you are looking for section 3.2). Here's a .metalink (minus most <url>s for brevity). The client can only use the information that's there. It's pretty straightforward. <file name="openSUSE-10.2-GM-DVD-i386.iso"> <os>Linux-x86</os> <size>3880814592</size> <verification> <hash type="md5">12b0f2fabb4d41586601f787d33ebc34</hash> </verification> <resources> <url type="http" location="jp" preference="10"> http://ftp.kddilabs.jp/Linux/packages/opensuse/distribution/openSUSE-current /iso/dvd/openSUSE-10.2-GM-DVD-i386.iso </url> <url type="ftp" location="tw" preference="10"> ftp://ftp.isu.edu.tw/Linux/SuSE/distribution/openSUSE-current/iso/dvd/openSU SE-10.2-GM-DVD-i386.iso </url> <url type="http" location="ch" preference="10"> http://mirror.switch.ch/ftp/mirror/opensuse/distribution/openSUSE-current/is o/dvd/openSUSE-10.2-GM-DVD-i386.iso </url> </resources> </file>

OK. The problem is that with previously mentioned ,missing talk about ideas, it gives feeling that whatever you offer is proprietary. If you would mention anywhere that described method is patent free and content of web site is protected with some kind of opensource license than there will be no reasons for questions.

Metalink is patent free, unlike BitTorrent. (BT is awesome and the best thing to happen to the net since the web). The content of my website is under Creative Commons Attribution License. I believe this is the correct license for this type of thing, but I'm not very familiar w/ licenses. If someone knows of a more suitable one, please let me know.

...

...

3) Downloads are verified for enhanced reliability. -Nothing new for YaST users.

Cool. Not everyone uses or can use YaST in all situations :)

I agree. That is actually trivial and missing feature in many download products.

Yes, I am trying to encourage full file checksumming to be automated, as you can see. All the clients support this now. It may not be much, but it simplifies things for some people who aren't familiar with doing that. The partial file/segment checksums are also available and I am encouraging that as well - this will allow the automatic error correction. Some of the clients are adding this now. I guess it isn't as simple as the full file checksums. I don't know how far along aria2 is if at all, but if someone wanted to submit a patch that might not hurt. (Might want to contact the author or wait until the next release).

...

...

7) OpenOffice.org uses Metalinks. http://distribution.openoffice.org/p2p/magnet.html I'm removing listed magnet clients from any computer and explain users that they should not install them again. Do you think that anything listed on the same page will gain my trust?

I'm not sure what this means. Can you explain it more? Is there something untrustworthy about magnet clients? I wasn't aware of that. Metalink just happens to be listed on the same page by OpenOffice.org. If there is something wrong with them, I will try to get removed from that page.

Google returns links like this: http://www.wired.com/news/privacy/0,1848,49430,00.html http://news.com.com/2100-1038_3-5215028.html http://www.pestpatrol.com/spywarecenter/pest.aspx?id=453088059 (note versions) Could be also something like this: http://www.gnutellaforums.com/showthread.php?t=40161

So after some time spent in research I can understand why OP didn't understand my comment about magnet clients. It can be that it was just old version or hacked copies that gave me a lot of problems on few occasions.

Ok, glad you understand that bad clients that may have used magnet links have no connection with metalink.

I actually do not disagree with idea and I stated that more than one time, but your presentation of metalinks is missing details. I would like that you give some technical background how it works and to clear legal status. Is it proprietary technology or opensource?

You are the first person to request details. I assume people really interested read the spec I mentioned above. Most people don't care about that much detail. As I've mentioned before, metalink is not proprietary. I don't know how it could be open source, since I don't offer any actual programs or source. It's something developers have implemented in both proprietary and open source programs.

I have no problem with any technology as long as it is explained, software works as described and the price is right :-) but I have problem with any black boxes that do something, but even author can't explain details. That is, looking to some other posts, not only my impression.

If you're familiar with download managers and checksums then you already understand it. Obviously, there are no black boxes in the open source clients and a plain text XML file!

The rsync should not be necessary. It is proof that either metalink concept or implementation is not good in all details. BTW, I an use rsync alone and skip any repair.

rsync was offered as a temporary solution until aria2 can use the partial file checksums for error correction.

From Francis: Anyhow, at the moment I think, considering all the flaws of the current implementation, it's still hands-down the top method I'd use. I explained why. Fastest ever download speeds, and if you use rsync it's always guaranteed. Again, not perfect, but still very very good for me. Considering I'd perhaps only ever have to do rsync 1 in 100 times.

The client can only go with what you give it. If it's downloading from ftp/http, that's all its downloading from. There is just the automated checksum at the end, which will tell you if there were errors. I suppose the larger the file, the larger the chance of errors. If you're downloading an ISO over 2, 4 or 8 hrs, chances are probably higher that you will have an error in there somewhere. I'm sure there are lots of factors, but the bottom line is that for most people errors and rare but unavoidable with ftp/http. I've downloaded over a thousand files with metalinks, testing various stuff, and have only had one checksum error. I guess that's a testament to the quality of my connection. And thankfully it was a small couple hundred k file, not a 3G monster ISO. Once aria2 and other metalink clients support the partial file checksums, all this will be moot. I can't afford to pay people to work on this, so they will do it on their own time schedule. I'm sorry some people had problems, especially the file overwrite bug that someone ran into. Some other neat features which are still unsupported are inclusion of PGP signatures and some other stuff. (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Sunday 2006-12-17 at 18:45 -0600, Rajko M. wrote: ...

...

Some other neat features which are still unsupported are inclusion of PGP signatures and some other stuff.

That is what is necessary to verify source of files.

IMO, it would be suficient to sign the xml metalink file itself. As it contains the md5sum check of the image, that would enough to certify that what you downloaded was the correct signed file. Also, segment md5sums could be used to certify mirror sites: if a segment downloaded from a site doesn't check, and a retry fails again, that would mark that site as "bad" or bogus or whatever. An alternative is to sign the image, but that would be better done by the image provider/maker. Tricky problem! ;-) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFFhrQTtTMYHG2NR9URAhvNAKCIr1TxlGYOHHnuTlNg1lyXp9oOfACfWfWQ wnHKqD2rk6UAxCd/Ny1YQ5U= =unP4 -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

From Carlos:

The Sunday 2006-12-17 at 18:45 -0600, Rajko M. wrote:

...

...

Some other neat features which are still unsupported are inclusion of PGP signatures and some other stuff.

That is what is necessary to verify source of files.

IMO, it would be suficient to sign the xml metalink file itself. As it contains the md5sum check of the image, that would enough to certify that what you downloaded was the correct signed file.

That is definitely a possibility.

Also, segment md5sums could be used to certify mirror sites: if a segment downloaded from a site doesn't check, and a retry fails again, that would mark that site as "bad" or bogus or whatever.

Yes.

An alternative is to sign the image, but that would be better done by the image provider/maker.

Tricky problem! ;-)

Yes, hopefully whoever makes the ISO will sign them (if they're interested in this). Here's the currently unimplemented idea for metalink, which could change if the clients find out it is clumsy or bad once they start putting it into practice. For those unfamiliar, Linux Kernels (linux-2.6.19.1.tar.bz2) are signed with a separate file (linux-2.6.19.1.tar.bz2.sign) which looks like this: -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: See http://www.kernel.org/signature.html for info iD8DBQBFfdY4yGugalF9Dw4RAhpVAJ91gKuKakVYF8VPltPwalOi0/1WcQCfQR1A TAtxcYJjqogTkkvcUoTD34I= =68Ad -----END PGP SIGNATURE----- This can be easily inside the Metalink, like this: <verification> <signature type="pgp" file="linux-2.6.19.1.tar.bz2.sign"> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: See http://www.kernel.org/signature.html for info iD8DBQBFfdY4yGugalF9Dw4RAhpVAJ91gKuKakVYF8VPltPwalOi0/1WcQCfQR1A TAtxcYJjqogTkkvcUoTD34I= =68Ad -----END PGP SIGNATURE----- </signature> </verification> Or just listed as another file within the metalink <resources> <url>http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.19.1.tar.bz2.sign </url> </resources> At the least, a client could recognize when a signature is included in the metalink and tell the client to verify it manually gpg --verify linux-2.6.19.1.tar.bz2.sign linux-2.6.19.1.tar.bz2 But, I'm hoping it won't be too hard to automate this.

From Rajko:

The metalink might be good to distribute load on servers, but it needs improvements.

Now each client is testing servers independently. This produces some overhead that can be skipped if traffic data will be collected to central server from clients that start using metalink and after that metalink file will be changed to point to free resources ie. servers that show good performance.

The problem is not trivial, as change in metalink file will change distribution of the load and than reports will be changed. That means that we have feedback that has to be regulated to prevent system to go wild and lock everybody out.

Yes, there is currently no program that monitors the mirror servers and dynamically updates .metalink files according to mirror load. I see this as being really useful at the busiest times, but maybe not as critical during normal times. But I might be wrong. I do think it would be really cool if someone was interested and would work on it tho.

Another problem is that each client should use minimum number of connections ie. servers to reach maximum download speed and than stop asking for more as each server has its maximum, so it will prevent others to download.

Yes, aria2 uses 15 connections to 15 different servers by default AFAIR. This can be changed with command line options. Or if a distribution shipped a binary, they could decide on a number they felt was better. I believe all the other clients go by whatever options the user has set, (I don't know the defaults off hand for all of them). It would be nice if clients could recognize they were downloading at the users maximum speed for their connection with 2 connections, so please don't open up 13 more. It might be hard to fine tune what is most efficient. For dialup users, 1 is probably plenty. For me on cable, 4-6 is usually good in most conditions. I'd also like to find a number that is respectful to mirror server resources. (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On Tuesday 19 December 2006 05:33, Carlos E. R. wrote:

I saw aria2c starting at 15, and later dropping to 4 or 5 automatically, maxing throughput. That's working well, I think.

That means it is implemented to use just enough to have maximum speed. Good. -- Regards, Rajko. http://en.opensuse.org/MiniSUSE http://en.opensuse.org/Portal --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org

On Tue, 19 Dec 2006, Carlos E. R. wrote:

...

The "snag" is that checking both checksum and signature doubles checking time. I suppose that the user could select which one to use, same as rpm does, having both systems too, if I'm not mistaken.

A cryptographic signature should detect data corruption as well as a checksum -- after all it needs to detect willful tampering, not just technical corruption. So that should suffice, shouldn't it?

Definitely. To put this in context, we were talking about metalink which is an XML list of mirrors, checksums, and signatures for easier downloading/file distribution. metalinks can contain: partial file checksums for repairing a download full file checksums cryptographic signature The problem is that there are multiple clients on multiple operating systems. Almost all clients support MD5/SHA-1 full file checksums now. None of them support partial file checksums or signatures so far. I doubt the Windows and Mac clients will support PGP signatures, but maybe. It seems like the Linux clients will be more likely to have GPG and integrate it. So, a client that supports signatures could use the partial file checksums for errors in transfer, then just use the signature and skip the full file checksum. Ones that don't support signatures can use the partial and full file checksums. Support for metalink in KGet in KDE4 seems to be making progress. (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org