Eberhard Moenkeberg wrote:
Hi,
On Sat, 18 Mar 2006, Rajko M wrote:
Eberhard Moenkeberg wrote:
Torrent, according to their web site, is answer to overloaded central servers (FTP,HTTP). Even if there will be no security risks, than setting p2p seeder on FTP server is just against basic idea.
This sounds a little bit puristic. The spotted "server overload" phaenomenon is a matter with a duration of one day or even longer. Using ftp/http servers automatically ("programmed in") as initial seed could help a lot to get a p2p net running. Once there is enough "food" spreaded, the p2p contacts to ftp/http servers will reduce automatically.
That would be good, and not so puristic :-)
Plus design details explained in Peter Czanik email.
Yes; but I'm not sure about the relevance in total. If a server has 1000 "regular" ftp/http sessions anyways, I guess it would not matter to have 200 additional sessions which request very small packets.
I've seen Christian Boltz mail, and I can agree that in days after release iso files will be probably in RAM anyway, so mentioned I/O would be from RAM to network adapter, which in home computers is not a big deal, but how it is with dedicated server designs?
RAM is a limited resource... VERY limited, regarding today's distribution sizes... Each 10.1-beta is about 11 GB, the at the same time newly published factory tree is 33 GB. Together with 10.1-final (35 GB or more), 10.2-alpha1 will appear. Another 11 GB.
And from the mirror server's situation: let Debian, Mandrake, Slackware, FreeBSD release something new at the hot SUSE phase (seen one of them almost each "hot SUSE" time here): Debian ISOs are 60 GB, FreeBSD and Mandrake are similar in sizes of new releases.
A "release moment" of a SUSE distribution (even a beta release) is - since 10.0 - showing at ftp.gwdg.de as an increase of ftp sessions (>200 000 per day, almost doubled, http and rsync also increased), but reduced total output volume (1.5 TB per day usually, falling down to about 1 TB). This is due to the higher filesystem load, because the buffercache is not big enough any longer.
ftp.gwdg.de has 12 GB RAM, and it is NOT ENOUGH anymore since 10.0. It was enough until 9.x (top output was 850 MBit/sec), but 10.x and the new scheme is too much. Today, I seldom reach half of that. Last month was about 40 TB total, so not even the usual 1.5 TB/day.
My theory is: calculate the "focus" of the actual requests and add so much RAM that half of it can be covered. If you can't reach it, you will have some hard days with the need to limit the number of sessions, make single directories inaccessible for some time and so on.
Filesystem I/O is the limiting bottleneck if the network connection is good enough.
The "future strategy" at ftp.gwdg.de is:
1. increase RAM (needs new hardware with more RAM slots + RAM, very expensive) 2. increase disk I/O (needs additional raid arrays and I/O channels, almost as expensive as 1.)
Both steps have to work together, but each needs time (in fact it needs something totally different and is very difficult to achieve, but that results in needing time).
So a well-working p2p network really could help a bit.
But if it is not able to saturate the end user's line, lots of people will end using it. Guess what happens... So we need a good working p2p network or it is worth nothing.
I think that main problem, anyway, is how to convince people to use BitTorrent. Whoever wants to take that task can start with me. If that would be digital art, who cares for few bad bits, but this is computer program and I want to know much more in order to trust the p2p.
Good start would be to explain on opensuse wiki, what kind of p2p is BitTorrent, what are the security measures that are used, how much attention is given to client programs to prevent misuse trough security holes, etc.
Explaining is only one half - the other half is to help the p2p connections to "start better". The clients should - programmed in - use the existing ftp/http servers until the "seeding community" has grown enough.
Cheers -e
It is obvious that you gave a lot of thoughts how to improve downloads. I didn't really thought about volume increase with all new source trees that are simultaneously present. I'll see what I can do to convince myself to start using torrents, and give my 2 pfening. I did first experiments with clients today. I have to see why there is no uploads. It is not much, but you can remind me of old saying that tell something about pfenig by pfenig that makes a value. Azureus reported 900,000 users, so there is potential. Yes I know it is not all about opensuse, but the number is interesting. One with connection like mine will need 12 seeders, for full speed download. It can be interesting to try. -- Regards, Rajko.