Mailinglist Archive: mirror (76 mails)

< Previous Next >
Re: [suse-mirror] Blogging about 11.3 launch - THANKS!
On Sun, Jul 18, 2010 at 12:32:28AM +0200, Marcus Meissner wrote:
On Sat, Jul 17, 2010 at 08:55:45PM +0200, Peter Poeml wrote:
pointing to variable edge servers depending on your location).

This cname cdn.novell.com is handled like any other mirror in
MirrorBrain, with the exception that it is not configured with a certain
country, but with a wildcard, which matches all countries. Thus it's
considered for requests from all countries. (For the files that it
serves, namely those few ISOs.)

After the first release day, this special mirror is simply switched off
in MirrorBrain, and from that point everything is redirected to the
mirrors as normal.

AFAIK 48 hours actually, so 2 days.

36 hours only, minus ample time (~4-6hours) to finish running downloads
before the file are removed from Akamai.

To explain this: the bandwidth is surplus bandwidth that is available to
Novell anyway (as a customer), and it donates this bandwidth to the
openSUSE project. It's a nice gesture albeit it's of arguable use. I
guess the motivation needs to be understood in a historic context; there
was a time before openSUSE had a good download redirector.

Yes and we really melted the mirrors at that time.

"melted the mirrors" simply because only 8 or 10 mirrors were used at
the time, as far as I remember. Worldwide. That's what the
infrastructure could handle.

Since then, MirrorBrain wasn't the only change. The other change was
that I persistently collected mirrors, one by one, day after day,
actively searching them, keeping them running. Now there are ~150
mirrors.

Probably coolo has the Akamai stats when he is back in the office on Monday
on how much they served out.

We can safely assume a maximum throughput of 20GB/s delivered via
Akamai. For a popular openSUSE release. (They could handle a hundred
times more, if need be.) This is to be regarded as a peak value,
occurring for some hours at most.

(GB, not GBit)

Now everybody can easily calculate: How many 1GBit/s mirrors does it
need to provide 20GB/s total throughput? :-)

~150.

However, only fast mirrors will have 1GBit (or more). The majority is
still 100 MBit I'd say. There aren't 150 1GBit mirrors.

So, offloading some requests to Akamai may indeed have its use during
the peak demand on the release day.

Only then, though. Outside this presumed peak, we have seen often
enough that the mirror network is clearly able to handle the demand.
During the peak, we don't know because it hasn't been tried out. I'd
say, it should be tried!

It is definitely not necessary to send _all_ traffic to Akamai. Of
course that makes it easy to get download counts for that first day.
However, those counts could also be accurately obtained directly on
download.opensuse.org. I implemented that for OpenOffice.org's download
server, and the elimination of skew by repetitive/partial requests seems
to work well. (Maybe it works better then Akamai's ;) I don't know how
they count exactly, but since their business is "big bucks", I guess
they know very well how to count bytes instead of requests.) IMO, it
would be interesting to have download counts at all times and not only
for day 1.

Peter
< Previous Next >
List Navigation
Follow Ups