Mailinglist Archive: mirror (76 mails)

< Previous Next >
Re: [suse-mirror] Blogging about 11.3 launch - THANKS!
On Sun, Jul 25, 2010 at 05:46:29PM +0200, Carsten Otto wrote:
Dear Keld and all,

On Fri, Jul 23, 2010 at 06:24:51PM +0200, Keld Simonsen wrote:
It gave a download speed of 1.15 MB/S or 9 Mbit/s with wget.

That is not much. How good is your connection to the German/European
research network? How fast in general is your connection to Germany? I
don't think we can deliver 10 GBit/sec to Mexico, just to give some
example. Maybe this is problematic for Denmark, too.

My machines there are on the Danish research network, and close to the
Danish Internet exchange (DIX) - some 100 meters from it. We have 10 Gbit
on the university campus, but not (yet) to our machines.
I would think there is good connection
to German research network - I would think it would be a multi gigabit
connection. It should be good for multi 100 mbit/s traffic. - not just 10 Mbit.

But of coarse there can be bottlenecks - I am not sure where they are
and I am interested in finding out.

You have a 10 Gbit/s line and the overall bandwidth load at your site was
about 1.5 Gbit/s.
So there should be enough excessive bandwidth to play with.

Yes.

Do you have a limit on the line speed per connection?

No, there are not real limits. In my tests it is possible to get 1
GBit/sec with many machines at the same time.

We are talking about 2 things here:

1. the total bandwidth consumed at a server machine
2. the obtainable bandwidth for an extra machine (a user).

I am interested in both aspects.

Do you get 1 Gbit/s from a testing client machine towards your server?
Do you get it at this time (Sunday afternoon)?


http://ftp.klid.dk/ftp/opensuse/distribution/11.3/iso/openSUSE-11.3-KDE4-LiveCD-i686.iso

This was with wget 29.8 MB/s or about 240 Mbit/s - my system has a 1 Gbit/s
connection
and the load was 100 Mbit/s.

I just tried downloading it from ftp.halifax.rwth-aachen.de and got
about 4,7 MByte/sec (roughly 50 MBit/sec). So I guess the connection
between our servers is not the best.


Hmm, I think you should not try benchmarking from your ftp
machine, it could be that your file systems are very busy.
I think your tests on your own machine was conducted on a client machine?

I then tried to download via bittorrent from your site
http://ftp.halifax.rwth-aachen.de/opensuse/distribution/11.3/iso/openSUSE-11.3-GNOME-LiveCD-i686.iso.torrent

It was something like 3.2 MB/s = 25 Mbit/s. Better than 9 Mbit/s
So is this why your users use BT? They know it is faster than ftp?

I don't know why users actually use BitTorrent. It should be faster,
because many peers participate and provide upstream. This is the theory
and for hot files (think warez?) this works out just fine. We can reach
more than 1 GBit/sec with BitTorrent, there is no real difference to the
HTTP download you tried - I think it suffered the same problem. Besides,
with BitTorrent you need to get the right peer list. Maybe you did not
even download a single byte from our server.

Hmm, how do I control the peer list to be just from your ftp site?

I did not dee any big visible advertisement of the torrents.

Try www.opensuse.org and www.ubuntu.com (or whereever they announce
their downloads). We as a mirror do not advertise, we just seed
(upload).

So you are just part of the general opensuse or ubuntu seed?
That should not be too difficult for me to do also.

We are seeding where we feel like it. This is mostly OpenSUSE, Ubuntu
and CentOS, as far as I remember.

On Fri, Jul 23, 2010 at 03:07:21PM -0300, Carlos Carvalho wrote:
How did you get this number?

http://ftp.halifax.rwth-aachen.de shows some basic stats and
http://ftp.halifax.rwth-aachen.de/~cotto/ gives more in-depth
statistics.


On Sat, Jul 24, 2010 at 10:46:48AM +0200, Keld Simonsen wrote:
My link for the testing machine running the BT client is 1 Gbit/s.
So it should not be the bottleneck.

1 GBit/sec is not the universal answer. I guess your international
routing is not well-equipped.

I don't know how well equipped things are here, I am not in charge of
routing. Anyway I would expect it to be quite standard, they run a 10 Gbit
backbone at the university campus, and they are hosting the national internet
exchange, wgeer they then also have a number of international lines.
Are you talking about international connection bandwidth?

You should not judge other servers or
protocols if you can't manage to download _anything_ from _some_ server
located in the same country or region with the speed you want.

Well, I am not trying to blame anybody. I am just puzzled over some things.
And then I am asking people on this list as I think they are quite
knowledgeable in what is going on.

I do get fine speeds from other servers in Denmark, say 200 - 300 Mbit/s.

So you are saying that the bottlenecks are not the ftp servers, but rather
the national infrastructure, at least in some cases. This could give med
some insights in what mirrors to chose for rsyncing, eg. I should
prefer Danish or Nordic servers to European servers. And BT could also
give and advantage.

This also have inpact on normal users, many users have 20 - 50 - 100 Mbit/s
download
connectivity, and having only 10 Mbit/s because of national infrastructure
issues
could make the users want to have particular priorities.


Yes, I cannot ensure that the 25 Mbit/s is only coming from the
ftp.halifax.rwth-aachen.de
machine. But I think this is how the BT that they operate is supposed to
work. How else could I measure the BT speed for the halifax seeds?

Most clients show speed per client (or IP). The IP for our server
currently is 137.226.34.42. If this appears in your peer list, you can
check how fast we upload to your client.

good!

Anyway it is the user experience I am interested in.

We made the experience that a huge number of users actually use
BitTorrent. We uploaded about 5 TByte of the 11.3 OpenSUSE DVDs only via
BitTorrent, that is roughly 500-600x per DVD. With CentOS 5.5 I see 2.5
TByte in Total and roughly 300x uploads per DVD. This is peanuts
compared to the HTTP traffic (30 TByte in one day for Mozilla!), but
still significant enough so that it makes sense to support BitTorrent.

So what are the main types of traffic for you? 30 TB a day is almost
3 gigabit...

Thanks for your advice. I will have a look into it.

best regards
keld
--
To unsubscribe, e-mail: mirror+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: mirror+help@xxxxxxxxxxxx

< Previous Next >
List Navigation