Mailinglist Archive: opensuse-buildservice (375 mails)

< Previous Next >
Re: [opensuse-buildservice] Re: more statistics on repository (space) usage
  • From: Adrian Schröter <adrian@xxxxxxx>
  • Date: Thu, 17 Apr 2008 15:56:09 +0200
  • Message-id: <200804171556.09807.adrian@xxxxxxx>
On Thursday 17 April 2008 15:20:02 wrote Dr. Peter Poeml:
On Tue, Apr 15, 2008 at 07:07:37PM +0200, Dr. Peter Poeml wrote:
On Tue, Apr 15, 2008 at 04:31:54PM +0200, Detlef Steuer wrote:
May be it's enough to stop mirroring (but keeping!) end-of-live
products plus introducing some kind of quota system?

I see two aspects here, one is diskspace on download.opensuse.org,
another is diskspace on the mirrors.

The former is a decision done elsewhere, let's see what opensuse-project
thinks about it, the suggestion to move the discussion there is a good
idea.

The latter is something that we easily can take care about, with a
little bit of preparation.

I just gathered some data. What they show is the following:

- 10.1 distro usage:
It is not downloaded in any significant amount, we can definitely
remove it from all rsync modules right now (or rather move into a
separate "discontinued" rsync module so it is actually still
available). We actually could have done so earlier, and even though no
mirror so far complained about the size of those rsync modules, it
is always good to keep them small because in the end it might enable
more mirrors.

- 10.1 BS repositories usage:
The popular repos (KDE:, server:, OpenOffice.org, and others) are all
accessed with about 1 req/s. The overall average traffic generated
looks like only 25 K/s. It should nicely work to exclude all those
from the rsync modules, thereby shrinking them and removing them from
the mirrors, without impact on our download infrastructure.
[Overall request rate for BS repos is 80 req/s, 1 req/s of that is for
10.1 repos if I don't mistake anything.]

I gathered some more (build service specific) data, which may be of
interest. The following numbers are space occupied on rsync.opensuse.org
by repositories matching the following expressions:

*Mandriva_2006* 1G
*Mandriva_2007* 2G
*Mandriva_2008* 2G
*Debian* 0.5G
*Ubuntu* 1G

*Fedora_7* 5G
*Fedora_8* 8G
*RHEL_4* 0.1G
*RHEL_5* 2G

*SLES_9* 9G
*SLE_10* 31G

*10.0* 19G
*10.1* 45G
*10.2* 73G
*10.3* 88G
*Factory* 100G


TOTAL 393G
TOTAL w/o home 223G


So, these numbers allow for some interesting insight. However they'll be
fully useful together with statistics about downloads, and I'll wait for

But I _seriously_ wonder already now, whether we need to have 100G of
Factory rpms on the mirrors, which are probably downloaded only by
extremely few users.

A quick look into the logs indicates that the requests on the 100G
Factory packages cause _less_ of traffic for us, as the requests on the
10.1 packages...

does this include one or both Factory distribution trees itself ?

How did you calculate what repodir comes from which project ?

--

Adrian Schroeter
SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg)
email: adrian@xxxxxxx

---------------------------------------------------------------------
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-buildservice+help@xxxxxxxxxxxx

< Previous Next >
Follow Ups