Mailinglist Archive: mirror (38 mails)

< Previous Next >
Re: [suse-mirror] Opensuse master rsync server performance
Hi

On Tue, 21 Feb 2017 10:10:39 +0100 (MET) Niklas Edmundsson wrote:
Have you investigated speeding up your main rsync servers?

We tried that multiple times, but the limited amount of available
personal resources and bandwidth (beside the fact that we do not had
access to some core infrastructure resources like DNS and other stuff)
is not easily fixable for the openSUSE Heroes.

If you can't increase the performance of the main rsync server it
might be beneficial to investigate a tiered approach where a few
select primary high-bandwidth mirrors sync from the master site and
the rest sync from the closest primary mirror. This is the way Debian
has been doing it for quite some time with great success.

We have a few mirror admins that allowed us to push content to them
(simply via separate rsync module on their side). This worked fine until
we got all those tons of repositories: OBS (the open build service) was
only able to support one "push server" in the past. So the workflow
ended up like: OBS pushing to stage.opensuse.org and stage.opensuse.org
pushing to our dedicated "push mirrors" (while other mirrors still pull
from the same storage). After that, a scan is triggered to check if the
content on the mirrors is correct - and only after that, the content is
added to the MirrorBrain database, which is behind the redirector on
download.o.o

You might notice multiple bottle necks just by reading the paragraph
above. While the developer resources to fix issues in OBS and in
MirrorBrain (both open source, both welcome patches, btw) are still
low, we try fix other bottle necks that are available. Any ideas, any
support is welcome and we hope that we see some progress here and in
many other areas, too, in the next weeks.

With kind regards,
Lars *member_of_the_openSUSE_heroes_team*
--
To unsubscribe, e-mail: mirror+unsubscribe@xxxxxxxxxxxx
To contact the owner, email: mirror+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups
References