Mailinglist Archive: mirror (38 mails)

< Previous Next >
Re: [suse-mirror] Opensuse master rsync server performance
On Wed, 22 Feb 2017, Carlos Carvalho wrote:

Lars Vogdt (Lars.Vogdt@xxxxxxxx) wrote on Wed, Feb 22, 2017 at 08:52:04AM BRT:
Hi

On Tue, 21 Feb 2017 10:10:39 +0100 (MET) Niklas Edmundsson wrote:
If you can't increase the performance of the main rsync server it
might be beneficial to investigate a tiered approach where a few
select primary high-bandwidth mirrors sync from the master site and
the rest sync from the closest primary mirror. This is the way Debian
has been doing it for quite some time with great success.

We have a few mirror admins that allowed us to push content to them
(simply via separate rsync module on their side).

Direct push-rsync with write access is generally frowned upon by mirror admins.

This isn't necessary. It's enough to trigger the updates via a ssh connection
that lauches the rsync from the mirror to the master. We do it with Debian.

Or you can do it the archlinux way and have a lastupdate timestamp that you download/check often and then trigger a full sync when it's changed. Sure, you get a slight delay compared to a true push syncscheme but it's much easier to set up (no additional auth steps). As long as your sync script uses a local lock file to avoid running multiple syncs concurrently you can run it say every 10 minutes without much harm.

See https://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors and https://git.server-speed.net/users/flo/bin/tree/syncrepo.sh for details. There are more projects using a timestamp file, but archlinux is the one that I came to think of first.

Their script uses curl to get the lastupdate file from the main mirror, which I personally think is a flaw since it can trigger a sync before the mirror you pull from has updated. It's easy enough to use rsync to pull the lastupdate file from the same mirror you sync from, a better option IMHO.

/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@xxxxxxxxxx
---------------------------------------------------------------------------
COMMITTEE: 12 people doing the work of one.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
--
To unsubscribe, e-mail: mirror+unsubscribe@xxxxxxxxxxxx
To contact the owner, email: mirror+owner@xxxxxxxxxxxx

< Previous Next >