Mailinglist Archive: mirror (38 mails)

< Previous Next >
Re: [suse-mirror] Opensuse master rsync server performance
Hi

On Thu, 23 Feb 2017 08:45:36 +0100 (MET) Niklas Edmundsson wrote:
We have a few mirror admins that allowed us to push content to them
(simply via separate rsync module on their side).

Direct push-rsync with write access is generally frowned upon by
mirror admins.

I like the Debian way. :-)
I will bring that up to the admins that maintain the stage.o.o machine.



Or you can do it the archlinux way and have a lastupdate timestamp
that you download/check often and then trigger a full sync when it's
changed. Sure, you get a slight delay compared to a true push
syncscheme but it's much easier to set up (no additional auth steps).
As long as your sync script uses a local lock file to avoid running
multiple syncs concurrently you can run it say every 10 minutes
without much harm.

See https://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors and
https://git.server-speed.net/users/flo/bin/tree/syncrepo.sh for
details. There are more projects using a timestamp file, but
archlinux is the one that I came to think of first.

Their script uses curl to get the lastupdate file from the main
mirror, which I personally think is a flaw since it can trigger a
sync before the mirror you pull from has updated. It's easy enough to
use rsync to pull the lastupdate file from the same mirror you sync
from, a better option IMHO.

Also an idea - but I need to check with the admins of the stage.o.o
server first. A problem might be that OBS alone has >80.000
repositories aka directories - that change all the time.

But I'm sure now that we will find a solution that works for everyone.

Thanks for all your input!

With kind regards,
Lars







--
To unsubscribe, e-mail: mirror+unsubscribe@xxxxxxxxxxxx
To contact the owner, email: mirror+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups