Mailinglist Archive: mirror (38 mails)

< Previous Next >
Re: [suse-mirror] Opensuse master rsync server performance
On Thu, 23 Feb 2017, Lars Vogdt wrote:

See https://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors and
https://git.server-speed.net/users/flo/bin/tree/syncrepo.sh for
details. There are more projects using a timestamp file, but
archlinux is the one that I came to think of first.

Their script uses curl to get the lastupdate file from the main
mirror, which I personally think is a flaw since it can trigger a
sync before the mirror you pull from has updated. It's easy enough to
use rsync to pull the lastupdate file from the same mirror you sync
from, a better option IMHO.

Also an idea - but I need to check with the admins of the stage.o.o
server first. A problem might be that OBS alone has >80.000
repositories aka directories - that change all the time.

You'll have the same issue with a push-approach.

The workaround is to trigger the update when needed, be it via push or updating the lastupdate file.

By the way, it might be beneficial to point out to the developers that gazillions of files and directories that are to be kept up to date at all times isn't really a design that's easy to mirror/distribute...

"But it works on my laptop" isn't really a good argument, even though it's frequently used by devs ;-)

I've seen various efforts on partial updates of large repositories, generally with the master server providing lists of what's changed.

For example, I've been intending to look into setting up instant mirroring for CPAN (perl). See http://www.cpan.org/misc/how-to-mirror.html#Instant_mirroring for their take on solving the problem. Looks easy to use on the client side at least, I have no clue on what's needed on the server side...


But I'm sure now that we will find a solution that works for everyone.

Thanks for all your input!

Glad it's appreciated :-)


/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@xxxxxxxxxx
---------------------------------------------------------------------------
* <- Regular Tribble * <- Odo disguised as a tribble
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
--
To unsubscribe, e-mail: mirror+unsubscribe@xxxxxxxxxxxx
To contact the owner, email: mirror+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups