Quoting Niklas Edmundsson
On Thu, 23 Feb 2017, Lars Vogdt wrote:
See https://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors and https://git.server-speed.net/users/flo/bin/tree/syncrepo.sh for details. There are more projects using a timestamp file, but archlinux is the one that I came to think of first.
Their script uses curl to get the lastupdate file from the main mirror, which I personally think is a flaw since it can trigger a sync before the mirror you pull from has updated. It's easy enough to use rsync to pull the lastupdate file from the same mirror you sync from, a better option IMHO.
Also an idea - but I need to check with the admins of the stage.o.o server first. A problem might be that OBS alone has >80.000 repositories aka directories - that change all the time.
You'll have the same issue with a push-approach.
The workaround is to trigger the update when needed, be it via push or updating the lastupdate file.
By the way, it might be beneficial to point out to the developers that gazillions of files and directories that are to be kept up to date at all times isn't really a design that's easy to mirror/distribute...
"But it works on my laptop" isn't really a good argument, even though it's frequently used by devs ;-)
I've seen various efforts on partial updates of large repositories, generally with the master server providing lists of what's changed.
For example, I've been intending to look into setting up instant mirroring for CPAN (perl). See http://www.cpan.org/misc/how-to-mirror.html#Instant_mirroring for their take on solving the problem. Looks easy to use on the client side at least, I have no clue on what's needed on the server side... We are using imm for that for a long time, works like a charm http://www.staff.science.uu.nl/~penni101/iim/ On the client it is very easy to setup, we used to rsync CPAN in the past. Since imm was installed we did not see long running rsyncs to CPAN anymore.
But I'm sure now that we will find a solution that works for everyone.
Thanks for all your input!
Glad it's appreciated :-)
/Nikke -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@acc.umu.se --------------------------------------------------------------------------- * <- Regular Tribble * <- Odo disguised as a tribble =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
-- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org