On Tue, 2017-02-21 at 10:10 +0100, Niklas Edmundsson wrote:
Related to the heads up about rolling updates for the upcoming release:
Have you investigated speeding up your main rsync servers?
ftp.acc.umu.se syncs from stage.opensuse.org, and we are seeing sync-times in excess of 24 hours at times.
Sync-bandwidth on our end is limited to 1 Gbps, but the speeds we are getting are much lower so we suspect that the upstream server is the bottleneck.
If you can't increase the performance of the main rsync server it might be beneficial to investigate a tiered approach where a few select primary high-bandwidth mirrors sync from the master site and the rest sync from the closest primary mirror. This is the way Debian has been doing it for quite some time with great success.
/Nikke -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@acc.umu.se --------------------------------------------------------------------------- I parked my harddisk - and got a ticket! =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Hi Niklas, We have a somewhat tiered system. Registered mirrors are meant to be using stage.opensuse.org meanwhile anyone can use rsync.opensuse.org. We recently had a problem with sync between stage.o.o and rsync.o.o, which effectively negated the effectiveness of that second tier of mirrors and also led to uses falling back to stage.o.o in the event of not finding a better mirror. This problem could have been ongoing as long as the last few months. This obviously resulted in a lot more contention on stage.opensuse.org, making things worse for both our registered mirror sync and our users. We've resolved that problem and to prevent a repeat of users flooding stage.o.o we are going to be asking everyone registered to use stage.o.o to do so (I have the email drafted, but you beat me to the question). rsync.opensuse.org will remain as a public mirror source and should now be able to better fulfil it's intended role as the 'mirror of last resort' for our users instead of stage.o.o, leaving more of stage.o.o's bandwidth for mirror syncing. I hope that this resolution & rebalancing of the secondary/public mirror tier over the last few days will have already sped up what you're seeing with stage.opensuse.org.
From my personal monitoring it seems to be the case, but there is a small risk that we could be temporarily making things a little worse, if a lot more consistent.
But with a consistent baseline of all of our registered mirror hosts we can come up with a sensible strategy for a more robust & effectively tiering. Debian's way doesn't sound like a bad idea at all. Meanwhile, in parallel to everything I said above, we are actively working on increasing our bandwidth for stage.opensuse.org. Regards, -- Richard Brown Technical Lead - openQA openSUSE Chairman Phone +4991174053-361 SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org