Hi On Tue, 21 Feb 2017 10:10:39 +0100 (MET) Niklas Edmundsson wrote:
Have you investigated speeding up your main rsync servers?
We tried that multiple times, but the limited amount of available personal resources and bandwidth (beside the fact that we do not had access to some core infrastructure resources like DNS and other stuff) is not easily fixable for the openSUSE Heroes.
If you can't increase the performance of the main rsync server it might be beneficial to investigate a tiered approach where a few select primary high-bandwidth mirrors sync from the master site and the rest sync from the closest primary mirror. This is the way Debian has been doing it for quite some time with great success.
We have a few mirror admins that allowed us to push content to them (simply via separate rsync module on their side). This worked fine until we got all those tons of repositories: OBS (the open build service) was only able to support one "push server" in the past. So the workflow ended up like: OBS pushing to stage.opensuse.org and stage.opensuse.org pushing to our dedicated "push mirrors" (while other mirrors still pull from the same storage). After that, a scan is triggered to check if the content on the mirrors is correct - and only after that, the content is added to the MirrorBrain database, which is behind the redirector on download.o.o You might notice multiple bottle necks just by reading the paragraph above. While the developer resources to fix issues in OBS and in MirrorBrain (both open source, both welcome patches, btw) are still low, we try fix other bottle necks that are available. Any ideas, any support is welcome and we hope that we see some progress here and in many other areas, too, in the next weeks. With kind regards, Lars *member_of_the_openSUSE_heroes_team* -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org