Dear mirror admin,
TLDR:
1. If you use 15.4 / rsync-3.2.3 with --delay-updates, beware of
https://bugzilla.opensuse.org/show_bug.cgi?id=1204538
2. If you mirror only our openSUSE distribution, history, source,
tumbleweed, or update directories, you can get them faster now.
If you need debug, ports, or repositories: keep rsyncing from stage.o.o
as before.
What was the problem?
stage.o.o regularly filled the available network bandwidth of 4 GBit/s
and also had limited disk-IO. Thus much of the time, half of the rsync
processes on stage.o.o got below 3MByte/s of updates.
At that speed, getting a moderately sized 40GByte Tumbleweed snapshot
out could easily take 4 hours.
What is the solution?
SUSE sponsored a new machine stage3 aka stage-main-repos.o.o that has
fast SSDs and a 10G uplink. I have it setup with the same rsyncd config
as the main stage.o.o.
To use it as a mirror admin, you just need to pull from
rsync://stage-main-repos.opensuse.org/
Please only use the stage-main-repos name, because the service might
move to a different host next year.
Note, that the "ports" and "repositories" dirs on stage-main-repos.o.o
have only a small collection (150GB each) of repos that are more likely
to be requested by users. Stay on stage.o.o, if you want the full thing.
stage-main-repos.o.o will have below 5 minutes delay to pick up first
new files and will ensure consistent repo state at all times.
Besides this, I saw with 'mtr' that during some hours of the day,
trans-ocean links were saturated (i.e. high jitter at New York
ae0-3356.nyk10.core-backbone.com hop). If you notice such network
bottleneck, you could find another regional mirror to rsync from them.
Also, rsync://provo-mirror.opensuse.org/ is available as main staging
point for the Americas, but with less bandwidth.
Furthermore, I added a line about how the --ignore-existing rsync option
can help compensate downsides of --delay-updates in
https://en.opensuse.org/openSUSE:Mirror_howto
It might get even better once
https://bugzilla.opensuse.org/show_bug.cgi?id=1204538 is fixed.
kind regards
Bernhard M. Wiedemann
