Mailinglist Archive: mirror (38 mails)

< Previous Next >
Re: [suse-mirror] Opensuse master rsync server performance
On Tue, 2017-02-21 at 10:10 +0100, Niklas Edmundsson wrote:
Related to the heads up about rolling updates for the upcoming
release:

Have you investigated speeding up your main rsync servers?

ftp.acc.umu.se syncs from stage.opensuse.org, and we are seeing
sync-times in excess of 24 hours at times.

Sync-bandwidth on our end is limited to 1 Gbps, but the speeds we are
getting are much lower so we suspect that the upstream server is the
bottleneck.

If you can't increase the performance of the main rsync server it
might be beneficial to investigate a tiered approach where a few
select primary high-bandwidth mirrors sync from the master site and
the rest sync from the closest primary mirror. This is the way Debian
has been doing it for quite some time with great success.


/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@xxxxxxxxxx
---------------------------------------------------------------------------
I parked my harddisk - and got a ticket!
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Hi Niklas,

We have a somewhat tiered system. Registered mirrors are meant to be using
stage.opensuse.org meanwhile anyone
can use rsync.opensuse.org.
We recently had a problem with sync between stage.o.o and rsync.o.o, which
effectively negated the
effectiveness of that second tier of mirrors and also led to uses falling back
to stage.o.o in the event of
not finding a better mirror.

This problem could have been ongoing as long as the last few months.

This obviously resulted in a lot more contention on stage.opensuse.org, making
things worse for both our
registered mirror sync and our users.

We've resolved that problem and to prevent a repeat of users flooding stage.o.o
we are going to be asking
everyone registered to use stage.o.o to do so (I have the email drafted, but
you beat me to the question).

rsync.opensuse.org will remain as a public mirror source and should now be able
to better fulfil it's intended
role as the 'mirror of last resort' for our users instead of stage.o.o, leaving
more of stage.o.o's bandwidth
for mirror syncing.

I hope that this resolution & rebalancing of the secondary/public mirror tier
over the last few days will have
already sped up what you're seeing with stage.opensuse.org.
From my personal monitoring it seems to be the case, but there is a small risk
that we could be temporarily
making things a little worse, if a lot more consistent.

But with a consistent baseline of all of our registered mirror hosts we can
come up with a sensible strategy
for a more robust & effectively tiering. Debian's way doesn't sound like a bad
idea at all.

Meanwhile, in parallel to everything I said above, we are actively working on
increasing our bandwidth for
stage.opensuse.org.

Regards,
--
Richard Brown
Technical Lead - openQA
openSUSE Chairman

Phone +4991174053-361
SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nuernberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)






--
To unsubscribe, e-mail: mirror+unsubscribe@xxxxxxxxxxxx
To contact the owner, email: mirror+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups
References