On 30/03/2021 11.08, Gerald Pfeifer wrote:
On Thu 2021-03-18, medwinz wrote:
My logic is simple, mirrors cannot catch up with
the new release if
several major updates happen at nearly the same time.
That, plus the timing of announcements also makes a difference.
Not everyone here probably follows the factory@ list closely (I,
too, read project@ more regularly), so allow me to share a change
Bernhard has recently made which should also help.
Thank you, Bernhard!
For the record, here is what I found during debugging:
After the previous full rebuilds,
IPv6 address had ~50% packet loss for several
hours. My nagios has reported this at
2021-02-16 12:20 UTC (20210212 glibc rebuild published)
2021-03-08 11:25 UTC (20210307 glibc rebuild published)
2021-03-12 18:35 UTC (20210311 KDE update published)
2021-03-13 10:37 UTC (still fallout from 20210311?)
2021-03-13 18:09 UTC (still fallout from 20210311?)
On the download.o.o VM, top showed ksoftirqd very busy at 98% CPU and
several active rsyncd processes using the CPU.
This indicates that syncing to the mirrors was still ongoing and that
probably also slowed mirrorbrain mirror scans, so that more traffic was
directed towards the downloadcontent.o.o site which happens to be the
same machine as mirrorbrain (just another IP). All that syncing,
scanning and downloading slowed each other down.
Apart from the longer publish delay for large Tumbleweed snapshots, I
plan to push the setup of the new widehat.o.o machine, so that it can
act as a fallback mirror most of the time and relieve
==== forwarded message ====
From: Bernhard M. Wiedemann <bernhardout(a)lsmod.de>
Date: Sun, 28 Mar 2021 15:09:28 +0200
Subject: factory publish delay [was New Tumbleweed snapshot 20210325]
20210325 was the first large snapshot to trigger my new logic for a
longer publish delay (6h instead of 2h).
The publisher became active when the repo age was 22203 s
and the repo rsync log had 32114 lines and 2091370 byte.
This time, download.o.o seems to have had a much better time handling
the load than with previous full rebuilds. Though this seems to have
been only a partial rebuild because there are 62802 entries in
> Medium-term we can do more fine-tuning to balance publish delay vs load
> of download.o.o
> Bernhard M.