[heroes] JFYI: download.o.o ....
Hi Just want to inform you about the things that happened around our mirror infrastructure in the last days.... First some Statistics: * 74 billion file entries in the mirrorbrain database * 43 mirrors for openSUSE-Tumbleweed-DVD-i586-Current.iso * 31 mirrors for openSUSE-Tumbleweed-DVD-i586-Snapshot20170724-Media.iso * 62 mirrors for openSUSE-Leap-42.3-DVD-x86_64.iso * 73 mirrors for the Leap 42.3 oss repository * down to 4 open mirror tickets in progress.opensuse.org (3 of them really mirror related, feedback pending) * 207 active mirrors worldwide Done/doing: * cleanup of the apache configuration on download.o.o, which brought back the tumbleweed mirrors and removed old stuff (11.X, 12.X and also redirected all 13.X ) * finally removed 11.X, 12.X and 13.X from download.o.o - ~750G more space on download.o.o and our mirrors * rsync module cleanup on widehat and pontifex3 - the 13.X cleanup gave also more room for "other" stuff in the "hotstuff" modules * the DB-slave of mirrorbrain was out of sync with the DB-master - so the mirrorbrain database used by apache (which is using the read-only slave) was outdated. This was the major reason why we did not see enough mirrors for all repos. => currently running with the master only to avoid even the shortest downtime with kind regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Still not entirely fixed I don't think. e.g., when I open https://download.opensuse.org/tumbleweed/iso/ click on an iso's details link, then click on "Map showing the closest mirrors" I get a map with 4 markers, one of which ID's my location in FL (extreme SE USA), and 3 others, numbered 1, 2 & 4, all of which are located in Europe, and up to 10 times as far away as mirrors that I know exist in the USA[1]. https://sfo-korg-mirror.kernel.org/opensuse/distribution/leap/42.3/iso/ is in CA on the opposite side of CONUS, approximately 57% of the distance to the closest of those three mirrors on the other side of the Atlantic. [1] http://maps.google.com/maps/api/staticmap?size=640x512&visual_refresh=true&scale=2&maptype=roadmap&sensor=false&markers=color:red|27.629400,-82.703903&markers=color:yellow|label:1|53.000000,-8.000000&markers=color:yellow|label:2|50.771000,6.105000&markers=color:yellow|label:3|50.771000,6.105000&markers=color:yellow|label:4|49.448002,11.068000 -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Hi On Wed, 26 Jul 2017 19:04:53 -0400 Felix Miata wrote:
Still not entirely fixed I don't think. e.g., when I open https://download.opensuse.org/tumbleweed/iso/ click on an iso's details link, then click on "Map showing the closest mirrors" I get a map with 4 markers, one of which ID's my location in FL (extreme SE USA), and 3 others, numbered 1, 2 & 4, all of which are located in Europe, and up to 10 times as far away as mirrors that I know exist in the USA[1].
I might not understand you correctly, but if there is currently no mirror in the US that hosts the tumbleweed directory (the provo-mirror machine is currently syncing, but not finished), download.opensuse.org can simply not provide you with more mirrors.
https://sfo-korg-mirror.kernel.org/opensuse/distribution/leap/42.3/iso/ is in CA on the opposite side of CONUS, approximately 57% of the distance to the closest of those three mirrors on the other side of the Atlantic.
http://sfo-korg-mirror.kernel.org/opensuse/distribution/leap/42.3/iso/openSU... (us, prio 100) ^^^ the mirror is listed for Leap 42.3, but did not contain the "right" tumbleweed iso when it got scanned two days ago. I triggered a complete full scan of this mirror (which takes roughly 2h) to get the tumbleweed iso(s) listed back. Note: As we have ~200 mirrors and a full scan of each mirror takes ~2 hours (on average), I guess everyone can simply calculate the amount of time that is needed for a full scan of all mirror servers... I'm currently running up to 6 scans in parallel to get below 2 days, but as each scan eats up to 12Gb of RAM, I'm currently looking to get the scanning a bit more parallel by using multiple scanners. Here the problem starts that some mirrors only allow us to scan them from a specific IP address. I need to find out which mirrors can be scanned faster from the US or China - and if the connection to the database will not be the limitation in case I switch over to a "more closer" scanner in this countries. with kind regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
participants (2)
-
Felix Miata -
Lars Vogdt