Andrii Nikitin wrote:
In my understanding MirrorBrain is lacking proper job queue and adding it + WebUI would result in managing two different projects inside single github repository. This is why we have two projects. I would be glad to do full help if somebody had a good vision how to fix it in mirrorbrain.
Thinking out loud - the two key shortcomings, https and ipv6 - are simply two more criteria for the mirror selection, nothing else. That sounds to me like some database changes for storing the mirror attributes, plus the appropriate changes to the selection algorithm.
If we implement it properly, we must regularly probe every mirror for each attribute: http, https, ipv4 and ipv6 (because certificates are added, expire, network changes, etc).
Without having thought it through - isn't that pretty much what the scanner does today? it checks a mirror to see what is available, if an address is bad or a certificate doesn't work, the availability is updated. A poorly configured mirror should not interfere with the proper working of the rest of mirrorbrain. It would however be nice to be a little more fine-grained and record which parts of a mirror are working: ipv4, http ipv4, https ipv6, http ipv6, https In comparison to the current setup, this is just like having four mirrors instead of one. Maybe that is the way to look at the design.
And then we must track that properly and consider when matching client request. This is actually what the PR does.
The scanning, the status keeping and the selection algorithm?
But MirrorBrain also has locking problems when two scanners scan the same folder on the same or differet mirrors, which is hard to track and troubleshoot.
We scan regularly on olaf, and pontifex scans an individual mirror after each push of OBS repos. New mirrors are scanned before we enable them. I have not looked at the scanning setup in any detail, but is there any significant risk of two scanners colliding ? If these locking problems occur frequently, we have to look at that, agreed. -- Per Jessen, Zürich (4.8°C) Member, openSUSE Heroes