On Fri, Mar 5, 2021 at 8:40 AM Andrii Nikitin <andrii.nikitin@suse.com> wrote:
From: Neal Gompa <ngompa13@gmail.com>
MirrorManager is *able* to tolerate divergence, but that leads to a much bigger database since you wind up tracking all the files independently. That is a noted point you mention in the MirrorCache vs MirrorBrain differences.
Maybe you can give me a link to documentation or source code which clarifies this point, because I don't understand it properly. Or do you mean that this may be implemented in MirrorManager, but is not actually needed?
You can implement this in MirrorManager if you want to, but in practice it is less important if you're tracking only the necessary files that are a proxy for the state of the mirror (e.g. repodata index files and images like ISOs and disk images). MirrorManager currently implements tracking based on repodata index files for RPM and OSTree repositories[1]. For the openSUSE use-case, that would need to be extended to more types or given a flag to just track all files. There would probably need to be some minor adjustments to the code to treat openSUSE main and OBS content differently. MirrorManager has a heuristic to determine what stuff is present when it checks a mirror to auto enable/disable mirrors for different content subsets[2]. [1]: https://github.com/fedora-infra/mirrormanager2/blob/master/mirrormanager2/li... [2]: https://github.com/fedora-infra/mirrormanager2/blob/master/mirrormanager2/li...
Fedora's system is configured to track and invalidate based on the repomd.xml file, which makes it simple and relatively easy to determine whether a mirror is eligible. This is combined with the usage of metalinks by default so that the client can select multiple mirrors and use them for parallel downloads across different mirrors as needed, as well as automatic mirror failover.
You mentioned repomd.xml - does this assume that mirrors host only repositories?
Hmm, but the https://mirrorcache.opensuse.org/download/tumbleweed/repo/oss/repodata/repom... URL does not work.
zypper doesn't use such requests as far as I know, but such feature may/should be added for sure. I probably didn't properly prioritize it yet. But MirrorCache still lacks some features that MirrorBrain has (ASN matching, country-specific configuaration of mirrors, maybe more)
I was using it for rpm-repos-openSUSE[3] and Mock[4] but reverted it[5][6] because MirrorBrain's metalink handling is totally broken. [3]: https://build.opensuse.org/package/show/Base:System/rpm-repos-openSUSE [4]: https://github.com/rpm-software-management/mock [5]: https://build.opensuse.org/package/rdiff/Base:System/rpm-repos-openSUSE?linkrev=base&rev=4 [6]: https://github.com/rpm-software-management/mock/commit/a7d588c22872d71a2f956... -- 真実はいつも一つ！/ Always, there's only one truth!
On Fri, Mar 5, 2021 at 9:11 AM Neal Gompa <ngompa13@gmail.com> wrote:
On Fri, Mar 5, 2021 at 8:40 AM Andrii Nikitin <andrii.nikitin@suse.com> wrote:
From: Neal Gompa <ngompa13@gmail.com>
MirrorManager is *able* to tolerate divergence, but that leads to a much bigger database since you wind up tracking all the files independently. That is a noted point you mention in the MirrorCache vs MirrorBrain differences.
Maybe you can give me a link to documentation or source code which clarifies this point, because I don't understand it properly. Or do you mean that this may be implemented in MirrorManager, but is not actually needed?
You can implement this in MirrorManager if you want to, but in practice it is less important if you're tracking only the necessary files that are a proxy for the state of the mirror (e.g. repodata index files and images like ISOs and disk images). MirrorManager currently implements tracking based on repodata index files for RPM and OSTree repositories[1]. For the openSUSE use-case, that would need to be extended to more types or given a flag to just track all files.
There would probably need to be some minor adjustments to the code to treat openSUSE main and OBS content differently. MirrorManager has a heuristic to determine what stuff is present when it checks a mirror to auto enable/disable mirrors for different content subsets[2].
[1]: https://github.com/fedora-infra/mirrormanager2/blob/master/mirrormanager2/li... [2]: https://github.com/fedora-infra/mirrormanager2/blob/master/mirrormanager2/li...
Gah, I actually got these links backwards. Footnote 2 is the file tracking code, and footnote 1 is the heuristic.
Fedora's system is configured to track and invalidate based on the repomd.xml file, which makes it simple and relatively easy to determine whether a mirror is eligible. This is combined with the usage of metalinks by default so that the client can select multiple mirrors and use them for parallel downloads across different mirrors as needed, as well as automatic mirror failover.
You mentioned repomd.xml - does this assume that mirrors host only repositories?
Hmm, but the https://mirrorcache.opensuse.org/download/tumbleweed/repo/oss/repodata/repom... URL does not work.
zypper doesn't use such requests as far as I know, but such feature may/should be added for sure. I probably didn't properly prioritize it yet. But MirrorCache still lacks some features that MirrorBrain has (ASN matching, country-specific configuaration of mirrors, maybe more)
I was using it for rpm-repos-openSUSE[3] and Mock[4] but reverted it[5][6] because MirrorBrain's metalink handling is totally broken.
[3]: https://build.opensuse.org/package/show/Base:System/rpm-repos-openSUSE [4]: https://github.com/rpm-software-management/mock [5]: https://build.opensuse.org/package/rdiff/Base:System/rpm-repos-openSUSE?linkrev=base&rev=4 [6]: https://github.com/rpm-software-management/mock/commit/a7d588c22872d71a2f956...
-- 真実はいつも一つ！/ Always, there's only one truth!
-- 真実はいつも一つ！/ Always, there's only one truth!
participants (1)
-
Neal Gompa