Zypper with MirrorCache pilot project
(Note: this email was sent weeks ago, but it looks it didn't come trough). Hi all, As part of improvements to download.opensuse.org experience, currently new pilot service is available for using with zypper. https://mirrorcache.opensuse.org Its primary task is to redirect zypper requests to a mirror in client's country, similar to what MirrorBrain project does. You can try to replace in /etc/zypp/repos.d/*.repo http://download.opensuse.org with http://mirrorcache.opensuse.org or https://mirrorcache.opensuse.org If you are in Europe or America, the best experience is expected with these addresses: mirrorcache-eu.opensuse.org<https://mirrorcache.opensuse.org> (Europe) mirrorcache-na.opensuse.org<https://mirrorcache.opensuse.org> (North America) Since it is still a pilot, the service comes without any guarantee or obligations. Reasons to try it may have those who: - live in North America and don't want to be redirected to send cross-continent requests when possible; - want to stick to https connection (mirrorcache will try to find a mirror in the same country, which supports https); - are on ipv4-only or ipv6-only connection (mirrorcache will try to find a mirror which supports ipv4 or ipv6, depending on the request's connection); - redirection from d.o.o leads to a mirror, which actually doesn't have the requested file (it is rare, but can happen); - want to try hosting a mirrorcache instance for own location (e.g. organization, country or continent) and enjoy fast and reliable redirection. (with very modest disk space / HW requirements). Additional reasons may have those who: - want to help with improving d.o.o experience and provide feedback; - are interested in mirror management, for which admin's UI is available, and non-admin UI is planned (so a user can add and manage own mirror without global admin rights); https://mirrorcache.opensuse.org/app/server (login using UI menu and ask me for admin rights if you want to use it or see more advanced controls, like job details). - need some related functionality: it is possible that it can be easily achievable with mirrorcache. Further read about differences from MirrorBrain: https://github.com/andrii-suse/MirrorCache/blob/master/doc/mb_compare.md How caching works: - caching happens on folder level, so e.g., when zypper tries to access file, unknown for mirrorcache - it will be redirected to d.o.o, then a background job will collect info about mirrors. Thereafter further requests to the same folder will be properly redirected to a mirror in the client's country, honoring http/https scheme and ipv4/ipv6 connection used: https://raw.githubusercontent.com/andrii-suse/MirrorCache/master/doc/flow.sv... - requests to remodata/repomd.xml are not cached, so zypper always gets the latest version of packages. If you are interested in current source code (most of architecture is stolen from OpenQA): https://github.com/andrii-suse/MirrorCache In case of questions or any feedback do not hesitate to contact me. Regards, -- Andrii Nikitin <andrii.nikitin@suse.com> DevOPS Automation and Build Service Engineer SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 247165, AG München) Managing Director: Felix Imendörffer
Sorry for the typo it had to be mirrorcache-us.opensuse.org<https://mirrorcache.opensuse.org> instead of mirrorcache-na.opensuse.org<https://mirrorcache.opensuse.org> ________________________________ From: Andrii Nikitin <andrii.nikitin@suse.com> Sent: Friday, March 5, 2021 11:37 AM To: heroes@lists.opensuse.org <heroes@lists.opensuse.org> Subject: Zypper with MirrorCache pilot project (Note: this email was sent weeks ago, but it looks it didn't come trough). Hi all, As part of improvements to download.opensuse.org experience, currently new pilot service is available for using with zypper. https://mirrorcache.opensuse.org Its primary task is to redirect zypper requests to a mirror in client's country, similar to what MirrorBrain project does. You can try to replace in /etc/zypp/repos.d/*.repo http://download.opensuse.org with http://mirrorcache.opensuse.org or https://mirrorcache.opensuse.org If you are in Europe or America, the best experience is expected with these addresses: mirrorcache-eu.opensuse.org<https://mirrorcache.opensuse.org> (Europe) mirrorcache-na.opensuse.org<https://mirrorcache.opensuse.org> (North America) Since it is still a pilot, the service comes without any guarantee or obligations. Reasons to try it may have those who: - live in North America and don't want to be redirected to send cross-continent requests when possible; - want to stick to https connection (mirrorcache will try to find a mirror in the same country, which supports https); - are on ipv4-only or ipv6-only connection (mirrorcache will try to find a mirror which supports ipv4 or ipv6, depending on the request's connection); - redirection from d.o.o leads to a mirror, which actually doesn't have the requested file (it is rare, but can happen); - want to try hosting a mirrorcache instance for own location (e.g. organization, country or continent) and enjoy fast and reliable redirection. (with very modest disk space / HW requirements). Additional reasons may have those who: - want to help with improving d.o.o experience and provide feedback; - are interested in mirror management, for which admin's UI is available, and non-admin UI is planned (so a user can add and manage own mirror without global admin rights); https://mirrorcache.opensuse.org/app/server (login using UI menu and ask me for admin rights if you want to use it or see more advanced controls, like job details). - need some related functionality: it is possible that it can be easily achievable with mirrorcache. Further read about differences from MirrorBrain: https://github.com/andrii-suse/MirrorCache/blob/master/doc/mb_compare.md How caching works: - caching happens on folder level, so e.g., when zypper tries to access file, unknown for mirrorcache - it will be redirected to d.o.o, then a background job will collect info about mirrors. Thereafter further requests to the same folder will be properly redirected to a mirror in the client's country, honoring http/https scheme and ipv4/ipv6 connection used: https://raw.githubusercontent.com/andrii-suse/MirrorCache/master/doc/flow.sv... - requests to remodata/repomd.xml are not cached, so zypper always gets the latest version of packages. If you are interested in current source code (most of architecture is stolen from OpenQA): https://github.com/andrii-suse/MirrorCache In case of questions or any feedback do not hesitate to contact me. Regards, -- Andrii Nikitin <andrii.nikitin@suse.com> DevOPS Automation and Build Service Engineer SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 247165, AG München) Managing Director: Felix Imendörffer
On Fri, Mar 5, 2021 at 5:37 AM Andrii Nikitin <andrii.nikitin@suse.com> wrote:
(Note: this email was sent weeks ago, but it looks it didn't come trough).
Hi all,
As part of improvements to download.opensuse.org experience, currently new pilot service is available for using with zypper.
https://mirrorcache.opensuse.org
Its primary task is to redirect zypper requests to a mirror in client's country, similar to what MirrorBrain project does.
You can try to replace in /etc/zypp/repos.d/*.repo http://download.opensuse.org with http://mirrorcache.opensuse.org or https://mirrorcache.opensuse.org
If you are in Europe or America, the best experience is expected with these addresses: mirrorcache-eu.opensuse.org (Europe) mirrorcache-na.opensuse.org (North America)
Since it is still a pilot, the service comes without any guarantee or obligations.
Reasons to try it may have those who: - live in North America and don't want to be redirected to send cross-continent requests when possible; - want to stick to https connection (mirrorcache will try to find a mirror in the same country, which supports https); - are on ipv4-only or ipv6-only connection (mirrorcache will try to find a mirror which supports ipv4 or ipv6, depending on the request's connection); - redirection from d.o.o leads to a mirror, which actually doesn't have the requested file (it is rare, but can happen); - want to try hosting a mirrorcache instance for own location (e.g. organization, country or continent) and enjoy fast and reliable redirection. (with very modest disk space / HW requirements).
Additional reasons may have those who: - want to help with improving d.o.o experience and provide feedback; - are interested in mirror management, for which admin's UI is available, and non-admin UI is planned (so a user can add and manage own mirror without global admin rights); https://mirrorcache.opensuse.org/app/server (login using UI menu and ask me for admin rights if you want to use it or see more advanced controls, like job details). - need some related functionality: it is possible that it can be easily achievable with mirrorcache.
Further read about differences from MirrorBrain: https://github.com/andrii-suse/MirrorCache/blob/master/doc/mb_compare.md
How caching works: - caching happens on folder level, so e.g., when zypper tries to access file, unknown for mirrorcache - it will be redirected to d.o.o, then a background job will collect info about mirrors. Thereafter further requests to the same folder will be properly redirected to a mirror in the client's country, honoring http/https scheme and ipv4/ipv6 connection used: https://raw.githubusercontent.com/andrii-suse/MirrorCache/master/doc/flow.sv... - requests to remodata/repomd.xml are not cached, so zypper always gets the latest version of packages.
If you are interested in current source code (most of architecture is stolen from OpenQA): https://github.com/andrii-suse/MirrorCache
In case of questions or any feedback do not hesitate to contact me.
This looks very cool, but I'm confused why we would do this instead of using MirrorManager2[1] and collaborating on that project? The documentation showing the differences between MirrorCache and MirrorBrain seem to mirror the differences between MirrorManager2 and MirrorBrain based on my memory of discussing this at oSC two years ago. Moreover, MirrorManager2 already has self-service management of mirrors. The main difference between MirrorManager2 and MirrorCache seems to be that MirrorCache lacks metalink support, which I think we would actually *want* to have. Are there any other specific features that MirrorCache has that I'm missing? [1]: https://github.com/fedora-infra/mirrormanager2/ -- 真実はいつも一つ！/ Always, there's only one truth!
Hi Neal, Correct me if I am wrong - MirrorManager expects all mirrors have the same content, while MirrorBrain and MirrorCache tolerate divergents and are able to use from them whatever is available. Actually MirrorCache does support metalinks, just the core problem is that it doesn't have access to exact file sizes yet. You can try e.g. : curl -H 'Accept: */*, application/metalink+xml' -s https://mirrorcache-eu.opensuse.org/download/tumbleweed/repo/oss/x86_64/2048... Regards, Andrii Nikitin ________________________________ From: Neal Gompa <ngompa13@gmail.com> This looks very cool, but I'm confused why we would do this instead of using MirrorManager2[1] and collaborating on that project? The documentation showing the differences between MirrorCache and MirrorBrain seem to mirror the differences between MirrorManager2 and MirrorBrain based on my memory of discussing this at oSC two years ago. Moreover, MirrorManager2 already has self-service management of mirrors. The main difference between MirrorManager2 and MirrorCache seems to be that MirrorCache lacks metalink support, which I think we would actually *want* to have. Are there any other specific features that MirrorCache has that I'm missing? [1]: https://github.com/fedora-infra/mirrormanager2/ -- 真実はいつも一つ！/ Always, there's only one truth!
On Fri, Mar 5, 2021 at 8:01 AM Andrii Nikitin <andrii.nikitin@suse.com> wrote:
Hi Neal,
Correct me if I am wrong - MirrorManager expects all mirrors have the same content, while MirrorBrain and MirrorCache tolerate divergents and are able to use from them whatever is available.
MirrorManager is *able* to tolerate divergence, but that leads to a much bigger database since you wind up tracking all the files independently. That is a noted point you mention in the MirrorCache vs MirrorBrain differences. Fedora's system is configured to track and invalidate based on the repomd.xml file, which makes it simple and relatively easy to determine whether a mirror is eligible. This is combined with the usage of metalinks by default so that the client can select multiple mirrors and use them for parallel downloads across different mirrors as needed, as well as automatic mirror failover.
Actually MirrorCache does support metalinks, just the core problem is that it doesn't have access to exact file sizes yet. You can try e.g. : curl -H 'Accept: */*, application/metalink+xml' -s https://mirrorcache-eu.opensuse.org/download/tumbleweed/repo/oss/x86_64/2048...
Hmm, but the https://mirrorcache.opensuse.org/download/tumbleweed/repo/oss/repodata/repom... URL does not work. -- 真実はいつも一つ！/ Always, there's only one truth!
Neal Gompa wrote:
This looks very cool, but I'm confused why we would do this instead of using MirrorManager2[1] and collaborating on that project?
I agree it sounds cool, but I remain confused about the purpose of mirrorcache, as well as why the effort wasn't directed at mirrorbrain instead. MirrorBrain currently has only two real shortcomings - IPv6 and https support. Fixing these as well as other more or less imagined flaws with a bolt-on just sounds like a kludge to me. The issue of clients in North America being redirected to other continents really only applies to the OBS repositories for which we simply don't have enough mirrors in the Americas. (one was recently been added in Uruguay though). -- Per Jessen, Zürich (5.0°C) Member, openSUSE Heroes
On Fri, Mar 5, 2021 at 8:05 AM Per Jessen <per@opensuse.org> wrote:
Neal Gompa wrote:
This looks very cool, but I'm confused why we would do this instead of using MirrorManager2[1] and collaborating on that project?
I agree it sounds cool, but I remain confused about the purpose of mirrorcache, as well as why the effort wasn't directed at mirrorbrain instead. MirrorBrain currently has only two real shortcomings - IPv6 and https support. Fixing these as well as other more or less imagined flaws with a bolt-on just sounds like a kludge to me.
The issue of clients in North America being redirected to other continents really only applies to the OBS repositories for which we simply don't have enough mirrors in the Americas. (one was recently been added in Uruguay though).
It definitely applies to openSUSE as a whole. I *rarely* get directed to an American mirror, I usually get something in Eastern Europe or Asia, which is frustrating because they often time out. -- 真実はいつも一つ！/ Always, there's only one truth!
Neal Gompa wrote:
On Fri, Mar 5, 2021 at 8:05 AM Per Jessen <per@opensuse.org> wrote:
Neal Gompa wrote:
This looks very cool, but I'm confused why we would do this instead of using MirrorManager2[1] and collaborating on that project?
I agree it sounds cool, but I remain confused about the purpose of mirrorcache, as well as why the effort wasn't directed at mirrorbrain instead. MirrorBrain currently has only two real shortcomings - IPv6 and https support. Fixing these as well as other more or less imagined flaws with a bolt-on just sounds like a kludge to me.
The issue of clients in North America being redirected to other continents really only applies to the OBS repositories for which we simply don't have enough mirrors in the Americas. (one was recently been added in Uruguay though).
It definitely applies to openSUSE as a whole. I *rarely* get directed to an American mirror, I usually get something in Eastern Europe or Asia, which is frustrating because they often time out.
I guess we could do with a ticket for that - we have plenty of mirrors in the US, thirty of them or thereabouts. If you are frequently directed to another continent for non-OBS downloads, something is wrong with the geo-location. Seriously, open a ticket and include your typical IP-address. Or maybe just post a mirrorlist where this can be seen? -- Per Jessen, Zürich (5.1°C) Member, openSUSE Heroes
From: Per Jessen <per@opensuse.org>
I agree it sounds cool, but I remain confused about the purpose of mirrorcache, as well as why the effort wasn't directed at mirrorbrain instead. MirrorBrain currently has only two real shortcomings - IPv6 and https support. Fixing these as well as other more or less imagined flaws with a bolt-on just sounds like a kludge to me.
I did try to do it in MirrorBrain e.g. see https://github.com/openSUSE/mirrorbrain/pull/43 But it didn't come through, I'd say because nobody had enough drive and resources and good understanding how it can be achieved. In my understanding MirrorBrain is lacking proper job queue and adding it + WebUI would result in managing two different projects inside single github repository. This is why we have two projects. I would be glad to do full help if somebody had a good vision how to fix it in mirrorbrain.
The issue of clients in North America being redirected to other continents really only applies to the OBS repositories for which we simply don't have enough mirrors in the Americas. (one was recently been added in Uruguay though).
We may have geo database outdated, MirrorCache has a route to check that: http://mirrorcache.opensuse.org/rest/myip Regards, -- Andrii Nikitin <andrii.nikitin@suse.com> DevOPS Automation and Build Service Engineer SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 247165, AG München) Managing Director: Felix Imendörffer
Andrii Nikitin wrote:
From: Per Jessen <per@opensuse.org>
I agree it sounds cool, but I remain confused about the purpose of mirrorcache, as well as why the effort wasn't directed at mirrorbrain instead. MirrorBrain currently has only two real shortcomings - IPv6 and https support. Fixing these as well as other more or less imagined flaws with a bolt-on just sounds like a kludge to me.
I did try to do it in MirrorBrain e.g. see https://github.com/openSUSE/mirrorbrain/pull/43 But it didn't come through, I'd say because nobody had enough drive and resources and good understanding how it can be achieved.
That is certainly a shame.
In my understanding MirrorBrain is lacking proper job queue and adding it + WebUI would result in managing two different projects inside single github repository. This is why we have two projects. I would be glad to do full help if somebody had a good vision how to fix it in mirrorbrain.
Thinking out loud - the two key shortcomings, https and ipv6 - are simply two more criteria for the mirror selection, nothing else. That sounds to me like some database changes for storing the mirror attributes, plus the appropriate changes to the selection algorithm.
The issue of clients in North America being redirected to other continents really only applies to the OBS repositories for which we simply don't have enough mirrors in the Americas. (one was recently been added in Uruguay though).
We may have geo database outdated,
The Maxmind database is refreshed every Monday morning, I did the latest fix to the new maxmind interface myself, in 2019. It is worth checking up on though. -- Per Jessen, Zürich (5.8°C) Member, openSUSE Heroes
In my understanding MirrorBrain is lacking proper job queue and adding it + WebUI would result in managing two different projects inside single github repository. This is why we have two projects. I would be glad to do full help if somebody had a good vision how to fix it in mirrorbrain.
Thinking out loud - the two key shortcomings, https and ipv6 - are simply two more criteria for the mirror selection, nothing else. That sounds to me like some database changes for storing the mirror attributes, plus the appropriate changes to the selection algorithm.
If we implement it properly, we must regularly probe every mirror for each attribute: http, https, ipv4 and ipv6 (because certificates are added, expire, network changes, etc). And then we must track that properly and consider when matching client request. This is actually what the PR does. But MirrorBrain also has locking problems when two scanners scan the same folder on the same or differet mirrors, which is hard to track and troubleshoot. Plus lacks proper WebUI for managing servers and reports, etc. -- Per Jessen, Zürich (5.8°C) Member, openSUSE Heroes
Andrii Nikitin wrote:
In my understanding MirrorBrain is lacking proper job queue and adding it + WebUI would result in managing two different projects inside single github repository. This is why we have two projects. I would be glad to do full help if somebody had a good vision how to fix it in mirrorbrain.
Thinking out loud - the two key shortcomings, https and ipv6 - are simply two more criteria for the mirror selection, nothing else. That sounds to me like some database changes for storing the mirror attributes, plus the appropriate changes to the selection algorithm.
If we implement it properly, we must regularly probe every mirror for each attribute: http, https, ipv4 and ipv6 (because certificates are added, expire, network changes, etc).
Without having thought it through - isn't that pretty much what the scanner does today? it checks a mirror to see what is available, if an address is bad or a certificate doesn't work, the availability is updated. A poorly configured mirror should not interfere with the proper working of the rest of mirrorbrain. It would however be nice to be a little more fine-grained and record which parts of a mirror are working: ipv4, http ipv4, https ipv6, http ipv6, https In comparison to the current setup, this is just like having four mirrors instead of one. Maybe that is the way to look at the design.
And then we must track that properly and consider when matching client request. This is actually what the PR does.
The scanning, the status keeping and the selection algorithm?
But MirrorBrain also has locking problems when two scanners scan the same folder on the same or differet mirrors, which is hard to track and troubleshoot.
We scan regularly on olaf, and pontifex scans an individual mirror after each push of OBS repos. New mirrors are scanned before we enable them. I have not looked at the scanning setup in any detail, but is there any significant risk of two scanners colliding ? If these locking problems occur frequently, we have to look at that, agreed. -- Per Jessen, Zürich (4.8°C) Member, openSUSE Heroes
On 05/03/2021 17.18, Per Jessen wrote:
It would however be nice to be a little more fine-grained and record which parts of a mirror are working:
ipv4, http ipv4, https ipv6, http ipv6, https
In comparison to the current setup, this is just like having four mirrors instead of one. Maybe that is the way to look at the design.
We dont need to ask the mirror 4 times for every file. Just figure out once for the whole mirror, which of the 4 work.
Per Jessen wrote:
Without having thought it through - isn't that pretty much what the scanner does today? it checks a mirror to see what is available, if an address is bad or a certificate doesn't work, the availability is updated. A poorly configured mirror should not interfere with the proper working of the rest of mirrorbrain.
It would however be nice to be a little more fine-grained and record which parts of a mirror are working:
ipv4, http ipv4, https ipv6, http ipv6, https
In comparison to the current setup, this is just like having four mirrors instead of one. Maybe that is the way to look at the design.
Correction - no, of course not, we already keep track of different access ways for each mirror - rsync, ftp, http. Maybe just add some more variations? -- Per Jessen, Zürich (4.2°C) Member, openSUSE Heroes
Citeren Andrii Nikitin <andrii.nikitin@suse.com>:
We may have geo database outdated, MirrorCache has a route to check that: http://mirrorcache.opensuse.org/rest/myip
That puts my (static) IP in Moldova instead of The Netherlands, so I assume the GeoIP database used is somewhat outdated. Which brings me to the request that it would be really nice to be able to override the GeoIP location by setting an environment variable in case the location is really far off.
From: Arjen de Korte <suse+build@de-korte.org>
We may have geo database outdated, MirrorCache has a route to check that: http://mirrorcache.opensuse.org/rest/myip
That puts my (static) IP in Moldova instead of The Netherlands, so I assume the GeoIP database used is somewhat outdated. Which brings me
I've replied privately to address this.
to the request that it would be really nice to be able to override the GeoIP location by setting an environment variable in case the location is really far off.
That is probably question for zypper team, I will try to follow up. E.g. for direct MirrorCache (or MirrorBrain) requests you can overwrite IP with X-Forwarded-For header, e.g.: curl -H 'X-Forwarded-For: 142.250.185.206' http://mirrorcache.opensuse.org/rest/myip {"country":"US","ip":"142.250.185.206","region":"NA"} Regards, Andrii Nikitin
Arjen de Korte wrote:
Citeren Andrii Nikitin <andrii.nikitin@suse.com>:
We may have geo database outdated, MirrorCache has a route to check that: http://mirrorcache.opensuse.org/rest/myip
That puts my (static) IP in Moldova instead of The Netherlands, so I assume the GeoIP database used is somewhat outdated. Which brings me to the request that it would be really nice to be able to override the GeoIP location by setting an environment variable in case the location is really far off.
In fact you can, but in the Netherlands you should hardly have an issue? zypper probably won't support this though, but never mind: wget --header='X-Forwarded-For: 1.2.3.4,your.own.ip.addr" \ http://download.o.o/something/something/item.mirrorlist. 1.2.3.4 is the IP you wish to use instead of your own. -- Per Jessen, Zürich (4.6°C) Member, openSUSE Heroes
Citeren Per Jessen <per@opensuse.org>:
Arjen de Korte wrote:
Citeren Andrii Nikitin <andrii.nikitin@suse.com>:
We may have geo database outdated, MirrorCache has a route to check that: http://mirrorcache.opensuse.org/rest/myip
That puts my (static) IP in Moldova instead of The Netherlands, so I assume the GeoIP database used is somewhat outdated. Which brings me to the request that it would be really nice to be able to override the GeoIP location by setting an environment variable in case the location is really far off.
In fact you can, but in the Netherlands you should hardly have an issue?
I wish that was true. A few years ago, 'zypper' would consistently locate my IPv6 address in the US. I only noticed this after a while because of the horrendous latency and terrible speed. I had to disable IPv6 before runnning updates.
zypper probably won't support this though, but never mind:
wget --header='X-Forwarded-For: 1.2.3.4,your.own.ip.addr" \ http://download.o.o/something/something/item.mirrorlist.
1.2.3.4 is the IP you wish to use instead of your own.
As far as I know, 'zypper' defaults to using '(lib)curl', so I doubt this will work. But it has been fixed in the meantime by loading a newer GeoIP database. Which is the problem with many GeoIP services: stale databases.
Arjen de Korte wrote:
Citeren Per Jessen <per@opensuse.org>:
Arjen de Korte wrote:
Citeren Andrii Nikitin <andrii.nikitin@suse.com>:
We may have geo database outdated, MirrorCache has a route to check that: http://mirrorcache.opensuse.org/rest/myip
That puts my (static) IP in Moldova instead of The Netherlands, so I assume the GeoIP database used is somewhat outdated. Which brings me to the request that it would be really nice to be able to override the GeoIP location by setting an environment variable in case the location is really far off.
In fact you can, but in the Netherlands you should hardly have an issue?
I wish that was true. A few years ago, 'zypper' would consistently locate my IPv6 address in the US. I only noticed this after a while because of the horrendous latency and terrible speed. I had to disable IPv6 before runnning updates.
Yeah, that's not nice. We (openSUSE) are entirely dependent on 3rd parties supplying quality geoip data. "a few years ago", I can well accept the quality of IPv6 geo-location data may have been less than desired.
zypper probably won't support this though, but never mind:
wget --header='X-Forwarded-For: 1.2.3.4,your.own.ip.addr" \ http://download.o.o/something/something/item.mirrorlist.
1.2.3.4 is the IP you wish to use instead of your own.
As far as I know, 'zypper' defaults to using '(lib)curl', so I doubt this will work.
There is in fact a CURLOPT_HTTPHEADER, but I haven't tried it.
But it has been fixed in the meantime by loading a newer GeoIP database. Which is the problem with many GeoIP services: stale databases.
Well, ours is supposed to be refreshed every Monday, but see issue#89566 -- Per Jessen, Zürich (4.5°C) Member, openSUSE Heroes
On 05/03/2021 14.05, Per Jessen wrote:
The issue of clients in North America being redirected to other continents really only applies to the OBS repositories for which we simply don't have enough mirrors in the Americas. (one was recently been added in Uruguay though).
The issue is also with latency. US clients always go to download.o.o that is in NUE and then get redirected to the actual mirror. If we wanted to have a mirrorbrain server in the US, we would need another 20TB disk space to mirror all of published OBS - and would need the bandwidth to keep it updated. Also, mirrorbrain is written in C as an apache module and the original author is not coding anymore. So not easy to maintain.
Bernhard M. Wiedemann wrote:
On 05/03/2021 14.05, Per Jessen wrote:
The issue of clients in North America being redirected to other continents really only applies to the OBS repositories for which we simply don't have enough mirrors in the Americas. (one was recently been added in Uruguay though).
The issue is also with latency. US clients always go to download.o.o that is in NUE and then get redirected to the actual mirror.
Yes, that is something we could look at improving on.
If we wanted to have a mirrorbrain server in the US, we would need another 20TB disk space to mirror all of published OBS - and would need the bandwidth to keep it updated.
It seems to me it ought to be possible to _just_ run mirrorbrain without also being a mirror? Just to improve on the latency. -- Per Jessen, Zürich (6.1°C) Member, openSUSE Heroes
Citeren Andrii Nikitin <andrii.nikitin@suse.com>:
(Note: this email was sent weeks ago, but it looks it didn't come trough).
Hi all,
As part of improvements to download.opensuse.org experience, currently new pilot service is available for using with zypper.
https://mirrorcache.opensuse.org
Its primary task is to redirect zypper requests to a mirror in client's country, similar to what MirrorBrain project does.
I've used 'mirrorcache-eu.opensuse.org' for a while, but since a couple of weeks it has become unusable. The desktop packagekit applet isn't able to update my Tumbleweed system anymore and when running 'zypper dup', it invariably requires several retries for failing downloads and it also frequently segfaults. Several systems. Neither of this was/is a problem when using 'download.opensuse.org' so I guess there is something wrong in the metadata that is returned by mirrorcache. Could it be that mirrorcache is overreporting (incomplete downloads and retry?) or underreporting (resulting in segfaults?) download size?
participants (5)
-
Andrii Nikitin -
Arjen de Korte -
Bernhard M. Wiedemann -
Neal Gompa -
Per Jessen