Hi mirror list, As we already know, openSUSE implements cdn now. Could someone explain to me what the impact of the implementation is to our mirror? AFAIK cdn will maintain the newest copy from one legitimate source, I imagine this would be the mirror in openSUSE infrastructure. If it is the case anyone will use the closest cdn rather than the mirror. My understanding might be wrong. Looking at the statistics in the mirror that I manage the traffic is still high, most likely the TX amount is more than 100GB daily, sometimes more than 200GB. Thousand daily hits and hundred visitors. Anyone see the significant traffic difference before and after the cdn implementation? Thanks! -- Edwin (repo.opensuse.id; twrepo.opensuse.id)
On 06/09/2023 07.07, medwinz wrote:
Hi mirror list,
As we already know, openSUSE implements cdn now. Could someone explain to me what the impact of the implementation is to our mirror? AFAIK cdn will maintain the newest copy from one legitimate source, I imagine this would be the mirror in openSUSE infrastructure. If it is the case anyone will use the closest cdn rather than the mirror. My understanding might be wrong.
Looking at the statistics in the mirror that I manage the traffic is still high, most likely the TX amount is more than 100GB daily, sometimes more than 200GB. Thousand daily hits and hundred visitors.
Anyone see the significant traffic difference before and after the cdn implementation?
It is expected that traffic remains: 1. The CDN is only partially used atm - either via opt-in or automatic for some regions. We need to improve the config some more before it can become the default for everyone. 2. The CDN contract comes with limited bandwidth, so the current plan is to serve only small files (<100kB) directly via CDN (because there latency matters most) and for larger files cache the redirect or metalink pointing to mirrors. Plus we serve files through CDN that are not yet found on any mirrors. In one random sample, we found that download.o.o redirected 1.5 TB/h worth of traffic to our 100+ mirrors - it would be expensive (around 6kUSD/mon) to replace all of that. And abandoning our mirrors would mean, we have no good fallback in case of CDN outages. 3. The CDN is a caching proxy. It is good because we only need to transfer files that are really requested. It is bad because it adds latency when caches are cold or expired. Using a mirror directly is still the fastest for many users. Ciao Bernhard M.
On Wed, Sep 6, 2023 at 5:40 PM Bernhard M. Wiedemann <bwiedemann@suse.de> wrote:
On 06/09/2023 07.07, medwinz wrote:
Hi mirror list,
As we already know, openSUSE implements cdn now. Could someone explain to me what the impact of the implementation is to our mirror? AFAIK cdn will maintain the newest copy from one legitimate source, I imagine this would be the mirror in openSUSE infrastructure. If it is the case anyone will use the closest cdn rather than the mirror. My understanding might be wrong.
Looking at the statistics in the mirror that I manage the traffic is still high, most likely the TX amount is more than 100GB daily, sometimes more than 200GB. Thousand daily hits and hundred visitors.
Anyone see the significant traffic difference before and after the cdn implementation?
It is expected that traffic remains:
1. The CDN is only partially used atm - either via opt-in or automatic for some regions. We need to improve the config some more before it can become the default for everyone.
2. The CDN contract comes with limited bandwidth, so the current plan is to serve only small files (<100kB) directly via CDN (because there latency matters most) and for larger files cache the redirect or metalink pointing to mirrors.
Plus we serve files through CDN that are not yet found on any mirrors.
In one random sample, we found that download.o.o redirected 1.5 TB/h worth of traffic to our 100+ mirrors - it would be expensive (around 6kUSD/mon) to replace all of that. And abandoning our mirrors would mean, we have no good fallback in case of CDN outages.
3. The CDN is a caching proxy. It is good because we only need to transfer files that are really requested. It is bad because it adds latency when caches are cold or expired. Using a mirror directly is still the fastest for many users.
Ciao
Bernhard M.
Hi Bernhard, Thanks for your thorough explanation. That explains why the traffic seems to remain the same. Best, -- Edwin
participants (2)
-
Bernhard M. Wiedemann -
medwinz