Hi Alberto, On Tue, Sep 30, 2008 at 01:34:29PM -0500, Alberto Passalacqua wrote:
Hi Peter,
Could you please elaborate what exactly you mean with "our redirector fails ... quite dramatically at every release"?
I would be very interested in details.
I'm sorry for my delayed answer. I have noticed your question only now.
My point about the redirector is quite simple. Since when it was introduced, it had problems, especially under high load, which means immediately after a new openSUSE release.
The machine was never under high load during openSUSE release. It had a remarkably low load during peak times, less than 1.0 accounting for Apache and 1.something accounting for other services running on the machine (mostly rsync, but rsync is mostly busy before release, not after. Once we have released, everything is already synced.). Do you maybe interpret something else as an assumed load problem? It seems an assumption of yours.
How this is related to this discussion? Again, it is simple. When 11.0 was released, it was impossible to access to the download site and the automatically added repositories were not working because the redirector was overloaded and timing out the connection.
I don't know what you are talking about - I am not aware of any such instance. And I look quite closely after the machine, so I should know. It's also the first time I hear about it. Usual response during release is that people are quite surprised about the snappiness of download.opensuse.org. In fact, download.opensuse.org is the only machine that did *not* go down during release (the wiki did, for instance). Is it possible that you were sent to a *mirror* which was overloaded? Against overloaded mirrors, there are two things that can be done: 1) when downloading an ISO image, use a metalink client. download.opensuse.org will give it the list of best mirrors and the metalink client will automatically cope with failing mirrors. 2) for zypp/YaST, there is early support for metalinks as well, which would help for the download of repository metadata and packages (which this discussion refers to). I *do* see, understand and appreciate your point though, because the situation you describe does transfer to the situation of failing mirrors; for the end user it doesn't make a difference if the redirector fails or the mirror that it is sent to. So your point of adding online stuff at install time is valid of course. However I see possibilities to alleviate this (see above). For 11.1, we won't have this, and in any case we need to be careful to implement online accesses in a way that they don't block anything, can be delayed and fail gracefully. Obviously, if there really was a load problem with the redirector, I need to know about it. But I doubt there is. If you wonder how download.opensuse.org survives the release: it is very simple - the machine gets about 100-200 requests per seconds right now (quiet time), and during release this increases to about 200-300 requests per second; it hardly doubles. Thus, there *is* no real load spike during release. The reason for this is that the download requests for the new release, and new installations, get lost in the noise of ongoing update requests of existing installations. Even if we get 3 requests per seconds for an ISO image during release, you can't notice that in the 200 req/s that are constantly hitting the server. The circadian (day/night) shift is in fact larger than what a release day can add to it. Hope this makes it clear :-)
As a consequence, moving the non-OSS software mentioned in this thread to a non-OSS repository and asking the user if he wants to add it at installation time will lead to the following problem: a lot of people will download openSUSE 11.1, will install it and due to the overload it won't be able to use the repository at the time of installation. The consequence of this will be a lot of complaints, and potentially of some "bad press", which is what seems to worry Novell more.
I hope I clarified.
With kind regards, Alberto
Thank you very much for the details! Peter -- Contact: admin@opensuse.org (a.k.a. ftpadmin@suse.com) #opensuse-mirrors on freenode.net Info: http://en.opensuse.org/Mirror_Infrastructure SUSE LINUX Products GmbH Research & Development