Mailinglist Archive: opensuse-buildservice (156 mails)

< Previous Next >
[opensuse-buildservice] central download redirection on software.openSUSE.org
  • From: Christoph Thiel <cthiel@xxxxxxx>
  • Date: Wed, 30 Aug 2006 10:53:30 +0200 (CEST)
  • Message-id: <Pine.LNX.4.64.0608301044130.8880@xxxxxxxxxxxxx>
Hi everyone,

with the exponential growth of Build Service repositories, we need to find
a better way to keep a local "state cache" on software.openSUSE.org.
Currently, we poll each mirror (basically do a "find
/pub/opensuse/repositories") to keep the cache coherent. While this took
only some minutes for the few repos we started with, this is taking > 1
hour for fast mirrors already and will get even worse in the future :(

Therefore we are looking for suggestions for improvement!

One option would be to download the repomd.xml for each repository and
figure if it's current to mark all files in that repository "up-to-date".
But still, this would require > 700 GETs per server with the current
number of repositories.

Another option is to just have "pushed mirrors" in our
software.openSUSE.org redirection, because it's quite easy to parse the
rsync output and updated the cache based on that output.

The code that keeps the cache is available from
https://forgesvn1.novell.com/svn/opensuse/trunk/tools/download-redirector


Regards
Christoph
---------------------------------------------------------------------
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-buildservice+help@xxxxxxxxxxxx

< Previous Next >