From the view of mirror feeding, I am strongly opposed to adding more
https://bugzilla.novell.com/show_bug.cgi?id=233122 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=233122#c56 --- Comment #56 from Peter Poeml <poeml@novell.com> 2008-02-08 03:41:16 MST --- directories below /pub/opensuse/update/. Especially large ones. The rsync modules on {stage,rsync}.opensuse.org (which the mirrors depend on) are an increasing mess. What typically happens (resp. happened in the past) is that we added new content right into the trees which mirrors pull their copies from. Some examples: - the existing debuginfo packages. The 10.3 debuginfo for instance comprises 24 GB of 51 GB repo size, but they are hardly ever downloaded. It doesn't make sense to occupy 24 GB of space on mirrors with files which are hardly ever downloaded. To give you some perspective: yesterday's log (a typical day) shows 1.896.276 downloads for rpm files 4.028 downloads for rpm debuginfo files which is a ridiculously small number. Likely no mirror needed at all. It's sufficient if those files are on download.opensuse.org. - the drpmsync tree (30 GB wasted on mirrors). Mirrors were mirroring this tree even though they didn't know about it, because it was "hidden" in the Factory directory. There were more mirrors than active downloaders of the tree (5)! - the 10.3-test update tree (12 GB wasted on mirrors). This tree is for testing and surely doesn't need to be mirrored. (But it was mirrored until recently, because it was added without considering mirroring.) I'm sure we could have a lot more mirrors, if we wouldn't waste so much of their disk space anymore. Mirrors that don't even consider mirroring us because of the required disk space, mirrors that have too bad international connectivity to mirror hundreds of gigs, ... In addition, we have a load problem on our rsync servers, which is largely caused by those proliferated trees. Configured rsync excludes are not a real solution to this, because they don't make the tree smaller which the origin server (download.opensuse.org) has to stat. For instance, it takes over half an hour meanwhile just to find out that /repositories has no new files (full sync over the tree, even omitting all repoview files). drpmsync is another example, as the extreme number of files (200.000) takes insane amounts of time to rsync (try du -sc on the directory). Therefore, I strongly suggest to put extra stuff like debuginfos in separate trees from now on. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.