On Thu, Jun 25, 2009 at 07:12:17 +0200, Cristian Morales Vega wrote:
The files are already hosted in a lot of places, so it's a shame not to use them. But I would like a way that after installation there is no difference from a real RPM with the files packaged. I could put %ghost file entries and download the real files in a scriplet (or autodownloader, as Packman does)... but that would be still not the same. I know everything about these files, hash included, so I would like a way that puts that info into the RPM DB. Something that would list all the correct info when I do a "rpm -qvl" or a "rpm -V".
Then I'd rather recommend to package the files as RPM packages. In addition, a repository would make sense for clients to install and update from. If there are packages somewhere, it is trivial to create a repository with the createrepo tool, and the repository could be hosted in a different location than the RPM packages (but link to them). But it sounds as if the files are not distributed as packages so far, but rather in a different form, right? You could go as far as packaging a .repo file in an openSUSE build service package, which is installed into /etc/zypp/repos.d so to automatically subscribe a users to the "foreign" repository hosting the packaged large files.
I though about this when I saw FreeSpace 2. The enhanced game contect is already 1,4 GiB uncompressed (perhaps 1 GiB compressed). Add 900 MiB for the FreeSpace 1 port... and even 250 MiB more for cutscenes. That's without thinking about other MODs.
With files that large, downloading presents a challenge in itself, because calling wget in %post won't work well. I'd recomment to solve this in the same way as openSUSE does for CD and DVD images, which is to provide metalinks and use a metalink client for downloading, which automatically handles mirror issues and also adds downloading via BitTorrent into the picture. You would need to set up MirrorBrain on a server that hosts the files, collect URLs of existing mirrors, and let the MirrorBrain server do the rest. Starting with openSUSE 11.2, the metalink client aria2c will be on every system anyway, and automatically used by YaST/zypper. On earlier openSUSE's you could still draw aria2c in via package dependencies, and do things in %post with it. With files that large, it's worth going this route.
For this specific case I'm not really thinking about putting all this in the OBS, but made me think about the problem. But the big data content is an increasing problem with games. A more real example would be Warzone 2100, since 2.2 version it added support for videos... 162 MiB. That's a number that makes one wonder if it's ok, needs any special care or what. Do you have any stats for the downloads of the previous versions of the game? If isn't very popular perhaps a README.openSUSE saying that the user can download the videos file and put it in his home dir would be enough.
I would need to grep the logs. Unfortunately, we don't log build service downloads anymore, because then the question would be easy to answer (we even used to have the stats available right in the build service website). The download stats could and should be revived; there ideas how to do it but nobody had the time so far to really work on it. Yesterday, these were all downloads I see grepping for "warzone" (shortened from the access_log): GET /repositories/games/openSUSE_11.0/x86_64/warzone2100-2.1.3-1.2.x86_64.rpm HTTP/1.1" 302 "ZYpp 4.28.1 (curl 7.18.1)" unixheads.net NA:CA GET /repositories/games/openSUSE_11.0/i586/warzone2100-2.1.3-1.2.i586.rpm HTTP/1.1" 302 "ZYpp 4.28.1 (curl 7.18.1)" mirror.leaseweb.com EU:AT GET /repositories/games/openSUSE_11.1/i586/warzone2100-2.1.3-1.2.i586.rpm HTTP/1.1" 302 "ZYpp 5.30.3 (curl 7.19.0) openSUSE-11.1-i586" ftp.twaren.net AS:UZ GET /repositories/games/openSUSE_11.1/x86_64/warzone2100-2.1.3-1.2.x86_64.rpm HTTP/1.1" 302 "ZYpp 5.30.3 (curl 7.19.5) openSUSE-11.1-x86_64" ftp.uni-heidelberg.de EU:FR GET /repositories/games/openSUSE_11.1/x86_64/warzone2100-2.1.3-1.2.x86_64.rpm HTTP/1.1" 302 "ZYpp 5.30.3 (curl 7.19.0) openSUSE-11.1-x86_64" anorien.csc.warwick.ac.uk EU:GB GET /repositories/games/openSUSE_11.1/i586/warzone2100-2.1.3-1.2.i586.rpm HTTP/1.1" 302 "ZYpp 5.30.3 (curl 7.19.0) openSUSE-11.1-i586" anorien.csc.warwick.ac.uk EU:IT GET /repositories/games/openSUSE_11.1/i586/warzone2100-2.1.3-1.2.i586.rpm HTTP/1.1" 302 "ZYpp 5.30.3 (curl 7.19.0) openSUSE-11.1-i586" ftp5.gwdg.de EU:FR The mirror database knows 16 mirrors that have the files. The logged downloads are all the same version. I don't know the game or its existing versions, so I can't derive much from the log. Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development