[opensuse-packaging] Download BIG data in %post script. What do you think?
I got this https://build.opensuse.org/request/show/135388 What do you think? It actually solves a real problem: the model of having a copy of everything in our servers doesn't scale. Specially with big game data. Problems I can think of: - You don't have control over the external servers. Somebody could modify the files there add hack into users machines. - No information about the files is stored in the RPM database. It's impossible to verify the integrity of the installed package. - It just looks bad, ugly and fragile. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Thursday 27 of September 2012 14:05EN, Cristian Morales Vega wrote:
Problems I can think of: ...
- makes installation on machines with slow/no Internet connectivity complicated/impossible. Michal Kubeček -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Thu, Sep 27, 2012 at 03:09:58PM +0200, Michal Kubeček wrote:
On Thursday 27 of September 2012 14:05EN, Cristian Morales Vega wrote:
Problems I can think of: ...
- makes installation on machines with slow/no Internet connectivity complicated/impossible.
It also should probably be handled outside of the %post magic as it will take very long. fetchmsttfonts has a post-install script to do it. But then again, if there is possibility to redistribute we have the choice to include it. (check licenses!) Or tell the user to call a script. Ciao, Marcus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On 27 September 2012 14:14, Marcus Meissner
On Thu, Sep 27, 2012 at 03:09:58PM +0200, Michal Kubeček wrote:
On Thursday 27 of September 2012 14:05EN, Cristian Morales Vega wrote:
Problems I can think of: ...
- makes installation on machines with slow/no Internet connectivity complicated/impossible.
It also should probably be handled outside of the %post magic as it will take very long.
fetchmsttfonts has a post-install script to do it.
I would actually like to have a nice little library to add entries in the RPM database. Making it easy to create such a script/installer that actually integrates with the packaging system.
But then again, if there is possibility to redistribute we have the choice to include it. (check licenses!)
What I don't want is losing mirrors because we have too much data. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Thursday, September 27, 2012 14:05:34 Cristian Morales Vega wrote:
I got this https://build.opensuse.org/request/show/135388
What do you think? It actually solves a real problem: the model of having a copy of everything in our servers doesn't scale. Specially with big game data.
Problems I can think of: - You don't have control over the external servers. Somebody could modify the files there add hack into users machines. - No information about the files is stored in the RPM database. It's impossible to verify the integrity of the installed package. - It just looks bad, ugly and fragile.
A side note: During the build process, the packages get installed and the %post section is executed. In the build process you have no network - so be carefull what you execute in the post script, Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Dne 27.9.2012 15:05, Cristian Morales Vega napsal(a):
I got this https://build.opensuse.org/request/show/135388
What do you think? It actually solves a real problem: the model of having a copy of everything in our servers doesn't scale. Specially with big game data.
Downloading _anything_ in %post is a bad idea, even for small files. These commands should be included in a script which can be started by users after installation, at the time they want/can download big files. The problem is that the servers (or network in general) might not be available in time of package installation, but they can be available later. (Also this would not work in SUSE Studio as there is no network during build.) And there are other problems with the "script": - you cannot simply retry download again if it fails you would need to reinstall package again (not user friendly) - the script does not check whether the files are already installed, it downloads a new copy at every installation/update - it does not continue failed downloads ("-c" wget option would be nice here) - you cannot simply skip the download and obtain the files via other way or share the downloaded files (i.e. you have 2 computers at home, you need to download the files only once, then you can just copy them to the other machine) - no integrity check, no authenticity check
Problems I can think of: - You don't have control over the external servers. Somebody could modify the files there add hack into users machines. - No information about the files is stored in the RPM database. It's impossible to verify the integrity of the installed package. - It just looks bad, ugly and fragile.
Yes, very fragile. -- Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
OK. So please now critique this one:
https://build.opensuse.org/package/view_file?project=games&package=smokin-guns-data&file=smokin-guns-data.spec
New approach...
On 27 September 2012 14:57, Ladislav Slezak
And there are other problems with the "script":
- you cannot simply retry download again if it fails you would need to reinstall package again (not user friendly)
Allows it.
- the script does not check whether the files are already installed, it downloads a new copy at every installation/update
Does with the data-version.txt file. I'm not 100% happy with it, but...
- it does not continue failed downloads ("-c" wget option would be nice here)
Not sure if Autodownloader does that. In general I would like to have something better than Autodownloader.
- you cannot simply skip the download and obtain the files via other way or share the downloaded files (i.e. you have 2 computers at home, you need to download the files only once, then you can just copy them to the other machine)
You could but you would need to know that the main package expects the data-version.txt with the specific content.
- no integrity check, no authenticity check
The zip file is verified with MD5. But once uncompressed you lose the capacity to verify. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
participants (5)
-
Andreas Jaeger -
Cristian Morales Vega -
Ladislav Slezak -
Marcus Meissner -
Michal Kubeček