Feature changed by: Ján Kupec (jkupec) Feature #305624, revision 95 Title: Allow downloading all packages before installing openSUSE-11.2: Candidate Priority Requester: Important Projectmanager: Important Requested by: Jiri Srain (jsrain) Description: Libzypp should allow to set a policy to first download all packages and afterwards install all packages at once. It should be considered whether to enable this policy under some scenarios (e.g. makes sense for online update, but not for distro installation) References: https://bugzilla.novell.com:443/show_bug.cgi?id=448040 Relations: - Suggestion for download failure handling from openSUSE-factory (url: http://lists.opensuse.org/opensuse-factory/2009-01/msg00564.html) Discussion: #1: Stanislav Visnovsky (visnov) (2009-01-28 15:56:51) I see this as a pre-requisite for in running-system distribution upgrade reliability. #3: Robert Davies (robopensuse) (2009-01-29 16:56:54) (reply to #1) Could this be better achieved by some general package cache-ing tool that can serve a LAN, providing packages from HDD, DVD or http/ftp download as appropriate? Upgrading by making a local copy of all packages in somewhere like /var/cache is going to explode disk space requirements. If the disk is nearly full, then both throughput and fragmentation worsen. In degenerate case of single PC LAN, the server can be run locally, which might ease the opposite policy of pipelining downloads to accelerate install speeds when most rpm's are available locally, but some need an update downloaded. #4: Rajko Matovic (rajko_m) (2009-01-30 01:41:51) (reply to #3) It could be, but here is the word about download of packages involved in update before attempt to install them. It is just about improvement of current workflow where some packages are already installed when come to download error. This can have unwanted consequences to system, or applications. In some scenarios it can render system unbootable, in other package management will be rendered inoperational. In any case to improve dependability of package management first download, then install is needed feature. #6: Robert Davies (robopensuse) (2009-01-31 14:07:54) (reply to #4) I understand, but breaking the problem up, would make the logic simpler, and be more flexible for other requirements, permit interleaving of bandwidth & cpu limitted operations in parallel. Even downloading first, doesnt' guarantee a needed package, will be installable, unless you check all the signatures etc. Really the reliable update problem, requires installing a package set as a transaction, download first just avoids 1 class of errors, but if it's implemented naively could cause problems for other ppl. #7: Manuel Trujillo (toomany) (2009-02-02 15:47:03) (reply to #1) I agree completly with you. #9: Diederik van der Boor (vdboor) (2009-02-26 09:12:38) (reply to #1) Agreed. I've broken my system once before with a dist-upgrade. Half way the mirrors couldn't provide some packages (probably just got a buildrev-upgrade) and I got stuck with an half-upgraded system. Had to boot from CD, get the system bootable again and do rpm magic to find all packages which weren't upgraded. This just shouldn't happen, and downloading all packages first should fix it too. Nowadays I always do a dist-upgrade from DVD as workarround. #2: Stanislav Visnovsky (visnov) (2009-01-29 13:02:26) A suggestion on openSUSE-factory mailing list was to continue downloading all packages, deferring the errors until all that are donwloadable are indeed on the disk. #5: Jan Engelhardt (jengelh) (2009-01-30 15:26:32) Both modes should be supported. Install-as-downloaded (like it does today) for space-contrained systems, and install-after-download for where there is enough space and/or the user enabled it. #8: Jay Mistry (zmdmw52) (2009-02-15 09:15:05) (reply to #5) I agree; (in addition to space-constraints), may also be relevant in places that have limited/time-based Internet connectivity. For those who (and many users don't require this) do not want/need latest desktop effects, etc; it would make sense to have a cache (locally) of executables (RPM's), that can be saved to backup medium (network server, CD/DVD, etc) once a stable working system configuration is obtained. This can be used as a 'local repo' for system file updates as well as software (program) RPM's (similar to how the install DVD is listed as a repo source). For those that prefer to do or don't mind periodic Internet updates, they can disable the former option. #10: Pascal Bleser (pbleser) (2009-03-02 08:39:41) You may note that this is how smart does it: it first downloads all the required packages to disk (into /var/cache/smart/packages/) and then applies the updates in a single RPM transaction, without resorting to -- nodeps. Also, when the transaction fails, it leaves the downloaded packages into the cache directory. And obviously picks them up from there when available, and doesn't download them again. There is an explicit command ("smart clean") to clean up the cache directory. #13: Reza Davoudi (rd1381) (2009-06-23 21:49:00) (reply to #10) smart is great in this feature but lacks other robust features : like when i tried to install the new version of amarok with smart , it downloaded the packages but when instaling said that there is a conflict of file between an installed package and one of the installing packages . i tried it with yast and it went smoothly. maybe yast ignores this minor file conflicts? :) but i agree that a download and install later feature (or at list an option to do so before installing ) would be great , especialy now that there is many mirror problems and that yast sticks to one mirror for every repository and retring ,doesnt change the mirror to a better one . #14: Peter Poeml (poeml) (2009-06-23 22:06:37) (reply to #13) Oh, yast/zypper do use fallback mirrors meanwhile. Implemented in Factory/11.2, they use metalinks. Check this link (http://en.opensuse.org/Libzypp/Failover) . Any testing would be welcome. #15: Reza Davoudi (rd1381) (2009-06-24 02:34:48) (reply to #14) i tried updating zypp (to test it in 11.1) and it required updating yast2-* packages so i did but now yast software manager and repositoties don't work what should i do? #16: Reza Davoudi (rd1381) (2009-06-24 02:37:06) (reply to #15) actually none of the yast modle work i click on them but nothing happens #17: Peter Poeml (poeml) (2009-06-24 03:45:22) (reply to #16) You have two options. Report a bug (http://bugs.opensuse.org/) or post to the zypp-devel mailing list (http://en.opensuse.org/Zypper/Development#Contacting_Developers) . Just a note - please keep this feature tracking tool issue clean, in particular because it is about an entirely different subject. Thanks! #11: Cristian Morales Vega (reddwarf) (2009-03-05 10:20:10) A problem with the current Install-as-downloaded system is that rpm needs to be executed with "--replacefiles" (or something equivalent). See http://bugzilla.novell.com/411574 An install-after-download system should behave like Smart, checking first if there two packages trying to own the same (but different) file. #18: Glenn Doig (doiggl) (2009-07-07 07:32:50) I agree with comment #11 - Cristian. It also needs to handle interrupted downloads - resume them as necessary. #19: Segundo Luis Martín Díaz Sotomayor (zchronos) (2009-07-07 17:57: 47) And something like AptonCD of Ubuntu ? #20: andrea florio (anubisg1) (2009-07-10 00:05:37) (reply to #19) i began to work on it: that's my zypperonDVD http://pastebin.com/f770b3398 #21: Michael Andres (mlandres) (2009-07-21 12:29:49) libzypp-6.11.0 allows to set different download policies in ZYppCommitPolicy (DownloadOnly, DownloadInAdvance, DownloadInHeaps, DownloadAsNeeded). #23: Martin Vidner (mvidner) (2009-08-27 16:08:51) zypper-1.2.3 (in home:mvidner, SRI 19048 to zypp:Head) implements -- download-only. #25: Michael Andres (mlandres) (2009-09-10 15:03:17) (reply to #23) Jano: Will there also be options in zypper to choose the other download policies (inAdvance, inHeaps, asNeeded)? #26: (ulenrich) (2009-10-11 13:21:03) (reply to #25) /etc/zypp/zypp.conf: ## Commit download policy to use as default. ## ## DownloadOnly, Just download all packages to the local cache. ## Do not install. Implies a dry-run. ## ## DownloadInAdvance, First download all packages to the local cache. ## Then start to install. ## ## DownloadInHeaps, Similar to DownloadInAdvance, but try to split ## the transaction into heaps, where at the end of ## each heap a consistent system state is reached. ## ## DownloadAsNeeded Alternating download and install. Packages are ## cached just to avid CD/DVD hopping. This is the ## traditional behaviour. ## ## <UNSET> If a value is not set, empty or unknown, we pick ## some save default. ## commit.downloadMode = DownloadInAdvance #27: (ulenrich) (2009-10-11 13:23:46) (reply to #26) ...bad text formatting... ## Commit download policy to use as default. ## DownloadOnly, Just download all packages to the local cache. ## DownloadInAdvance, First download all packages to the local cache. ## DownloadInHeaps, Similar to DownloadInAdvance, but try to split ## DownloadAsNeeded Alternating download and install. Packages are #28: Ján Kupec (jkupec) (2009-10-13 13:08:39) (reply to #25) I wish i could say they will, but i'm pretty busy with other things. And we're already pretty close to the 11.2 GM. I will still try to add them, though. + #30: Ján Kupec (jkupec) (2009-10-13 14:34:05) (reply to #23) + this was done as part of FATE #302159 #24: Ralph Ulrich (ulenrich) (2009-09-05 16:07:03) I see this is the workaround: zypper ref && zypper dup -d && zypper -v dup #29: Ján Kupec (jkupec) (2009-10-13 13:09:15) (reply to #24) yes, exactly. -- openSUSE Feature: https://features.opensuse.org/305624