[Bug 1192157] New: libzypp caching strategy
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
http://bugzilla.opensuse.org/show_bug.cgi?id=1192157 Bug ID: 1192157 Summary: libzypp caching strategy Classification: openSUSE Product: openSUSE Distribution Version: Leap 15.2 Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: libzypp Assignee: zypp-maintainers@suse.de Reporter: jdelvare@suse.com QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- I would like to understand libzypp's strategy for caching packages (under /var/cache/zypp/packages). My practical issue is with wine-staging packages which I get from the Emulators:Wine repository. Once in a while, a wine-staging update breaks my game and I would like to get back to the previous, known-working version. My expectation is that this package should be in the cache, as it was installed not long ago. However, looking in /var/cache/zypp/packages/Emulators_Wine/x86_64, I see old versions of wine-staging (from 6.3 to 6.12) but *not* the recent versions (latest is 6.20, so I would be looking for 6.19). What is the logic of libzypp to decide whether or not a package file should be kept in the cache by default? I would expect the cache to be handled in a first in, first out style, that is, delete older files first to make room for newer files. Newer packages have a much higher chance to be needed again. For now I am forced to manually download rpm packages and store them in my own cache. But as everything manual, I eventually forget and only remember the next time a wine-staging update breaks my game - at which point it is too late as the package I need is no longer available in the online repository. Now I see that repository files have a keeppackages option, which could help here. However, I haven't set it on any repository so far, still my /var/cache/zypp contains 1.7 GB of caches packages, so there's definitely caching going on even without this option. Makes me wonder what the option is really doing (I'm going to give it a try now). Shouldn't /var/cache/zypp/packages be empty when all repositories are configured with keeppackages=0? This brings another question of how much data libzypp is going to keep in the cache. The keeppackages flag is binary, if implemented the way it is documented, it would keep no files at all (if set to 0), or all files forever (if set to 1). I would expect a limit to exist so as to not fill out the storage space completely, however I couldn't find any such option in /etc/zypp/zypp.conf. Did I miss the option? Is there are hard-coded limit? -- You are receiving this mail because: You are on the CC list for the bug.
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
http://bugzilla.opensuse.org/show_bug.cgi?id=1192157 http://bugzilla.opensuse.org/show_bug.cgi?id=1192157#c1 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |CONFIRMED --- Comment #1 from Michael Andres <ma@suse.com> --- You're right, it's broken. With keeppackages=1 all packages downloaded from that repo stay in the cache. With keeppackages=0 (default) packages downloaded stay in the cache until they are installed. Purging downloaded, but not installed packages after some time does not work, so they accumulate over time. -- You are receiving this mail because: You are on the CC list for the bug.
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
http://bugzilla.opensuse.org/show_bug.cgi?id=1192157 http://bugzilla.opensuse.org/show_bug.cgi?id=1192157#c2 Neil Rickert <nwr10cst-oslnx@yahoo.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nwr10cst-oslnx@yahoo.com --- Comment #2 from Neil Rickert <nwr10cst-oslnx@yahoo.com> --- I'll just add a comment here. I use keeppackages=1 on several repos. I keep the cache on a network wide NFS server, so that packages don't need to be separately downloaded on each machine. I periodically do a manual cleanup with: find . -name '*rpm' -atime +14 -print0 | xargs -0 rm -f This is working out quite well here. -- You are receiving this mail because: You are on the CC list for the bug.
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
http://bugzilla.opensuse.org/show_bug.cgi?id=1192157 http://bugzilla.opensuse.org/show_bug.cgi?id=1192157#c3 --- Comment #3 from Michael Andres <ma@suse.com> --- Yes, but the keeppackages=0 case suffers. However if we fix it we can on the fly offer a zypp.conf parameter for the expiry of downloaded but not installed packages. A simple '[ca]time,days' rule like yours above should be sufficient. One for keeppackages=0 case (atime,30) and one for the keeppackages=1 (never). @Jean: `zypper clean` will clear your package cache. -- You are receiving this mail because: You are on the CC list for the bug.
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
http://bugzilla.opensuse.org/show_bug.cgi?id=1192157 http://bugzilla.opensuse.org/show_bug.cgi?id=1192157#c4 --- Comment #4 from Jean Delvare <jdelvare@suse.com> --- (In reply to Michael Andres from comment #3)
@Jean: `zypper clean` will clear your package cache.
Yes, I'll do that eventually, but first I wanted to check what had been cached so far. 1.7 GB is a lot, and I couldn't see how downloaded-but-not-installed packages could amount for that much. Looking at what is actually in my cache : * sparse in repository OSS, date July 31st. I remember hitting a bug when trying to update that package (see bug #1188947). The cached package is the old version, I have the update installed already and locked to workaround the versioning issue. There's no reason why the old version would have been downloaded as I did not intend to install it, unless updates are fetched in advance without taking locks into account (or such a pre-fetch happened before I locked the package). But I don't remember exactly what I did back then, so I can't conclude. * libx265-199 and x265 in repository packman, date May 20th. I didn't do anything special with these packages, and can't remember any failure to install them either. I can confirm (from /var/log/zypp/history) that these specific packages have never been installed on my system, I have more recent versions installed since May 21st. Again that's too old for me to remember what I did exactly back then, but the only scenario I can think of is some kind of conflict at installation time. I'd be surprised though, as libx265 gets a new major version integrated in its name for every incompatible change, so multiple versions of libx265 can be installed in parallel, which makes conflicts virtually impossible. So that one is strange. * A lot of dxvk, wine-staging and wine-staging-32bit packages in repositories Emulators and Emulators:Wine, from March 12th to July 4th. Total weight 1.6 GB. I remember there were many updates of the wine-staging package at that time, even when no change was mentioned in the changelog. I see that Marcus Meissner made a change on July 10th to make builds "reproducible". If there's some pre-fetching in place, that would explain why many of these packages were cached. There may also have been failures to install some of the packages back then, I seem to recall some conflicts being reported. * Many small packages in the standard update repository. 9 from March 4th (all related to bind-utils, I can't remember what happened with that back then), and three dozens from November 1st, that is yesterday. But I did not attempt to install any update yesterday, so I have no idea why these packages are already in the cache. I suppose these packages would get deleted after installation if I'd run "zypper up" now. All in all, the presence of most of these files would be explained if available updates get pre-fetched somehow, but a more recent version becomes available by the time I actually install the updates. I was not aware of such a pre-fetching mechanism. Is the GNOME PackageKit front-end doing that? Or some systemd service? -- You are receiving this mail because: You are on the CC list for the bug.
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
http://bugzilla.opensuse.org/show_bug.cgi?id=1192157 http://bugzilla.opensuse.org/show_bug.cgi?id=1192157#c5 --- Comment #5 from Jean Delvare <jdelvare@suse.com> --- (In reply to Jean Delvare from comment #4)
I suppose these packages would get deleted after installation if I'd run "zypper up" now.
Confirmed. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply@suse.com