[opensuse-packaging] Orphaned files galore
Hi all, I've written a small utility that checks for files, symlinks and directories on your system that are installed in the system directories but are not owned by any package. https://github.com/AdamMajer/rpmorphan To compile this, all you need is rpm-devel and gcc. Running on my Leap 15.1 system, it found quite a number of symlinks and directories and files that are not owned by any package. For example, /usr/lib64/libreoffice/share/config/images_sifr.zip is installed by not owned by any libreoffice packages. There are also number of files that install symlinks with update-alternatives but do not actually own these symlinks. The correct method is to install these as %ghost entries in the %files section. More interestingly, rpm itself doesn't own its own files either. /usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ... Overall, I've had 8592 orphaned files out of 231008 in my rpm database. This means close to one in 25 files is not tracked! Feedback welcome - Adam -- Adam Majer - amajer@suse.de SUSE Software Solutions Germany GmbH HRB 36809 (AG Nürnberg), GF: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Hello, On Fri, 6 Dec 2019, Adam Majer wrote:
I've written a small utility that checks for files, symlinks and directories on your system that are installed in the system directories but are not owned by any package.
https://github.com/AdamMajer/rpmorphan
To compile this, all you need is rpm-devel and gcc.
Nice.
Running on my Leap 15.1 system, it found quite a number of symlinks and directories and files that are not owned by any package. For example,
/usr/lib64/libreoffice/share/config/images_sifr.zip
(Not trying to steal your thunder) An alternative for GUI lovers is qdirstat from Stefan Hundhammer. It can show the same graphically (File/Show unpackaged files...), including the usual features of it, like treemap and friends. (FWIW, I have only about 2000 unpackaged files below /usr :-) ) Ciao, Michael. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Fri, 6 Dec 2019 17:37:27 +0100 Adam Majer wrote:
I've written a small utility that checks for files, symlinks and directories on your system that are installed in the system directories but are not owned by any package.
Hello. It looks, the name "rpmorphan" is reserved already. https://build.opensuse.org/package/show/openSUSE%3AFactory/rpmorphan -- WBR Kyrill
* Kyrill Detinov
On Fri, 6 Dec 2019 17:37:27 +0100 Adam Majer wrote:
I've written a small utility that checks for files, symlinks and directories on your system that are installed in the system directories but are not owned by any package.
Hello.
It looks, the name "rpmorphan" is reserved already. https://build.opensuse.org/package/show/openSUSE%3AFactory/rpmorphan
has been a provided package for quite some years ... -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Am Freitag, 6. Dezember 2019, 17:37:27 CET schrieb Adam Majer:
Hi all,
I've written a small utility that checks for files, symlinks and directories on your system that are installed in the system directories but are not owned by any package.
https://github.com/AdamMajer/rpmorphan
To compile this, all you need is rpm-devel and gcc.
Running on my Leap 15.1 system, it found quite a number of symlinks and directories and files that are not owned by any package. For example,
/usr/lib64/libreoffice/share/config/images_sifr.zip
is installed by not owned by any libreoffice packages. There are also number of files that install symlinks with update-alternatives but do not actually own these symlinks. The correct method is to install these as %ghost entries in the %files section.
More interestingly, rpm itself doesn't own its own files either.
/usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ...
Overall, I've had 8592 orphaned files out of 231008 in my rpm database. This means close to one in 25 files is not tracked!
Feedback welcome
I've used the predecessor occasionally: https://build.opensuse.org/package/show/utilities/rpmorphan Hopefully, this will have some impact on the distribution as well. Cheers, Pete -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On 2019-12-06 10:23 p.m., Hans-Peter Jansen wrote:
I've used the predecessor occasionally:
Ha! I should have used a different name instead, rpm-orphan-files, maybe :) - Adam -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Friday 2019-12-06 17:37, Adam Majer wrote:
More interestingly, rpm itself doesn't own its own files either.
/usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ...
That is quite expected. I would not expect to lose the actual databases. `rpm -e rpm` should not remove (what was once) /var/lib/rpm, `rpm -e mariadb` should not remove /var/lib/mysql, and `rpm -e postfix` should not delete my mails. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Am Freitag, 6. Dezember 2019, 18:08:13 CET schrieb Michael Matz:
(Not trying to steal your thunder) An alternative for GUI lovers is qdirstat from Stefan Hundhammer. It can show the same graphically (File/Show unpackaged files...), including the usual features of it, like treemap and friends. (FWIW, I have only about 2000 unpackaged files below /usr :-) )
I installed that just now, and there's no "Show unpackaged files" menu option anywhere... Cheers MH -- Mathias Homann Mathias.Homann@openSUSE:.org irc: [Lemmy] @ freenode, ircnet obs: lemmy04 keybase: https://keybase.io/lemmy gpg key fingerprint: 8029 2240 F4DD 7776 E7D2 C042 6B8E 029E 13F2 C102 -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On 12/8/19 8:55 AM, Mathias Homann wrote:
Am Freitag, 6. Dezember 2019, 18:08:13 CET schrieb Michael Matz:
(Not trying to steal your thunder) An alternative for GUI lovers is qdirstat from Stefan Hundhammer. It can show the same graphically (File/Show unpackaged files...), including the usual features of it, like treemap and friends. (FWIW, I have only about 2000 unpackaged files below /usr :-) )
I installed that just now, and there's no "Show unpackaged files" menu option anywhere...
I have it installed (Tumbleweed) and "Show Unpackaged Files" is under the "File" menu for me. -- Jason Craig -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Adam Majer schrieb:
I've written a small utility that checks for files, symlinks and directories on your system that are installed in the system directories but are not owned by any package.
https://github.com/AdamMajer/rpmorphan
To compile this, all you need is rpm-devel and gcc.
Running on my Leap 15.1 system, it found quite a number of symlinks and directories and files that are not owned by any package. For example,
/usr/lib64/libreoffice/share/config/images_sifr.zip
is installed by not owned by any libreoffice packages. There are also number of files that install symlinks with update-alternatives but do not actually own these symlinks. The correct method is to install these as %ghost entries in the %files section.
Just recently I have been bitten by some stray font config file in /usr that messed up a system (Emojis are nice but not as default font...). It was really hard to spot in the mess left behind after years over years of zypper dup from one release to the next. IMO we need to pay more attention to making sure files are properly owned by packages. Then if there's anything left we know those files do not belong to the system. That helps if we wanted to offer a factory reset or cleanup feature. Or at least a way to visualize differences to how a clean installation would look like. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Jan Engelhardt schrieb:
On Friday 2019-12-06 17:37, Adam Majer wrote:
More interestingly, rpm itself doesn't own its own files either.
/usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ...
That is quite expected. I would not expect to lose the actual databases. `rpm -e rpm` should not remove (what was once) /var/lib/rpm, `rpm -e mariadb` should not remove /var/lib/mysql, and `rpm -e postfix` should not delete my mails.
I can see the case for removing rpm *and* the database in order to e.g. strip down container images. Which situation do you envision where you would want to remove the package but keep the db? What's missing from rpm indeed is a feature to somehow (recursively) flag data files and directories. Thinking of Android where it's possible to delete data and/or cache of apps. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Monday 2019-12-09 09:42, Ludwig Nussel wrote:
More interestingly, rpm itself doesn't own its own files either.
/usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ...
That is quite expected. I would not expect to lose the actual databases. `rpm -e rpm` should not remove (what was once) /var/lib/rpm, `rpm -e mariadb` should not remove /var/lib/mysql, and `rpm -e postfix` should not delete my mails.
I can see the case for removing rpm *and* the database in order to e.g. strip down container images. Which situation do you envision where you would want to remove the package but keep the db?
When you are installing an alternative by way of splitting deletion and new install into two separate transactions (sometimes that is possible, sometimes there is a Require on e.g. smtp_daemon or whatever that inhibits it in the first step, but still.). -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Jan Engelhardt schrieb:
On Monday 2019-12-09 09:42, Ludwig Nussel wrote:
More interestingly, rpm itself doesn't own its own files either.
/usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ...
That is quite expected. I would not expect to lose the actual databases. `rpm -e rpm` should not remove (what was once) /var/lib/rpm, `rpm -e mariadb` should not remove /var/lib/mysql, and `rpm -e postfix` should not delete my mails.
I can see the case for removing rpm *and* the database in order to e.g. strip down container images. Which situation do you envision where you would want to remove the package but keep the db?
When you are installing an alternative by way of splitting deletion and new install into two separate transactions (sometimes that is possible, sometimes there is a Require on e.g. smtp_daemon or whatever that inhibits it in the first step, but still.).
Referring to any package except rpm I guess. I meant rpm specifically. There is no alternative implementation that would use the rpmdb AFAIK :-) cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Hi, On Mon, 9 Dec 2019, Ludwig Nussel wrote:
/usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ...
That is quite expected. I would not expect to lose the actual databases. `rpm -e rpm` should not remove (what was once) /var/lib/rpm, `rpm -e mariadb` should not remove /var/lib/mysql, and `rpm -e postfix` should not delete my mails.
I can see the case for removing rpm *and* the database in order to e.g. strip down container images. Which situation do you envision where you would want to remove the package but keep the db?
When you are installing an alternative by way of splitting deletion and new install into two separate transactions (sometimes that is possible, sometimes there is a Require on e.g. smtp_daemon or whatever that inhibits it in the first step, but still.).
Referring to any package except rpm I guess. I meant rpm specifically. There is no alternative implementation that would use the rpmdb AFAIK :-)
But there could be (and there are consumers of the rpmdb, e.g. the libsolv-tools). So from fundamental principles I'd agree with Jan, removal of foo should not remove foo's user-data, and there's no reason why rpm should be special cased. (Of course, it would be nice to optionally do remove data associated but not owned by foo, as you say, but until we have that ...) Ciao, Michael. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Michael Matz schrieb:
On Mon, 9 Dec 2019, Ludwig Nussel wrote:
/usr/lib/sysimage/rpm/.rpm.lock /usr/lib/sysimage/rpm/Basenames /usr/lib/sysimage/rpm/Conflictname /usr/lib/sysimage/rpm/Dirnames /usr/lib/sysimage/rpm/Enhancename ...
That is quite expected. I would not expect to lose the actual databases. `rpm -e rpm` should not remove (what was once) /var/lib/rpm, `rpm -e mariadb` should not remove /var/lib/mysql, and `rpm -e postfix` should not delete my mails.
I can see the case for removing rpm *and* the database in order to e.g. strip down container images. Which situation do you envision where you would want to remove the package but keep the db?
When you are installing an alternative by way of splitting deletion and new install into two separate transactions (sometimes that is possible, sometimes there is a Require on e.g. smtp_daemon or whatever that inhibits it in the first step, but still.).
Referring to any package except rpm I guess. I meant rpm specifically. There is no alternative implementation that would use the rpmdb AFAIK :-)
But there could be (and there are consumers of the rpmdb, e.g. the libsolv-tools). So from fundamental principles I'd agree with Jan, removal of foo should not remove foo's user-data, and there's no reason why rpm should be special cased.
If it really was user data I'd probably agree. I don't think the analogy to mysql works here though. The rpm database is basically a collection of the rpm headers that belong to the content in /usr. A rollback of the system will also roll back that information. We wouldn't do that with actual user data. There is some relation of the data in the rpm db to eg /usr/share/info/dir. But then that one is constructed from other files in the system and could therefor just as well live in /var. Unlike the rpm headers that are not installed anywhere else. Also, what the special case is depends on the direction you are coming from. If everything in /usr needs to be owned by packages, not having the rpm db owned would be the special case :-) At the end of the day it's still a bit academic. I still wonder what the actual use case is. Also, rpm could in theory just generate a header into the DB that owns the DB itself :-) cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On 12/6/19 6:08 PM, Michael Matz wrote:
Running on my Leap 15.1 system, it found quite a number of symlinks and directories and files that are not owned by any package. For example,
/usr/lib64/libreoffice/share/config/images_sifr.zip
(Not trying to steal your thunder) An alternative for GUI lovers is qdirstat from Stefan Hundhammer. It can show the same graphically (File/Show unpackaged files...), including the usual features of it, like treemap and friends. (FWIW, I have only about 2000 unpackaged files below /usr :-) )
I would prefer to not have any thunder - this should list no or very few files ;) I wasn't aware of the GUI tool but it's nice someone else thought this is a problem. While my example with rpm may be a little funny and debatable, there is at least *one* file that it should own, /usr/lib/sysimage/rpm/.rpm.lock Nevertheless, the purpose here is that we have a lot of bugs in packages and the system installation can become inconsistent. At least there is no guarantee that an installed system is behaving correctly if we have these untracked files installed. /usr/lib/systemd/scripts/.migrate-sysconfig-i18n.sh~done hmmmm.. - Adam PS. I've renamed this tiny program to https://github.com/AdamMajer/rpm-orphan-files -- Adam Majer - amajer@suse.de SUSE Software Solutions Germany GmbH HRB 36809 (AG Nürnberg), GF: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
participants (9)
-
Adam Majer -
Hans-Peter Jansen -
Jan Engelhardt -
Jason Craig -
Kyrill Detinov -
Ludwig Nussel -
Mathias Homann -
Michael Matz -
Patrick Shanahan