Packages owning configuration files and configuration file samples: Current best practices?

newer
Re: Your confirmation is needed to...

Johannes Kastl

7 Apr 2024 7 Apr '24

14:24

Hi all, while working on Hashicorp Vault and OpenBao packages (stay tuned, mail to Factory will come soon) I stumbled upon something I did not have to do for quite a while. Hence my question what the current practice is on that. 1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)? Not having the file in the package means that querying RPM will not yield which package this file belongs to. Of course, the directory name might give a hint, but I would like to have rpm answer that question properly. My most educated guess whould be to %ghost those files. Not sure if that is the right way]tm]. :-) 2. How to treat example configuration files? Files that are not used and thus can be safely overwritten by package upgrades. My first idea was to put them into /etc/openbao, with e.g. a sample suffix. But packages should[tm] no longer write to /etc/, AFAIK. So it would be better to put them into /usr/share/doc/packages/openbao? Which also allows to safeley overwrite them on package upgrades. Would be great to get some recommendations! Thanks in advance! Kind Regards, Johannes

Show replies by date

Jan Engelhardt

7 Apr 7 Apr

19:05

On Sunday 2024-04-07 16:24, Johannes Kastl wrote:

...

1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)? Not having the file in the package means that querying RPM will not yield which package this file belongs to. Of course, the directory name might give a hint, but I would like to have rpm answer that question properly. My most educated guess whould be to %ghost those files. Not sure if that is the right way]tm]. :-)

/etc/passwd is not owned either, nor was it in SUSE 5.3.

...

2. How to treat example configuration files? Files that are not used and thus can be safely overwritten by package upgrades.

Examples are documentation. Relegate them to /usr/share/doc!

Johannes Kastl

8 Apr 8 Apr

04:44

Hi Jan, On 07.04.24 21:05 Jan Engelhardt wrote:

...

On Sunday 2024-04-07 16:24, Johannes Kastl wrote:

...
1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)?

...

/etc/passwd is not owned either, nor was it in SUSE 5.3.

Yes, but that file is a) widely known and b) not really belonging to a specific package. I would deem that a system-wide file.

...

...
2. How to treat example configuration files? Files that are not used and thus can be safely overwritten by package upgrades.

Examples are documentation. Relegate them to /usr/share/doc!

Just what I thought, thanks! Kind Regards, Johannes

Thorsten Kukuk

06:58

On Sun, Apr 7, 2024 at 4:24 PM Johannes Kastl wrote:

...

Hi all,

while working on Hashicorp Vault and OpenBao packages (stay tuned, mail to Factory will come soon) I stumbled upon something I did not have to do for quite a while. Hence my question what the current practice is on that.

1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)? Not having the file in the package means that querying RPM will not yield which package this file belongs to. Of course, the directory name might give a hint, but I would like to have rpm answer that question properly. My most educated guess whould be to %ghost those files. Not sure if that is the right way]tm]. :-)

%ghost entries are the worst you can do. I know here are several people who think files created by an admin should belong to a package, but: since RPM does not know anything about this file and if it got modified or not, making package changes here later is impossible. We have enough packages which suffer from this. So: don't use %ghost entries if not absolutely necessary. And: if the admin creates the file, it does not belong to the RPM! So don't create the wrong impression that this config file does belong to a RPM if it does not. Third: no RPM should install or own files in /etc, this includes %ghost. See https://en.opensuse.org/openSUSE:Packaging_UsrEtc I updated that page now with upstream references.

...

2. How to treat example configuration files? Files that are not used and thus can be safely overwritten by package upgrades

As somebody wrote already: mark them as %doc. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)

Johannes Kastl

10:55

Hi Thorsten, On 08.04.24 08:58 Thorsten Kukuk wrote:

...

On Sun, Apr 7, 2024 at 4:24 PM Johannes Kastl wrote:

...

...
1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)? Not having the file in the package means that querying RPM will not yield which package this file belongs to. Of course, the directory name might give a hint, but I would like to have rpm answer that question properly. My most educated guess whould be to %ghost those files. Not sure if that is the right way]tm]. :-)

%ghost entries are the worst you can do. I know here are several people who think files created by an admin should belong to a package, but: since RPM does not know anything about this file and if it got modified or not, making package changes here later is impossible. We have enough packages which suffer from this. So: don't use %ghost entries if not absolutely necessary.

OK.

...

And: if the admin creates the file, it does not belong to the RPM! So don't create the wrong impression that this config file does belong to a RPM if it does not.

Thanks for this insight. I have not looked at it from this point of view. My idea was more like "files is related to package" or "file is used by package".

...

Third: no RPM should install or own files in /etc, this includes %ghost. See https://en.opensuse.org/openSUSE:Packaging_UsrEtc I updated that page now with upstream references.

Hmmm, so not even the config directory should be owned by the package? Even if the application is not adapted to stuff being in /usr/etc/ or /etc? Kind Regards, Johannes

Neal Gompa

11:01

On Mon, Apr 8, 2024 at 6:55 AM Johannes Kastl wrote:

...

Hi Thorsten,

On 08.04.24 08:58 Thorsten Kukuk wrote:

...
On Sun, Apr 7, 2024 at 4:24 PM Johannes Kastl wrote:

...
...
1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)? Not having the file in the package means that querying RPM will not yield which package this file belongs to. Of course, the directory name might give a hint, but I would like to have rpm answer that question properly. My most educated guess whould be to %ghost those files. Not sure if that is the right way]tm]. :-)

%ghost entries are the worst you can do. I know here are several people who think files created by an admin should belong to a package, but: since RPM does not know anything about this file and if it got modified or not, making package changes here later is impossible. We have enough packages which suffer from this. So: don't use %ghost entries if not absolutely necessary.

OK.

...
And: if the admin creates the file, it does not belong to the RPM! So don't create the wrong impression that this config file does belong to a RPM if it does not.

Thanks for this insight. I have not looked at it from this point of view. My idea was more like "files is related to package" or "file is used by package".

This guidance is wrong. %ghost entries ensure that data is cleaned up only on final uninstall. Leaving random crap all over the filesystem even after uninstall is a nightmare. You should use %ghost for known names of files that you do not create and either the admin or the software creates.

...

...
Third: no RPM should install or own files in /etc, this includes %ghost. See https://en.opensuse.org/openSUSE:Packaging_UsrEtc I updated that page now with upstream references.

Hmmm, so not even the config directory should be owned by the package? Even if the application is not adapted to stuff being in /usr/etc/ or /etc?

The configuration directory should be owned by the package. -- 真実はいつも一つ！/ Always, there's only one truth!

Johannes Kastl

11:43

Hi Neal, On 08.04.24 13:01 Neal Gompa wrote:

...

On Mon, Apr 8, 2024 at 6:55 AM Johannes Kastl wrote:

...

...
...
And: if the admin creates the file, it does not belong to the RPM! So don't create the wrong impression that this config file does belong to a RPM if it does not.

Thanks for this insight. I have not looked at it from this point of view. My idea was more like "files is related to package" or "file is used by package".

This guidance is wrong. %ghost entries ensure that data is cleaned up only on final uninstall. Leaving random crap all over the filesystem even after uninstall is a nightmare. You should use %ghost for known names of files that you do not create and either the admin or the software creates.

Hmmm. Not sure if it is a good idea to remove the handcrafted config file just because the package is being uninstalled. But of course, now that I think of it, once the RPM got uninstalled rpm will no longer show a file as belonging to this RPM, either.

...

...
...
Third: no RPM should install or own files in /etc, this includes %ghost. See https://en.opensuse.org/openSUSE:Packaging_UsrEtc I updated that page now with upstream references.

Hmmm, so not even the config directory should be owned by the package? Even if the application is not adapted to stuff being in /usr/etc/ or /etc?

The configuration directory should be owned by the package.

What happens on uninstall, if only the directory is being owned by the package? Not the files within? Kind Regards, Johannes

Neal Gompa

11:45

On Mon, Apr 8, 2024 at 7:43 AM Johannes Kastl wrote:

...

Hi Neal,

On 08.04.24 13:01 Neal Gompa wrote:

...
On Mon, Apr 8, 2024 at 6:55 AM Johannes Kastl wrote:

...
...
...
And: if the admin creates the file, it does not belong to the RPM! So don't create the wrong impression that this config file does belong to a RPM if it does not.

Thanks for this insight. I have not looked at it from this point of view. My idea was more like "files is related to package" or "file is used by package".

This guidance is wrong. %ghost entries ensure that data is cleaned up only on final uninstall. Leaving random crap all over the filesystem even after uninstall is a nightmare. You should use %ghost for known names of files that you do not create and either the admin or the software creates.

Hmmm. Not sure if it is a good idea to remove the handcrafted config file just because the package is being uninstalled. But of course, now that I think of it, once the RPM got uninstalled rpm will no longer show a file as belonging to this RPM, either.

...
...
...
Third: no RPM should install or own files in /etc, this includes %ghost. See https://en.opensuse.org/openSUSE:Packaging_UsrEtc I updated that page now with upstream references.

Hmmm, so not even the config directory should be owned by the package? Even if the application is not adapted to stuff being in /usr/etc/ or /etc?

The configuration directory should be owned by the package.

What happens on uninstall, if only the directory is being owned by the package? Not the files within?

RPM throws a warning and doesn't clean the directory out due to untracked files in there. -- 真実はいつも一つ！/ Always, there's only one truth!

Andrei Borzenkov

12:14

On Mon, Apr 8, 2024 at 2:44 PM Johannes Kastl wrote:

...

Hmmm. Not sure if it is a good idea to remove the handcrafted config file just because the package is being uninstalled. But of course, now that I think of it, once the RPM got uninstalled rpm will no longer show a file as belonging to this RPM, either.

This is one thing dpkg does better. It distinguishes between "uninstall" and "purge". "uninstall" keeps (modified) configuration files and "purge" wipes them out. I wonder if this was ever considered for RPM.

Thorsten Kukuk

12:17

On Mon, Apr 8, 2024 at 2:15 PM Andrei Borzenkov wrote:

...

On Mon, Apr 8, 2024 at 2:44 PM Johannes Kastl wrote:

...
Hmmm. Not sure if it is a good idea to remove the handcrafted config file just because the package is being uninstalled. But of course, now that I think of it, once the RPM got uninstalled rpm will no longer show a file as belonging to this RPM, either.

This is one thing dpkg does better. It distinguishes between "uninstall" and "purge". "uninstall" keeps (modified) configuration files and "purge" wipes them out. I wonder if this was ever considered for RPM.

It does if you use %config and not %ghost. With %ghost RPM cannot detect if the file got modified by the admin or not. Another reason why %ghost is evil. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)

Andrei Borzenkov

12:20

On Mon, Apr 8, 2024 at 3:17 PM Thorsten Kukuk wrote:

...

On Mon, Apr 8, 2024 at 2:15 PM Andrei Borzenkov wrote:

...
On Mon, Apr 8, 2024 at 2:44 PM Johannes Kastl wrote:

...
Hmmm. Not sure if it is a good idea to remove the handcrafted config file just because the package is being uninstalled. But of course, now that I think of it, once the RPM got uninstalled rpm will no longer show a file as belonging to this RPM, either.

This is one thing dpkg does better. It distinguishes between "uninstall" and "purge". "uninstall" keeps (modified) configuration files and "purge" wipes them out. I wonder if this was ever considered for RPM.

It does if you use %config and not %ghost.

How? If you remove a package, it is gone from the database. It is not (only) about removing configuration files, it is in the first place about tracking which package was using these configuration files after the package has been removed.

Jan Engelhardt

15:02

On Monday 2024-04-08 14:14, Andrei Borzenkov wrote:

...

On Mon, Apr 8, 2024 at 2:44 PM Johannes Kastl wrote:

...
Hmmm. Not sure if it is a good idea to remove the handcrafted config file just because the package is being uninstalled. But of course, now that I think of it, once the RPM got uninstalled rpm will no longer show a file as belonging to this RPM, either.

This is one thing dpkg does better. It distinguishes between "uninstall" and "purge". "uninstall" keeps (modified) configuration files and "purge" wipes them out. I wonder if this was ever considered for RPM.

install freeswan uninstall freeswan (/etc/ipsec.conf remains, being tracked as belonging to freeswan) install strongswan uninstall strongswan Now what: Do you associate ipsec.conf with freeswan or strongswan? Both? Neither? (Someone has probably made a choice in the apt implementation to solve this conundrum one way or the other. -- And RPM chose its solution to just not bother with purge tracking.)

Johannes Meixner

12:21

Hello, On 2024-04-08 13:43, Johannes Kastl wrote:

...

On 08.04.24 13:01 Neal Gompa wrote:

...
... %ghost entries ensure that data is cleaned up only on final uninstall. Leaving random crap all over the filesystem even after uninstall is a nightmare. You should use %ghost for known names of files that you do not create and either the admin or the software creates.

Hmmm. Not sure if it is a good idea to remove the handcrafted config file just because the package is being uninstalled. But of course, now that I think of it, once the RPM got uninstalled rpm will no longer show a file as belonging to this RPM, either.

All user's own data is sacrosanct. No automatism must remove user's own data. Removing user's own data requires explicit user confirmation. E.g. when a program has an interactive user confirmation dialog or when a program has a command line option which the user must deliberately specify, then it is OK when a program removes user's own data. Simply put: Only the user decides what happens with his own data. It is a different thing what programs currently do, in particular when something is done in a certain way since a very long time (in IT time scale), for example: ---------------------------------------------------------- $ echo "my data" >a $ echo "my other data" >b $ cp a b ---------------------------------------------------------- 'cp' overwrites "my other data" so this data is "removed". FWIW: This pattern is the only one how I unintendedly removed my own data in the past (I think two or three times). Kind Regards Johannes Meixner -- SUSE Software Solutions Germany GmbH Frankenstr. 146 - 90461 Nuernberg - Germany (HRB 36809, AG Nuernberg) GF: Ivo Totev

Thorsten Kukuk

12:15

On Mon, Apr 8, 2024 at 1:02 PM Neal Gompa wrote:

...

On Mon, Apr 8, 2024 at 6:55 AM Johannes Kastl wrote:

...
Hi Thorsten,

On 08.04.24 08:58 Thorsten Kukuk wrote:

...
On Sun, Apr 7, 2024 at 4:24 PM Johannes Kastl wrote:

...
...
1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)? Not having the file in the package means that querying RPM will not yield which package this file belongs to. Of course, the directory name might give a hint, but I would like to have rpm answer that question properly. My most educated guess whould be to %ghost those files. Not sure if that is the right way]tm]. :-)

%ghost entries are the worst you can do. I know here are several

...
...
think files created by an admin should belong to a package, but: since RPM does not know anything about this file and if it got modified or not, making package changes here later is impossible. We have enough

people who packages

...
...
which suffer from this. So: don't use %ghost entries if not absolutely necessary.

OK.

...
And: if the admin creates the file, it does not belong to the RPM! So don't create the wrong impression that this config file does belong to a RPM if it does not.

Thanks for this insight. I have not looked at it from this point of view. My idea was more like "files is related to package" or "file is used by package".

This guidance is wrong. %ghost entries ensure that data is cleaned up only on final uninstall. Leaving random crap all over the filesystem even after uninstall is a nightmare. You should use %ghost for known names of files that you do not create and either the admin or the software creates.

And this guidance is wrong, since your only example ignores all the problems we face with %ghost and has drawbacks. %ghost files will always get completely removed without *.rpmsave file, which also makes it impossible to e.g. rename the RPM if necessary. %ghost files are becoming a pain in the ass after some years, as written, we have enough examples. And as written, the file does not belong to the RPM, it's created by the admin. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)

Johannes Kastl

13 Apr 13 Apr

20:24

On 07.04.24 16:24 Johannes Kastl wrote:

...

1. How should I handle files in /etc/openbao that should "belong" to the package, but should not be packaged at all (the admin needs to create that herself)?

...

2. How to treat example configuration files? Files that are not used and thus can be safely overwritten by package upgrades.

I adjusted the vault package with the recommendations in the thread:

...

https://build.opensuse.org/package/show/home:ojkastl_buildservice:Branch_sec...

The openbao package will be adjusted to match this package, once openbao reaches a final release (openbao does not include the UI currently, for example). The vault package will NOT get submitted to Factory, due to the license, but it is in a working state and serves as a playground. If anyone wants to comment on the package, feel free... Kind Regards, Johannes

Age (days ago)

Last active (days ago)

List overview

Download

14 comments

6 participants

participants (6)

Andrei Borzenkov
Jan Engelhardt
Johannes Kastl
Johannes Meixner
Neal Gompa
Thorsten Kukuk

Packages owning configuration files and configuration file samples: Current best practices?

tags

participants (6)