[RFC] Switching to drop-in files (systemd)
Hi, The way the systemd daemons (such as PID1, logind) parse their configuration file is specified in systemd-system.conf(5) man page (section CONFIGURATION DIRECTORIES AND PRECEDENCE). Basically the daemons parse first their "main" configuration file (e.g /etc/systemd/system.conf for PID1) and then read and apply the optional drop-in files located in the following directories: /usr/lib/systemd/*.conf.d/ /usr/local/lib/systemd/*.conf.d/ /run/systemd/*.conf.d/ /etc/systemd/*.conf.d/ It's important to note that drop-in files have higher precedence than the main configuration file. Hence if a package customizes an option by shipping a drop-in file in /usr/lib/systemd/*.conf.d/, it might override the value already set by the user if the same option was customized by the sysadmin in the main configuration. There were various discussions [1] about addressing this concern but at the end upstream preferred keeping the existing logic. This implies that using the main configuration files is not 100% safe (even if it's been documented since a long time) especially since openSUSE distros relies on drop-in files to tune a few upstream defaults, see systemd-default-settings package to get the list. To avoid inconvenient divergences with upstream, we'll follow the existing logic but this implies that: - use of main config files, such as /etc/systemd/{system.conf,logind.conf}, should be avoided and instead customization should be done via drop-in files, such as /etc/systemd/system.conf.d/50-local.conf. - Some drop-in priority ranges should be defined to make sure that drop-in created by users have always higher precedence over the ones shipped by packages. The priority ranges reserved for users and downstream will be defined and documented in the relevant wiki page. - at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined). - optionally, the packaging stuff will take care of converting existing main configuration files into drop-in files during package update. If there's no objection with the above, this change will be probably take place during the upgrade to systemd v254 or quickly after. Thanks. [1] https://github.com/systemd/systemd/pull/17161
Moin, to be honest, I'm afraid the whole UsrMerge activitiy told us something completely different! On Fri, 11 Aug 2023, 18:40:44 +0200, Franck Bui wrote:
Hi,
The way the systemd daemons (such as PID1, logind) parse their configuration file is specified in systemd-system.conf(5) man page (section CONFIGURATION DIRECTORIES AND PRECEDENCE).
Basically the daemons parse first their "main" configuration file (e.g /etc/systemd/system.conf for PID1) and then read and apply the optional drop-in files located in the following directories:
/usr/lib/systemd/*.conf.d/ /usr/local/lib/systemd/*.conf.d/ /run/systemd/*.conf.d/ /etc/systemd/*.conf.d/
*If* systemd (PID1) has access to /usr/lib/systemd/system.conf, it *must* use that file - unless there is a file /etc/systemd/system.conf. During the process to migrate all /{bin,etc,lib}/* files, we were told to leave the file at /usr/* untouched and make our local modifications to the files at /etc/... Now systemd wants to tell us a different logic :(
It's important to note that drop-in files have higher precedence than the main configuration file. Hence if a package customizes an option by shipping a drop-in file in /usr/lib/systemd/*.conf.d/, it might override the value already set by the user if the same option was customized by the sysadmin in the main configuration.
There were various discussions [1] about addressing this concern but at the end upstream preferred keeping the existing logic. This implies that using the main configuration files is not 100% safe (even if it's been documented since a long time) especially since openSUSE distros relies on drop-in files to tune a few upstream defaults, see systemd-default-settings package to get the list.
To avoid inconvenient divergences with upstream, we'll follow the existing logic but this implies that:
- use of main config files, such as /etc/systemd/{system.conf,logind.conf}, should be avoided and instead customization should be done via drop-in files, such as /etc/systemd/system.conf.d/50-local.conf.
- Some drop-in priority ranges should be defined to make sure that drop-in created by users have always higher precedence over the ones shipped by packages. The priority ranges reserved for users and downstream will be defined and documented in the relevant wiki page.
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
- optionally, the packaging stuff will take care of converting existing main configuration files into drop-in files during package update.
If there's no objection with the above, this change will be probably take place during the upgrade to systemd v254 or quickly after.
Again, to be honest, we had a lot of effort and work to teach the various packages to be able to deal with two different configuration directories below /usr and /etc for overriding stuff, now systemd wants to revert that? I strongly vote NO!
Thanks.
Cheers. l8er manfred
On 11.08.2023 19:59, Manfred Hollstein wrote:
Moin,
to be honest, I'm afraid the whole UsrMerge activitiy told us something completely different!
On Fri, 11 Aug 2023, 18:40:44 +0200, Franck Bui wrote:
Hi,
The way the systemd daemons (such as PID1, logind) parse their configuration file is specified in systemd-system.conf(5) man page (section CONFIGURATION DIRECTORIES AND PRECEDENCE).
Basically the daemons parse first their "main" configuration file (e.g /etc/systemd/system.conf for PID1) and then read and apply the optional drop-in files located in the following directories:
/usr/lib/systemd/*.conf.d/ /usr/local/lib/systemd/*.conf.d/ /run/systemd/*.conf.d/ /etc/systemd/*.conf.d/
*If* systemd (PID1) has access to /usr/lib/systemd/system.conf, it *must* use that file - unless there is a file /etc/systemd/system.conf.
During the process to migrate all /{bin,etc,lib}/* files, we were told to leave the file at /usr/* untouched and make our local modifications to the files at /etc/...
Now systemd wants to tell us a different logic :(
It's important to note that drop-in files have higher precedence than the main configuration file. Hence if a package customizes an option by shipping a drop-in file in /usr/lib/systemd/*.conf.d/, it might override the value already set by the user if the same option was customized by the sysadmin in the main configuration.
There were various discussions [1] about addressing this concern but at the end upstream preferred keeping the existing logic. This implies that using the main configuration files is not 100% safe (even if it's been documented since a long time) especially since openSUSE distros relies on drop-in files to tune a few upstream defaults, see systemd-default-settings package to get the list.
To avoid inconvenient divergences with upstream, we'll follow the existing logic but this implies that:
- use of main config files, such as /etc/systemd/{system.conf,logind.conf}, should be avoided and instead customization should be done via drop-in files, such as /etc/systemd/system.conf.d/50-local.conf.
- Some drop-in priority ranges should be defined to make sure that drop-in created by users have always higher precedence over the ones shipped by packages. The priority ranges reserved for users and downstream will be defined and documented in the relevant wiki page.
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
- optionally, the packaging stuff will take care of converting existing main configuration files into drop-in files during package update.
If there's no objection with the above, this change will be probably take place during the upgrade to systemd v254 or quickly after.
Again, to be honest, we had a lot of effort and work to teach the various packages to be able to deal with two different configuration directories below /usr and /etc for overriding stuff, now systemd wants to revert that? I strongly vote NO!
NO to what? Doing nothing and leaving it as is will not magically make systemd honor /usr/lib/systemd/system.conf, nor will it stop it from overriding settings from /etc/systemd/system.conf with something from /usr/lib/sytemd/system.conf.d. That said, I do not think this change will be beneficial short term. Users will continue to edit /etc/systemd/system.conf because they found description on Internet. Having link /etc/systemd/system.conf -> /etc/systemd/system.conf.d/50-main.conf mai mitigate it as long as systemd allows it in the first place.
Thanks.
Cheers.
l8er manfred
On Fri, 11 Aug 2023, 19:13:05 +0200, Andrei Borzenkov wrote:
On 11.08.2023 19:59, Manfred Hollstein wrote:
Moin,
to be honest, I'm afraid the whole UsrMerge activitiy told us something completely different!
On Fri, 11 Aug 2023, 18:40:44 +0200, Franck Bui wrote:
Hi,
The way the systemd daemons (such as PID1, logind) parse their configuration file is specified in systemd-system.conf(5) man page (section CONFIGURATION DIRECTORIES AND PRECEDENCE).
Basically the daemons parse first their "main" configuration file (e.g /etc/systemd/system.conf for PID1) and then read and apply the optional drop-in files located in the following directories:
/usr/lib/systemd/*.conf.d/ /usr/local/lib/systemd/*.conf.d/ /run/systemd/*.conf.d/ /etc/systemd/*.conf.d/
*If* systemd (PID1) has access to /usr/lib/systemd/system.conf, it *must* use that file - unless there is a file /etc/systemd/system.conf.
During the process to migrate all /{bin,etc,lib}/* files, we were told to leave the file at /usr/* untouched and make our local modifications to the files at /etc/...
Now systemd wants to tell us a different logic :(
It's important to note that drop-in files have higher precedence than the main configuration file. Hence if a package customizes an option by shipping a drop-in file in /usr/lib/systemd/*.conf.d/, it might override the value already set by the user if the same option was customized by the sysadmin in the main configuration.
There were various discussions [1] about addressing this concern but at the end upstream preferred keeping the existing logic. This implies that using the main configuration files is not 100% safe (even if it's been documented since a long time) especially since openSUSE distros relies on drop-in files to tune a few upstream defaults, see systemd-default-settings package to get the list.
To avoid inconvenient divergences with upstream, we'll follow the existing logic but this implies that:
- use of main config files, such as /etc/systemd/{system.conf,logind.conf}, should be avoided and instead customization should be done via drop-in files, such as /etc/systemd/system.conf.d/50-local.conf.
- Some drop-in priority ranges should be defined to make sure that drop-in created by users have always higher precedence over the ones shipped by packages. The priority ranges reserved for users and downstream will be defined and documented in the relevant wiki page.
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
- optionally, the packaging stuff will take care of converting existing main configuration files into drop-in files during package update.
If there's no objection with the above, this change will be probably take place during the upgrade to systemd v254 or quickly after.
Again, to be honest, we had a lot of effort and work to teach the various packages to be able to deal with two different configuration directories below /usr and /etc for overriding stuff, now systemd wants to revert that? I strongly vote NO!
NO to what? Doing nothing and leaving it as is will not magically make systemd honor /usr/lib/systemd/system.conf, nor will it stop it from overriding settings from /etc/systemd/system.conf with something from /usr/lib/sytemd/system.conf.d.
That said, I do not think this change will be beneficial short term. Users will continue to edit /etc/systemd/system.conf because they found description on Internet. Having link /etc/systemd/system.conf -> /etc/systemd/system.conf.d/50-main.conf mai mitigate it as long as systemd allows it in the first place.
NO to what?
NO to systemd ignoring the results of the UsrMerge! *They* must accept that there are some standards and code their stuff accordingly! This was an RFC, wasn't it? This is my comment!
Thanks.
Cheers. l8er manfred
On Fri, Aug 11, Manfred Hollstein wrote:
Moin,
to be honest, I'm afraid the whole UsrMerge activitiy told us something completely different!
No, it does not.
On Fri, 11 Aug 2023, 18:40:44 +0200, Franck Bui wrote:
Hi,
The way the systemd daemons (such as PID1, logind) parse their configuration file is specified in systemd-system.conf(5) man page (section CONFIGURATION DIRECTORIES AND PRECEDENCE).
Basically the daemons parse first their "main" configuration file (e.g /etc/systemd/system.conf for PID1) and then read and apply the optional drop-in files located in the following directories:
/usr/lib/systemd/*.conf.d/ /usr/local/lib/systemd/*.conf.d/ /run/systemd/*.conf.d/ /etc/systemd/*.conf.d/
*If* systemd (PID1) has access to /usr/lib/systemd/system.conf, it *must* use that file - unless there is a file /etc/systemd/system.conf.
During the process to migrate all /{bin,etc,lib}/* files, we were told to leave the file at /usr/* untouched and make our local modifications to the files at /etc/...
Now systemd wants to tell us a different logic :(
No, it is not about /usr/lib/systemd/system.conf vs /etc/systemd/system.conf, it is about the logic how to handle ".d" directories and snippets. You are mixing up two differnt things. Read the next sentence again:
It's important to note that drop-in files have higher precedence than the main configuration file. Hence if a package customizes an option by shipping a drop-in file in /usr/lib/systemd/*.conf.d/, it might override the value already set by the user if the same option was customized by the sysadmin in the main configuration.
Again, to be honest, we had a lot of effort and work to teach the various packages to be able to deal with two different configuration directories below /usr and /etc for overriding stuff, now systemd wants to revert that? I strongly vote NO!
No, systemd does not revert anything and none of our effords is void with this. Between, libeconf behaves in the same way with with drop-in files as systemd. And with drop-in files, the main configuration file should no longer be used at all. For other parts is already normal, or do you know anybody who is still using /etc/pam.conf? So systemd is only consequent here. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
Fri, 11 Aug 2023 18:40:44 +0200 Franck Bui <fbui@suse.de>:
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
systemd is well documented, systemd.directives(7) is one potential entry point. Please stop polluting /etc with empty and rpm-owned configuration files. Thanks, Olaf
On Sat, Aug 12, Olaf Hering wrote:
Fri, 11 Aug 2023 18:40:44 +0200 Franck Bui <fbui@suse.de>:
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
systemd is well documented, systemd.directives(7) is one potential entry point.
Please stop polluting /etc with empty and rpm-owned configuration files.
+1 Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Sat, 2023-08-12 at 09:01 +0200, Olaf Hering wrote:
Fri, 11 Aug 2023 18:40:44 +0200 Franck Bui <fbui@suse.de>:
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50- {local,main,xxx}.conf, exact filename to be defined).
systemd is well documented, systemd.directives(7) is one potential entry point.
Please stop polluting /etc with empty and rpm-owned configuration files.
I've always found systemd's (past?) policy to ship config files with every directive commented out very convenient. All relevant config options and their defaults in a short, concise form, very easy to find the one setting that you want to modify, and very easy to grok some months later where an option had been changed. But having to copy stuff from /usr/lib to /etc has become commonplace, so perhaps storing the default config file as /usr/lib/systemd/xxx.conf or /usr/lib/systemd/xxx.d/00-defaults.conf should be sufficient. Martin
On 8/14/23 12:34, Martin Wilck via openSUSE Factory wrote:
But having to copy stuff from /usr/lib to /etc has become commonplace, so perhaps storing the default config file as /usr/lib/systemd/xxx.conf or /usr/lib/systemd/xxx.d/00-defaults.conf should be sufficient.
Wouldn't that suggest to override the defaults with /etc/systemd/xxx.conf or /etc/systemd/xxx.d/00-defaults.conf ?
On Thu, 2023-08-17 at 16:10 +0200, Franck Bui wrote:
On 8/14/23 12:34, Martin Wilck via openSUSE Factory wrote:
But having to copy stuff from /usr/lib to /etc has become commonplace, so perhaps storing the default config file as /usr/lib/systemd/xxx.conf or /usr/lib/systemd/xxx.d/00-defaults.conf should be sufficient.
Wouldn't that suggest to override the defaults with /etc/systemd/xxx.conf or /etc/systemd/xxx.d/00-defaults.conf ?
In a way, yes. I suppose the file would have comments at the top, hinting how to override the settings. Actually the files contain such hints already, e.g. in /etc/systemd.journald.conf: # Entries in this file show the compile time defaults. Local configuration # should be created by either modifying this file, or by creating "drop-ins" in # the journald.conf.d/ subdirectory. The latter is generally recommended. # Defaults can be restored by simply deleting this file and all drop-ins. # # Use 'systemd-analyze cat-config systemd/journald.conf' to display the full config. The text "by either modifying this file" would need to be adapted, and a hint would need to be added that drop-ins under /usr/lib override settings from /etc/journald.conf. Otherwise this should be fine. OTOH, if people memorize "systemd-analyze cat-config", we won't need these hints any more. Will they? Martin
On 8/18/23 11:42, Martin Wilck via openSUSE Factory wrote:
In a way, yes. I suppose the file would have comments at the top, hinting how to override the settings. Actually the files contain such hints already, e.g. in /etc/systemd.journald.conf:
# Entries in this file show the compile time defaults. Local configuration # should be created by either modifying this file, or by creating "drop-ins" in # the journald.conf.d/ subdirectory. The latter is generally recommended. # Defaults can be restored by simply deleting this file and all drop-ins. # # Use 'systemd-analyze cat-config systemd/journald.conf' to display the full config.
The text "by either modifying this file" would need to be adapted, and a hint would need to be added that drop-ins under /usr/lib override settings from /etc/journald.conf. Otherwise this should be fine.
Since the goal is to not install files in /etc anymore that sounds the best approach, yes.
OTOH, if people memorize "systemd-analyze cat-config", we won't need these hints any more. Will they?
No they won't indeed but I would still keep the hints for those who don't have a clue about systemd-analyze yet. Thanks.
On 8/12/23 09:01, Olaf Hering wrote:
Fri, 11 Aug 2023 18:40:44 +0200 Franck Bui <fbui@suse.de>:
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
systemd is well documented, systemd.directives(7) is one potential entry point.
Please stop polluting /etc with empty and rpm-owned configuration files.
That would be the simplest way to switch to drop-in files only from a packager POV. However I think that a lot of users would be confused by the fact that the main config files are missing. And even worse would be tempted to reintroduce them when they would need to override the defaults as the use of drop-ins is not wide spread yet. The advantages of shipping /etc/systemd/*.conf.d/50-local.conf are: - it implicitly teaches users to use a drop-in file to override defaults - it list all config options and their default in a concise form (as pointed out by Martin) which is convenient. So I would still be tempted to ship the "template" drop-ins at least for now and make them go away in the future when drop-ins will be common practice. That would be a more user-friendly transition IMHO. Thanks.
On Thu, Aug 17, Franck Bui wrote:
On 8/12/23 09:01, Olaf Hering wrote:
Fri, 11 Aug 2023 18:40:44 +0200 Franck Bui <fbui@suse.de>:
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
systemd is well documented, systemd.directives(7) is one potential entry point.
Please stop polluting /etc with empty and rpm-owned configuration files.
That would be the simplest way to switch to drop-in files only from a packager POV.
However I think that a lot of users would be confused by the fact that the main config files are missing. And even worse would be tempted to reintroduce them when they would need to override the defaults as the use of drop-ins is not wide spread yet.
I disagree. Other tools did that in the past, too (most prominent one: Linux-PAM). It did take some time until people did start learning, but in the end it was quite easy. Much simpler then having the pain to update this config files later.
The advantages of shipping /etc/systemd/*.conf.d/50-local.conf are:
- it implicitly teaches users to use a drop-in file to override defaults
If they are not aware of drop-ins they will follow the search engine hits and create the main config files again.
- it list all config options and their default in a concise form (as pointed out by Martin) which is convenient.
This is not convenient, this is the major pain for updates. Everytime, a config changes upstram and an admin made a local change, we will either have rpmsave or rpmnew files, which makes the whole thing void and all advantages are gone. And you will never get this cleaned up again.
So I would still be tempted to ship the "template" drop-ins at least for now and make them go away in the future when drop-ins will be common practice. That would be a more user-friendly transition IMHO.
Sorry, but with todays RPM you will never get rid of them again, we have enough other packages with exactly the same problem. Don't repeat all the mistakes we made in the past because somebody wrongly claims "it's more user friendly", in the end it is not. But learn from it and make it better. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Thu, Aug 17, 2023 at 02:38:42PM +0000, Thorsten Kukuk wrote:
On Thu, Aug 17, Franck Bui wrote:
On 8/12/23 09:01, Olaf Hering wrote:
Fri, 11 Aug 2023 18:40:44 +0200 Franck Bui <fbui@suse.de>:
- at installation, systemd packages will stop installing (empty/template) main configuration files and instead users will find the templates in drop-in directories (such as /etc/systemd/*.conf.d/50-{local,main,xxx}.conf, exact filename to be defined).
systemd is well documented, systemd.directives(7) is one potential entry point.
Please stop polluting /etc with empty and rpm-owned configuration files.
That would be the simplest way to switch to drop-in files only from a packager POV.
However I think that a lot of users would be confused by the fact that the main config files are missing. And even worse would be tempted to reintroduce them when they would need to override the defaults as the use of drop-ins is not wide spread yet.
I disagree. Other tools did that in the past, too (most prominent one: Linux-PAM). It did take some time until people did start learning, but in the end it was quite easy. Much simpler then having the pain to update this config files later.
These are much less likely to be changed.
The advantages of shipping /etc/systemd/*.conf.d/50-local.conf are:
- it implicitly teaches users to use a drop-in file to override defaults
If they are not aware of drop-ins they will follow the search engine hits and create the main config files again.
If the dropin is already there it's quite possible they will notice it.
- it list all config options and their default in a concise form (as pointed out by Martin) which is convenient.
This is not convenient, this is the major pain for updates. Everytime, a config changes upstram and an admin made a local change, we will either have rpmsave or rpmnew files, which makes the whole thing void and all advantages are gone. And you will never get this cleaned up again.
With the fifes shipping only documentation and no actual settings there is absolutely no problem with rpmnew files. Some people might clean them up, some not, whatever. Thanks Michal
On Thu, Aug 17, Michal Suchánek wrote:
On Thu, Aug 17, 2023 at 02:38:42PM +0000, Thorsten Kukuk wrote:
I disagree. Other tools did that in the past, too (most prominent one: Linux-PAM). It did take some time until people did start learning, but in the end it was quite easy. Much simpler then having the pain to update this config files later.
These are much less likely to be changed.
It doesn't matter how often they get changed, once is already one time too often. And if they should contain the default settings as comment, they will be changed very frequently.
- it list all config options and their default in a concise form (as pointed out by Martin) which is convenient.
This is not convenient, this is the major pain for updates. Everytime, a config changes upstram and an admin made a local change, we will either have rpmsave or rpmnew files, which makes the whole thing void and all advantages are gone. And you will never get this cleaned up again.
With the fifes shipping only documentation and no actual settings there is absolutely no problem with rpmnew files.
It is a problem because it contradicts the original idea to have the current settings there to make it easier for the admin. If you don't care if the defaults are valid, complete and correct, then we don't need them at all. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Thu, Aug 17, 2023 at 02:55:21PM +0000, Thorsten Kukuk wrote:
On Thu, Aug 17, Michal Suchánek wrote:
On Thu, Aug 17, 2023 at 02:38:42PM +0000, Thorsten Kukuk wrote:
I disagree. Other tools did that in the past, too (most prominent one: Linux-PAM). It did take some time until people did start learning, but in the end it was quite easy. Much simpler then having the pain to update this config files later.
These are much less likely to be changed.
It doesn't matter how often they get changed, once is already one time too often.
That Linux-PAM settings are less likely to be changed means there are fewer users affected by the settings migration. Almost everyone changes systemd settings, changing PAM settings is much more esoteric.
And if they should contain the default settings as comment, they will be changed very frequently.
- it list all config options and their default in a concise form (as pointed out by Martin) which is convenient.
This is not convenient, this is the major pain for updates. Everytime, a config changes upstram and an admin made a local change, we will either have rpmsave or rpmnew files, which makes the whole thing void and all advantages are gone. And you will never get this cleaned up again.
With the fifes shipping only documentation and no actual settings there is absolutely no problem with rpmnew files.
It is a problem because it contradicts the original idea to have the current settings there to make it easier for the admin. If you don't care if the defaults are valid, complete and correct, then we don't need them at all.
They are valid, complete, and correct so long as the user did not change the file, that is the first time they see it. The current, complete, and valid defaults will be in .rpmnew file if the user did change the file. The documentation can even advise to create separate file for the user setting. Thanks Michal
Thu, 17 Aug 2023 16:51:10 +0200 Michal Suchánek <msuchanek@suse.de>:
With the fifes shipping only documentation and no actual settings there is absolutely no problem with rpmnew files.
So you are saying we will never reach the goal of "no rpm-owned files in /etc"? Was that even a goal? Perhaps it was just an idea, and the current state has no practical downside. Olaf
On 2023-08-17 16:57, Olaf Hering wrote:
Thu, 17 Aug 2023 16:51:10 +0200 Michal Suchánek <msuchanek@suse.de>:
With the fifes shipping only documentation and no actual settings there is absolutely no problem with rpmnew files.
So you are saying we will never reach the goal of "no rpm-owned files in /etc"? Was that even a goal? Perhaps it was just an idea, and the current state has no practical downside.
Olaf
The goal of "no rpm-owned files in /etc" is very much a goal Distribution provided packages should avoid packaging fils in user-interactively managed directories like /home, /srv, /var and /etc In an ideal world all of our distribution provided configuration would be somewhere under /usr, such as /usr/lib already used by systemd, or /usr/etc used by various other packages /etc would then be used purely for user-provided overrides, either in the form of drop-ins (preferred as they are more likely to layer nicely with changes to the distribution-provided defaults) or in the form of replacement configuration files. In an ideal world, rpmnew files should never be triggered. We should work towards that ideal, and not accept the current status quo as acceptable. -- Richard Brown Distributions Architect SUSE Software Solutions Germany GmbH, Frankenstraße 146, D-90461 Nuremberg, Germany (HRB 36809, AG Nürnberg) Managing Directors/Geschäftsführer: Ivo Totev, Andrew McDonald, Werner Knoblich
On 8/18/23 13:04, Richard Brown wrote:
We should work towards that ideal, and not accept the current status quo as acceptable.
Ok then we'll head toward this goal by stopping shipping the various configuration files from systemd in /etc. Should we still consider migrating the current main config files into drop-ins ? Thanks all for the inputs.
On 2023-08-21 11:27, Franck Bui wrote:
On 8/18/23 13:04, Richard Brown wrote:
We should work towards that ideal, and not accept the current status quo as acceptable.
Ok then we'll head toward this goal by stopping shipping the various configuration files from systemd in /etc.
Should we still consider migrating the current main config files into drop-ins ?
I think migrating an existing systems contents of /etc is highly risky I say go ahead for the cases where you KNOW the contents in /etc are a 1:1 match for what you want to have in /usr Else, I think it's best to leave them there, with the assumption that the current user is happy with their current config as defined in /etc After all, /etc really should be seen as belonging to the users..so migrating/deleting anything from there really should be done with only the most extreme care -- Richard Brown Distributions Architect SUSE Software Solutions Germany GmbH, Frankenstraße 146, D-90461 Nuremberg, Germany (HRB 36809, AG Nürnberg) Managing Directors/Geschäftsführer: Ivo Totev, Andrew McDonald, Werner Knoblich
On 8/21/23 15:27, Richard Brown wrote:
Else, I think it's best to leave them there, with the assumption that the current user is happy with their current config as defined in /etc
Leaving them might be the best option indeed but is there a way to instruct rpm to not convert them into .rpmsave backup files when the package will be updated? IMU an .rpmsave file will be created each time rpm notices that a %config file has been modified by the user and is no more part of the package file list of the new package version. Thanks.
On Monday 2023-08-21 16:03, Franck Bui wrote:
On 8/21/23 15:27, Richard Brown wrote:
Else, I think it's best to leave them there, with the assumption that the current user is happy with their current config as defined in /etc
Leaving them might be the best option indeed but is there a way to instruct rpm to not convert them into .rpmsave backup files when the package will be updated?
There is (sort of) with loads of %post and %triggerpostun. I wouldn't invest the time again to find the snippet or re-implement it from scratch.
Franck Bui wrote:
On 8/21/23 15:27, Richard Brown wrote:
Else, I think it's best to leave them there, with the assumption that the current user is happy with their current config as defined in /etc
Leaving them might be the best option indeed but is there a way to instruct rpm to not convert them into .rpmsave backup files when the package will be updated?
IMU an .rpmsave file will be created each time rpm notices that a %config file has been modified by the user and is no more part of the package file list of the new package version.
You can mark the files in /etc as %ghost. That way they are still listed in the file list so rpm won't touch existing ones. The downside is that rpm also won't remove unmodified versions of such config files of a previous package. A missing upstream rpm feature IMO. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; HRB 36809 (AG Nürnberg)
On Friday 2023-08-11 18:40, Franck Bui wrote:
Basically the daemons parse first their "main" configuration file (e.g /etc/systemd/system.conf for PID1) and then read and apply the optional drop-in files located in the following directories:
/usr/lib/systemd/*.conf.d/ /usr/local/lib/systemd/*.conf.d/ /run/systemd/*.conf.d/ /etc/systemd/*.conf.d/
There were various discussions [1] about addressing this concern but at the end upstream preferred keeping the existing logic. [1] https://github.com/systemd/systemd/pull/17161
The reasoning there in [1] is sound; it's all getting too complicated and it is best to stick to just fragments (drop-ins). The use of "early config" files that you proposed makes it even harder. How about an alternate proposal: * Remember the "priority" where the main config file was found; /usr (low), /run (medium), /etc (high) * _Only_ read drop-ins from locations with same-or-higher priority. IOW, * if /usr/xx.conf was used, read fragments from /usr/xx.d,/run/xx.d,/etc/xx.d * if /run/xx.conf was used, read fragments from /run/xx.d,/etc/xx.d * if /etc/xx.conf was used, read fragments from /etc/xx.d (This is _also_ not backwards-compatible obviously, but at least it's better than reading /usr/xx.d at unexpected times.)
On 12.08.2023 12:14, Jan Engelhardt wrote:
On Friday 2023-08-11 18:40, Franck Bui wrote:
Basically the daemons parse first their "main" configuration file (e.g /etc/systemd/system.conf for PID1) and then read and apply the optional drop-in files located in the following directories:
/usr/lib/systemd/*.conf.d/ /usr/local/lib/systemd/*.conf.d/ /run/systemd/*.conf.d/ /etc/systemd/*.conf.d/
There were various discussions [1] about addressing this concern but at the end upstream preferred keeping the existing logic. [1] https://github.com/systemd/systemd/pull/17161
The reasoning there in [1] is sound; it's all getting too complicated and it is best to stick to just fragments (drop-ins).
The use of "early config" files that you proposed makes it even harder. How about an alternate proposal:
* Remember the "priority" where the main config file was found; /usr (low), /run (medium), /etc (high)
* _Only_ read drop-ins from locations with same-or-higher priority.
IOW,
* if /usr/xx.conf was used, read fragments from /usr/xx.d,/run/xx.d,/etc/xx.d * if /run/xx.conf was used, read fragments from /run/xx.d,/etc/xx.d * if /etc/xx.conf was used, read fragments from /etc/xx.d
Which makes it impossible for packages to install drop-ins in /usr if user edited file in /etc.
(This is _also_ not backwards-compatible obviously, but at least it's better than reading /usr/xx.d at unexpected times.)
On Saturday 2023-08-12 13:10, Andrei Borzenkov wrote:
IOW,
* if /usr/xx.conf was used, read fragments from /usr/xx.d,/run/xx.d,/etc/xx.d * if /run/xx.conf was used, read fragments from /run/xx.d,/etc/xx.d * if /etc/xx.conf was used, read fragments from /etc/xx.d
Which makes it impossible for packages to install drop-ins in /usr if user edited file in /etc.
If the user placed /etc/xx.conf (rather than /etc/xx.d/xxx), they didn't want anything from /usr anyway.
On Sat, 2023-08-12 at 15:55 +0200, Jan Engelhardt wrote:
On Saturday 2023-08-12 13:10, Andrei Borzenkov wrote:
IOW,
* if /usr/xx.conf was used, read fragments from /usr/xx.d,/run/xx.d,/etc/xx.d * if /run/xx.conf was used, read fragments from /run/xx.d,/etc/xx.d * if /etc/xx.conf was used, read fragments from /etc/xx.d
Which makes it impossible for packages to install drop-ins in /usr if user edited file in /etc.
If the user placed /etc/xx.conf (rather than /etc/xx.d/xxx), they didn't want anything from /usr anyway.
I don't follow. Even if a user made some customizations in xx.conf, she may want still to follow package maintainer's customizations in /usr/lib/xx.d/. Your proposal would mean that she'd have to copy all files with directives she wants applied from /usr/lib to /etc. Such files might be added by package updates, and she might not even be aware of them. I guess what could be done (with additional complexity) would be something like /etc/xx.d/00-directories.conf, where a user could say that she wants /usr/lib and /run (or whatever else) to be ignored. Martin
On Monday, 14 August 2023 7:44:03 PM ACST Martin Wilck via openSUSE Factory wrote:
On Sat, 2023-08-12 at 15:55 +0200, Jan Engelhardt wrote:
On Saturday 2023-08-12 13:10, Andrei Borzenkov wrote:
IOW,
* if /usr/xx.conf was used, read fragments from /usr/xx.d,/run/xx.d,/etc/xx.d * if /run/xx.conf was used, read fragments from /run/xx.d,/etc/xx.d * if /etc/xx.conf was used, read fragments from /etc/xx.d
Which makes it impossible for packages to install drop-ins in /usr if user edited file in /etc.
If the user placed /etc/xx.conf (rather than /etc/xx.d/xxx), they didn't want anything from /usr anyway.
I don't follow. Even if a user made some customizations in xx.conf, she may want still to follow package maintainer's customizations in /usr/lib/xx.d/. Your proposal would mean that she'd have to copy all files with directives she wants applied from /usr/lib to /etc. Such files might be added by package updates, and she might not even be aware of them.
I guess what could be done (with additional complexity) would be something like /etc/xx.d/00-directories.conf, where a user could say that she wants /usr/lib and /run (or whatever else) to be ignored.
Martin
Not so. If one uses systemctl edit <unit file>, it creates an override file in /etc/systemd/system/<unit>.d into which the admin can place directives that override those specific directives in the primary unit file in /lib/systemd/ system/. Only those directives specified in the override.conf file are affected - all other directives in the primary unit file stil apply. For example, from one of my systems, I have /lib/systemd/system/graylog- server.service, which has the default directives, but in /etc/systemd/system/ graylog-server.service.d/override.conf there are 2 directives: [Unit] After=opensearch.service [Service] AmbientCapabilities=CAP_NET_BIND_SERVICE These override the defaults in /lib/systemd/system/graylog-server.service. I was not a fan of systemd when it first appeared, but things like this actually make system admistration easier once you learn how to use them (and the logic behind them). -- ================================================================================================================== Rodney Baker rodney.baker@iinet.net.au ==================================================================================================================
On Monday, 14 August 2023 8:01:02 PM ACST Rodney Baker wrote:
On Monday, 14 August 2023 7:44:03 PM ACST Martin Wilck via openSUSE Factory [...]
I don't follow. Even if a user made some customizations in xx.conf, she may want still to follow package maintainer's customizations in /usr/lib/xx.d/. Your proposal would mean that she'd have to copy all files with directives she wants applied from /usr/lib to /etc. Such files might be added by package updates, and she might not even be aware of them.
I guess what could be done (with additional complexity) would be something like /etc/xx.d/00-directories.conf, where a user could say that she wants /usr/lib and /run (or whatever else) to be ignored.
Martin
Not so. If one uses systemctl edit <unit file>, it creates an override file in /etc/systemd/system/<unit>.d into which the admin can place directives that override those specific directives in the primary unit file in /lib/systemd/ system/. Only those directives specified in the override.conf file are affected - all other directives in the primary unit file stil apply.
For example, from one of my systems, I have /lib/systemd/system/graylog- server.service, which has the default directives, but in /etc/systemd/system/ graylog-server.service.d/override.conf there are 2 directives:
[Unit] After=opensearch.service
[Service] AmbientCapabilities=CAP_NET_BIND_SERVICE
These override the defaults in /lib/systemd/system/graylog-server.service.
I was not a fan of systemd when it first appeared, but things like this actually make system admistration easier once you learn how to use them (and the logic behind them).
It's also worth noting that systemctl status <unit> shows the unit file and drop-ins in use... root@vm-srv-graylog ~ # systemctl status graylog-server.service ● graylog-server.service - Graylog server Loaded: loaded (/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/graylog-server.service.d └─override.conf Active: active (running) since Sun 2023-08-06 15:58:08 ACST; 1 weeks 1 days ago [...output truncated...] -- ================================================================================================================== Rodney Baker rodney.baker@iinet.net.au ==================================================================================================================
On 8/14/23 09:55, Rodney Baker wrote:
It's also worth noting that systemctl status <unit> shows the unit file and drop-ins in use...
root@vm-srv-graylog ~ # systemctl status graylog-server.service ● graylog-server.service - Graylog server Loaded: loaded (/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/graylog-server.service.d └─override.conf Active: active (running) since Sun 2023-08-06 15:58:08 ACST; 1 weeks 1 days ago [...output truncated...]
Robert, Is there some config option that must be enabled to for systemctl status to show the Drop-In items ? I have a drop in for the journal configuration in /etc/systemd/journald.conf.d/ but it does not show up when I check the status... systemctl status systemd-journald.service ? systemd-journald.service - Journal Service Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static) Active: active (running) since Thu 2023-08-10 14:49:47 EDT; 3 days ago TriggeredBy: ? systemd-journald-audit.socket ? systemd-journald.socket ? systemd-journald-dev-log.socket Docs: man:systemd-journald.service(8) man:journald.conf(5) Main PID: 569 (systemd-journal) Status: "Processing requests..." Tasks: 1 (limit: 4915) CPU: 31.389s CGroup: /system.slice/systemd-journald.service +-569 /usr/lib/systemd/systemd-journald -- Regards, Joe
On Mon, Aug 14, Joe Salmeri wrote:
Robert, Is there some config option that must be enabled to for systemctl status to show the Drop-In items ?
I have a drop in for the journal configuration in /etc/systemd/journald.conf.d/ but it does not show up when I check the status...
That's for journald.conf, which is not a systemd unit.
systemctl status systemd-journald.service
That's for systemd units... No idea if journalctl can/will show you drop in files, too, but systemctl only works on systemd units, not something else. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On 8/14/23 11:18, Thorsten Kukuk wrote:
On Mon, Aug 14, Joe Salmeri wrote:
Robert, Is there some config option that must be enabled to for systemctl status to show the Drop-In items ?
I have a drop in for the journal configuration in /etc/systemd/journald.conf.d/ but it does not show up when I check the status... That's for journald.conf, which is not a systemd unit.
Thanks very much for the explanation. It is confusing that something that is not a systemd unit would have their configuration file in /etc/systemd.
systemctl status systemd-journald.service That's for systemd units...
No idea if journalctl can/will show you drop in files, too, but systemctl only works on systemd units, not something else.
Does not look like journalctl can do it. I know that my override is working because it is for the max retention of the journal and the older logs are purged according to my schedule. -- Regards, Joe
On Mon, Aug 14, Joe Salmeri wrote:
Thanks very much for the explanation. It is confusing that something that is not a systemd unit would have their configuration file in /etc/systemd.
/etc/systemd is for the full systemd eco system (which also includes e.g. systemd-network). /etc/systemd/system is for systemd units. It's not confusing at all. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Monday 2023-08-14 12:14, Martin Wilck via openSUSE Factory wrote:
Which makes it impossible for packages to install drop-ins in /usr if user edited file in /etc.
If the user placed /etc/xx.conf (rather than /etc/xx.d/xxx), they didn't want anything from /usr anyway.
I don't follow. Even if a user made some customizations in xx.conf, she may want still to follow package maintainer's customizations in /usr/lib/xx.d/.
No. If you wanted to follow package maintainer's customizations, you would use drop-ins (/etc/xx.d in my spelling), not a full config (xx.conf). This follows directly from when you eliminate drop-ins from the picture and consider a system that uses just "full" config files. Like we used to in SUSE 6.x or something.
On Mon, 2023-08-14 at 12:44 +0200, Jan Engelhardt wrote:
On Monday 2023-08-14 12:14, Martin Wilck via openSUSE Factory wrote:
Which makes it impossible for packages to install drop-ins in /usr if user edited file in /etc.
If the user placed /etc/xx.conf (rather than /etc/xx.d/xxx), they didn't want anything from /usr anyway.
I don't follow. Even if a user made some customizations in xx.conf, she may want still to follow package maintainer's customizations in /usr/lib/xx.d/.
No. If you wanted to follow package maintainer's customizations, you would use drop-ins (/etc/xx.d in my spelling), not a full config (xx.conf).
This assumes that every user understands and adopts the drop-in policy immediately. Also, it adds a non-obvious assumption about the user's intentions to the existence of the xx.conf file. More often than not, xx.conf will be a leftover from the past, created with no other intention than just modifying one specific setting. The current scheme with both conf files and drop-ins in different directories is complex, but it has a concise logic: files in one hierarchy override files _of the same name_ in another. Your scheme further complicates this by adding a rule "file X in one hierarchy overrides file Y in another". I can see that some admins may want to configure their systems in a way that no package-installed drop-in would ever override their own settings. But I think this must be achieved differently.
This follows directly from when you eliminate drop-ins from the picture and consider a system that uses just "full" config files. Like we used to in SUSE 6.x or something.
I don't remember 6.x, sorry. Martin
On Mon, Aug 14, 2023 at 03:47:42PM +0200, Martin Wilck via openSUSE Factory wrote:
On Mon, 2023-08-14 at 12:44 +0200, Jan Engelhardt wrote:
On Monday 2023-08-14 12:14, Martin Wilck via openSUSE Factory wrote:
Which makes it impossible for packages to install drop-ins in /usr if user edited file in /etc.
If the user placed /etc/xx.conf (rather than /etc/xx.d/xxx), they didn't want anything from /usr anyway.
I don't follow. Even if a user made some customizations in xx.conf, she may want still to follow package maintainer's customizations in /usr/lib/xx.d/.
No. If you wanted to follow package maintainer's customizations, you would use drop-ins (/etc/xx.d in my spelling), not a full config (xx.conf).
This assumes that every user understands and adopts the drop-in policy immediately. Also, it adds a non-obvious assumption about the user's intentions to the existence of the xx.conf file. More often than not, xx.conf will be a leftover from the past, created with no other intention than just modifying one specific setting.
The current scheme with both conf files and drop-ins in different directories is complex, but it has a concise logic: files in one hierarchy override files _of the same name_ in another. Your scheme further complicates this by adding a rule "file X in one hierarchy overrides file Y in another".
I can see that some admins may want to configure their systems in a way that no package-installed drop-in would ever override their own settings. But I think this must be achieved differently.
Whatever the design outcome, perhaps a systemd lint-like script to check the conf, drop-ins and whatever remnants of the past would be worthwhile? I always prefer following the breadcrumbs from verbose whingeing rather than tracking a trail from the corpses (#mondaymotivation). Daniel
On 8/12/23 11:14, Jan Engelhardt wrote:
How about an alternate proposal:
The point of this discussion is to stop diverging from upstream and other distros when it comes to parsing config files. So unless your proposal gets accepted by upstream, which is unlikely since it's not backwards compatible, I don't think we should try to explore new paths. Thanks.
On 8/11/23 18:40, Franck Bui wrote:
- Some drop-in priority ranges should be defined to make sure that drop-in created by users have always higher precedence over the ones shipped by packages. The priority ranges reserved for users and downstream will be defined and documented in the relevant wiki page.
participants (13)
-
Andrei Borzenkov -
Daniel Morris -
Franck Bui -
Jan Engelhardt -
Joe Salmeri -
Ludwig Nussel -
Manfred Hollstein -
Martin Wilck -
Michal Suchánek -
Olaf Hering -
Richard Brown -
Rodney Baker -
Thorsten Kukuk