Moving /etc/sysconfig files
Hi, Afaik package provided config files have to move to /usr/etc? There seem to be quite a lot packages making use of the ugly fillup macros. All seem to be config files setting environment variables for systemd service process via EnvironmentFile=-XY: ---------------------- grep 'EnvironmentFile=-/etc/sysconfig' /usr/lib/systemd/system -r |wc -l 54 ls -1 /etc/sysconfig/ |wc -l 51 ls -1 /etc/sysconfig/network/ |wc -l 3 ---------------------- Afaik the fillup scripts were mostly for merging user changes and add some extra safety compared to limited rpm %config backup features. But having the split between package provided and on-top user config in 2 separate files makes things much easier anyway. EnvironmentFile=-/usr/etc/env/${service} EnvironmentFile=-/etc/env/${service} Concrete example is irqbalance where mainline does it similar: -------------------------------------------- Add optional config file in /usr/lib immutable distributions have a need for distribution specific configs in a read only space, and so modify the service file to optionally read a config file from /usr/lib/irqbalance/default.env -EnvironmentFile=/path/to/irqbalance.env +Environmentfile=-/usr/lib/irqbalance/defaults.env +EnvironmentFile=-/path/to/irqbalance.env -------------------------------------------- But as I do not see a single package doing it this way, nearly all are still located in /etc/sysconfig I do not want to begin something which already is decided/thought to be done differently. Long story short: How are other distros doing this? Do we want to get rid of fillup scripts and /etc/sysconfig and how? Is the double Environmentfile (/usr/etc and /etc) a suitable solution, maybe collect them in separate dir? Thanks for any thoughts and feedback, Thomas
On Wed, Mar 26, 2025 at 10:39 PM Thomas Renninger <trenn@suse.de> wrote:
Don't introduce a new directory, we have already since > 25 years /etc/default/ and now /usr/etc/default/ Please also take a look at https://en.opensuse.org/openSUSE:Packaging_Requirements_for_Atomic_and_Image...
There are already several packages doing it that way, just with etc/default
Long story short: How are other distros doing this?
Defaults compiled in and overwrite in /etc/default or /etc/sysconfig (Debian vs. Fedora)
Do we want to get rid of fillup scripts and /etc/sysconfig and how?
Long term we need to get rid of fillup, it does not work with image based updates (image as of /usr, not full disk images)
Is the double Environmentfile (/usr/etc and /etc) a suitable solution, maybe collect them in separate dir?
Yes, and that's working fine. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Donnerstag, 27. März 2025 08:31:55 Mitteleuropäische Normalzeit Olaf Hering wrote:
As Torsten said: -------------------------
It's not that hot anyway, as built-in, read-only files could be moved to a common location later as well. What is bad about trying to find a common location/default which may be consistent not only for packages, but even across distros? No idea why there is {/usr,}/etc/default and what is the purpose of "default". This puzzles me since grub configs showed up there. Having {/usr,}/etc/sys_env or similar where all systemd Environment files sourced by services "should" be located would be nice? If it's {/usr,}/etc/default and mixed with some other (all "sourcable"?) configs as this is what people started to use, be it. Better than scattered all over /usr? Thomas
Thu, 27 Mar 2025 09:10:10 +0100 Thomas Renninger <trenn@suse.de>:
What is bad about trying to find a common location/default which may be consistent not only for packages, but even across distros?
That is all fine. But in the context of irqbalance, what content would be in the file? The manual lists a few variables, which are 100% for the admin only, hence the place is something like /etc/sysconfig/%name, or /etc/systemd/system/irqbalance.service.de/some.conf. And both paths need to be created by the admin, not by rpm. The name "default" is certainly weird because the place for built-in defaults is and was below /usr, so choosing "sysconfig" as a name was certainly better... Olaf
On Donnerstag, 27. März 2025 09:34:09 Mitteleuropäische Normalzeit Olaf Hering wrote:
---- ML off-topic ---- There is: https://bugzilla.suse.com/show_bug.cgi?id=1238559 [Bug 1238559] irqbalance: add "--banmod megaraid_sas" option I try to get this short: megaraid_sas driven PCI cards are known to segfault when their IRQs are migrated when already operating. In theory the kernel has to take care this cannot happen. But kernel_param/ driver blacklisting can only be done via smp_affinity_enable=0 which would cause ignoring platform Numa smp affinity hints and result in bad performance. There is no way to let the kernel initialize smp affinity hint provided irqs and disable "run-time" IRQ migration via /proc. At least this is how I understand the problem... Anway, outcome was which lead to a typical "ugly to maintain userspace config variable workaround" (TM) blacklisting/banning megaraid_sas driven IRQs from being touched by irqbalance (patch C&Ped): ---------------------- diff --git a/misc/irqbalance.env b/misc/irqbalance.env index 84cb843..3875dbe 100644 --- a/misc/irqbalance.env +++ b/misc/irqbalance.env @@ -41,4 +41,4 @@ # Append any args here to the irqbalance daemon as documented in the man # page. # -IRQBALANCE_ARGS="" +IRQBALANCE_ARGS="--banmod megaraid_sas" ---------------------- This is really bad in case other args may be added by user defined configs. Hmm, there should at least be a separate "BAN_MOD=" variable, but on the other hand side these env based configs/workarounds should more vanish than getting extended. In the end admins should be smart enough to get this right until at some point of time this vanishes again (when kernel makes sure the driver won't segfault by touching /proc/migrate_irq). I wonder whether someone has an "non-intrusive" idea how to solve this nicer... Thomas
Thu, 27 Mar 2025 10:36:17 +0100 Thomas Renninger <trenn@suse.de>:
I wonder whether someone has an "non-intrusive" idea how to solve this nicer...
There are kernel firmware packages, which will be installed when certain PCI devices exist. In this case create megaraid_sas specific rpm which contains a drop-in /usr/lib/systemd/system/irqbalance.service.d/bug1238559.conf [Service] Environment=IRQBALANCE_ARGS="--banmod megaraid_sas" Olaf
On Donnerstag, 27. März 2025 10:44:30 Mitteleuropäische Normalzeit Olaf Hering wrote:
Wow. This is a rather clever approach restricting the blacklisting to machines having such a card plugged in (at install time only?). Fiddling with and distributing an extra package looks somewhat overdosed, though (but could give one some insights of the new git obs approach ;) ). I found apply_pci_quirks() to disable balancing for specific PCI devices inside irqbalance, but this would be hard-coded without the possibility to whitelist it again. Above ugly blacklist workaround looks like the most reasonable for me. I asked on kernel scsi list about a possible kernel solution without getting an answer. Maybe I send a patch to get some attraction on this: pr_info("Megaraid card XY or newer detected, userspace irq affinity modifications could lead to system freezes") and be done with it. This should be handled by the affected kernel driver of the affected HW. Thanks for the insights, Thomas
On Thu, Mar 27, 2025 at 9:10 AM Thomas Renninger <trenn@suse.de> wrote:
What is bad about trying to find a common location/default which may be consistent not only for packages, but even across distros?
That the distros are not interested in this. When I started with UsrEtc for openSUSE, we tried to find such a place. For every location there was a veto from at least one distro. So we have to do it on package level...
No idea why there is {/usr,}/etc/default and what is the purpose of "default".
That's Debians /etc/sysconfig, only without the fillup overhead we have. And why there is no project adopting /etc/sysconfig, even if Fedora and openSUSE are using it, many projects adopted to /etc/default. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Thu, Mar 27, 2025 at 8:18 AM Thomas Renninger <trenn@suse.de> wrote:
If upstream has a common place in /usr used by everybody, you should not change it. /usr/etc should only be used for our configuration files where no "upstream" exists. There is no benefit to move/usr/lib/irqbalance/defaults.env to /usr/etc/default/irqbalance except that it will confuse users. Thorsten
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Donnerstag, 27. März 2025 09:53:35 Mitteleuropäische Normalzeit Olaf Hering wrote:
That's the "trick"...: dash is ugly to search the docs, it means: Ignore if the config file is missing and continue without error. So declaring both in systemd.service: EnvironmentFile=-/usr/etc/default/blah EnvironmentFile=-/etc/default/blah would first (try) load static defaults and then (possibly overwrite) user defined ones. Thomas
Thu, 27 Mar 2025 10:08:56 +0100 Thomas Renninger <trenn@suse.de>:
would first (try) load static defaults
This does not answer the question why an rpm provided file can be missing. In other words, do not use the dash for files that do exist (in case it makes sense to actually include any such file in the rpm, and reference it in the systemd service file). Olaf
On Mittwoch, 26. März 2025 22:55:42 Mitteleuropäische Normalzeit Thorsten Kukuk wrote: ...
If someone wants to give this a review, it would be appreciated: https://build.opensuse.org/requests/1265215/changes Relevant double EnvironmentFile= and get rid of fillup part (C&P) If the sed does not match anymore due to mainline changes, it would silently be ignored. Not sure how to solve that in an elegant way, maybe just a grep before each sed...: --------------------------------- +- Provide 2 systemd Environment files, a package provided one and a + possibly existing user provided one overwriting default env vars + provided by the package if found. + EnvironmentFile=/usr/lib/irqbalance/defaults.env + EnvironmentFile=-/etc/irqbalance/defaults.env +- Remove fillup dependency due to above changes -#Compat macro for new _fillupdir macro introduced in Nov 2017 -%if ! %{defined _fillupdir} - %define _fillupdir %{_localstatedir}/adm/fillup-templates -%endif ... -Requires(pre): %fillup_prereq ... # +mkdir -p %{buildroot}%{_prefix}/lib/irqbalance +mkdir -p %{buildroot}%{_sysconfdir}/irqbalance/ +sed -ie "s|EnvironmentFile=-%{_prefix}/lib/irqbalance/defaults.env|EnvironmentFile=%{_prefix}/lib/irqbalance/defaults.env|g" misc/irqbalance.service +sed -ie "s|EnvironmentFile=-/path/to/irqbalance.env|EnvironmentFile=-%{_sysconfdir}/irqbalance/defaults.env|g" misc/irqbalance.service install -D -m 0644 misc/irqbalance.service %{buildroot}%{_unitdir}/irqbalance.service +install -m 0644 misc/irqbalance.env %{buildroot}%{_prefix}/lib/irqbalance/defaults.env + -%fillup_only %{name} ... -%{_fillupdir}/sysconfig.irqbalance +%dir %{_prefix}/lib/irqbalance +%{_prefix}/lib/irqbalance/defaults.env
On Wednesday 2025-03-26 22:39, Thomas Renninger wrote:
Afaik package provided config files have to move to /usr/etc?
/etc/sysconfig is a bit special. sysconfig files get new VARIABLES= added if missing, but otherwise not modified or deleted on subsequent upgrades, which makes these files (quasi-)customized-configuration that must not be in /usr/etc.
Long story short: How are other distros doing this?
/etc/sysconfig and fillup-templates got introduced in SUSE 8.0. SUSE RPMs (today) do not own /etc/sysconfig/xyz which is in line with the idea that /etc contains host-specific configuration. AlmaLinux and Mandriva RPMs outright own /etc/sysconfig/xyz as %config(noreplace), with all the pain points that _not using_ the fillup mechanism brings (.rpmsave/.rpmnew files). AFAICT Debian never jumped on the sysconfig bandwagon, and the Debian 12 and Debian Sid chroots/nspawns I have around do not have /etc/sysconfig. If there are changes to /etc files, there is a semi-manual process offered <https://serverfault.com/questions/527789/>. Almost as bad as .rpmsave/.rpmnew.
Do we want to get rid of fillup scripts and /etc/sysconfig and how?
Simply put, by making all programs cope with the absence of /etc and, as far as possible, having sane defaults baked into the program executable.
On Mittwoch, 26. März 2025 23:46:41 Mitteleuropäische Normalzeit Jan Engelhardt wrote:
Yep. But if there is the package provided, read-only /usr/etc config, you have the fillup functionality already: A kind of 3-way diff of the unmodified base config (from whatever installed version), user modified /etc/sysconfig and upgrade config changes (new, deleted or evily modified) variables one has to merge on a version upgrade.
SUSE RPMs (today) do not own /etc/sysconfig/xyz which is in line with the idea that /etc contains host-specific configuration.
Yep. Maintaining bad defined config changes is a nightmare, despite the tool one uses. Also these kind of config file types seem to die and the last use-case seem to be feeding systemd service executables with user configurable environment variables. Thanks for the details, Thomas
On Wed, Mar 26, 2025 at 10:39 PM Thomas Renninger <trenn@suse.de> wrote:
Don't introduce a new directory, we have already since > 25 years /etc/default/ and now /usr/etc/default/ Please also take a look at https://en.opensuse.org/openSUSE:Packaging_Requirements_for_Atomic_and_Image...
There are already several packages doing it that way, just with etc/default
Long story short: How are other distros doing this?
Defaults compiled in and overwrite in /etc/default or /etc/sysconfig (Debian vs. Fedora)
Do we want to get rid of fillup scripts and /etc/sysconfig and how?
Long term we need to get rid of fillup, it does not work with image based updates (image as of /usr, not full disk images)
Is the double Environmentfile (/usr/etc and /etc) a suitable solution, maybe collect them in separate dir?
Yes, and that's working fine. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Donnerstag, 27. März 2025 08:31:55 Mitteleuropäische Normalzeit Olaf Hering wrote:
As Torsten said: -------------------------
It's not that hot anyway, as built-in, read-only files could be moved to a common location later as well. What is bad about trying to find a common location/default which may be consistent not only for packages, but even across distros? No idea why there is {/usr,}/etc/default and what is the purpose of "default". This puzzles me since grub configs showed up there. Having {/usr,}/etc/sys_env or similar where all systemd Environment files sourced by services "should" be located would be nice? If it's {/usr,}/etc/default and mixed with some other (all "sourcable"?) configs as this is what people started to use, be it. Better than scattered all over /usr? Thomas
Thu, 27 Mar 2025 09:10:10 +0100 Thomas Renninger <trenn@suse.de>:
What is bad about trying to find a common location/default which may be consistent not only for packages, but even across distros?
That is all fine. But in the context of irqbalance, what content would be in the file? The manual lists a few variables, which are 100% for the admin only, hence the place is something like /etc/sysconfig/%name, or /etc/systemd/system/irqbalance.service.de/some.conf. And both paths need to be created by the admin, not by rpm. The name "default" is certainly weird because the place for built-in defaults is and was below /usr, so choosing "sysconfig" as a name was certainly better... Olaf
On Donnerstag, 27. März 2025 09:34:09 Mitteleuropäische Normalzeit Olaf Hering wrote:
---- ML off-topic ---- There is: https://bugzilla.suse.com/show_bug.cgi?id=1238559 [Bug 1238559] irqbalance: add "--banmod megaraid_sas" option I try to get this short: megaraid_sas driven PCI cards are known to segfault when their IRQs are migrated when already operating. In theory the kernel has to take care this cannot happen. But kernel_param/ driver blacklisting can only be done via smp_affinity_enable=0 which would cause ignoring platform Numa smp affinity hints and result in bad performance. There is no way to let the kernel initialize smp affinity hint provided irqs and disable "run-time" IRQ migration via /proc. At least this is how I understand the problem... Anway, outcome was which lead to a typical "ugly to maintain userspace config variable workaround" (TM) blacklisting/banning megaraid_sas driven IRQs from being touched by irqbalance (patch C&Ped): ---------------------- diff --git a/misc/irqbalance.env b/misc/irqbalance.env index 84cb843..3875dbe 100644 --- a/misc/irqbalance.env +++ b/misc/irqbalance.env @@ -41,4 +41,4 @@ # Append any args here to the irqbalance daemon as documented in the man # page. # -IRQBALANCE_ARGS="" +IRQBALANCE_ARGS="--banmod megaraid_sas" ---------------------- This is really bad in case other args may be added by user defined configs. Hmm, there should at least be a separate "BAN_MOD=" variable, but on the other hand side these env based configs/workarounds should more vanish than getting extended. In the end admins should be smart enough to get this right until at some point of time this vanishes again (when kernel makes sure the driver won't segfault by touching /proc/migrate_irq). I wonder whether someone has an "non-intrusive" idea how to solve this nicer... Thomas
participants (5)
-
Jan Engelhardt -
Olaf Hering -
Takashi Iwai -
Thomas Renninger -
Thorsten Kukuk