[opensuse-factory] Default config in /usr/etc vs. YaST
I wonder was it coordinated with YaST? The first fallout is YaST Users and Security modules which now default to DES password encryption due to missing /etc/login.defs. I am not even sure what the right semantic should be, given that users now can create multiple files each containing single parameter so anything done via YaST is not guaranteed to actually have any effect. https://bugzilla.opensuse.org/show_bug.cgi?id=1155735 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Monday 2019-11-04 08:27, Andrei Borzenkov wrote:
I wonder was it coordinated with YaST? The first fallout is YaST Users and Security modules which now default to DES password encryption due to missing /etc/login.defs.
I am not even sure what the right semantic should be, given that users now can create multiple files each containing single parameter so anything done via YaST is not guaranteed to actually have any effect.
Do it like /etc/fonts/conf.d, and just use a filename like "/etc/login.confs.d/10-yast.conf"? Any user who manually goes into that directory and adds overrides before or after 10-yast "knows what he is doing". -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mon, Nov 04, Andrei Borzenkov wrote:
I wonder was it coordinated with YaST?
Yes, it was, but as nobody really knows which files YaST is reading we clearly missed some files. YaST should use libeconf to read login.defs, like shadow, util-linux and pam do already. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mon, 2019-11-04 at 09:39 +0100, Thorsten Kukuk wrote:
On Mon, Nov 04, Andrei Borzenkov wrote:
Hi all,
I wonder was it coordinated with YaST?
Yes, it was, but as nobody really knows which files YaST is reading we clearly missed some files.
Yes, YaST reads or writes quite a lot of files[1], especially under /etc/sysconfig.
YaST should use libeconf to read login.defs, like shadow, util-linux and pam do already.
The problem I see with libeconf is that, as far as I understood, it only supports key=value configuration files (as it is stated in the project's README[2]). That's fine for many configuration files (like sysconfig, login.defs, etc.), but I am not sure whether it will be good enough for other cases (ini files -having sections into account-, YAML- like (Salt), JSON (docker), etc.). Why not using augeas (or any other similar library) to parse the files instead of rolling our own? libaugeas and its lenses are quite small. The YaST team has a Ruby gem[3] that offers a high level API to configuration files using augeas to parse them. As a proof of concept, we already implemented support to behave more or less like libeconf. But I would be great if we could join forces and reduce code duplication. Of course, we could rely on libeconf when possible and use augeas when needed... Regards, Imo [1] https://etherpad.suse.de/p/yast-usr-etc [2] https://github.com/openSUSE/libeconf/blob/master/README.md [3] https://github.com/config-files-api/config_files_api [4] https://github.com/config-files-api/config_files_api/pull/32
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg) -- Imobach González Sosa YaST Team at SUSE LINUX GmbH https://imobachgs.github.io/
On Mon, Nov 04, Imobach González Sosa wrote:
On Mon, 2019-11-04 at 09:39 +0100, Thorsten Kukuk wrote:
On Mon, Nov 04, Andrei Borzenkov wrote:
Hi all,
I wonder was it coordinated with YaST?
Yes, it was, but as nobody really knows which files YaST is reading we clearly missed some files.
Yes, YaST reads or writes quite a lot of files[1], especially under /etc/sysconfig.
YaST should use libeconf to read login.defs, like shadow, util-linux and pam do already.
The problem I see with libeconf is that, as far as I understood, it only supports key=value configuration files (as it is stated in the project's README[2]). That's fine for many configuration files (like sysconfig, login.defs, etc.), but I am not sure whether it will be good enough for other cases (ini files -having sections into account-, YAML- like (Salt), JSON (docker), etc.).
The whole stuff makes only sense with ini style files.
Why not using augeas (or any other similar library) to parse the files instead of rolling our own? libaugeas and its lenses are quite small.
The parser is far the smallest part, and we haven't found anything which is merging the configuration files in the way we need it (the systemd developer were not willing to provide their code as standalone library or make the interface public). Thorsten
The YaST team has a Ruby gem[3] that offers a high level API to configuration files using augeas to parse them. As a proof of concept, we already implemented support to behave more or less like libeconf. But I would be great if we could join forces and reduce code duplication. Of course, we could rely on libeconf when possible and use augeas when needed...
Regards, Imo
[1] https://etherpad.suse.de/p/yast-usr-etc [2] https://github.com/openSUSE/libeconf/blob/master/README.md [3] https://github.com/config-files-api/config_files_api [4] https://github.com/config-files-api/config_files_api/pull/32
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg) -- Imobach González Sosa YaST Team at SUSE LINUX GmbH https://imobachgs.github.io/
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mon, 2019-11-04 at 13:19 +0100, Thorsten Kukuk wrote:
The problem I see with libeconf is that, as far as I understood, it only supports key=value configuration files (as it is stated in the project's README[2]). That's fine for many configuration files (like sysconfig, login.defs, etc.), but I am not sure whether it will be good enough for other cases (ini files -having sections into account-, YAML- like (Salt), JSON (docker), etc.).
The whole stuff makes only sense with ini style files.
OK, I see. I was simply thinking about a wider solution.
Why not using augeas (or any other similar library) to parse the files instead of rolling our own? libaugeas and its lenses are quite small.
The parser is far the smallest part, and we haven't found anything which is merging the configuration files in the way we need it (the systemd developer were not willing to provide their code as standalone library or make the interface public).
Well, what we do in YaST is to use the parser to read (and write) the information from those files and merge the content ourselves. We use a tree-like structure that can be replaced with any other thing if needed or modified in a per-application fashion. In this case, we might rely on libeconf when it is good enough and reach for augeas when needed. Thanks. -- Imobach González Sosa YaST Team at SUSE LINUX GmbH https://imobachgs.github.io/
V Mon, 4 Nov 2019 13:19:29 +0100 Thorsten Kukuk <kukuk@suse.de> napsáno:
On Mon, Nov 04, Imobach González Sosa wrote:
On Mon, 2019-11-04 at 09:39 +0100, Thorsten Kukuk wrote:
On Mon, Nov 04, Andrei Borzenkov wrote:
Hi all,
I wonder was it coordinated with YaST?
Yes, it was, but as nobody really knows which files YaST is reading we clearly missed some files.
Yes, YaST reads or writes quite a lot of files[1], especially under /etc/sysconfig.
YaST should use libeconf to read login.defs, like shadow, util-linux and pam do already.
The problem I see with libeconf is that, as far as I understood, it only supports key=value configuration files (as it is stated in the project's README[2]). That's fine for many configuration files (like sysconfig, login.defs, etc.), but I am not sure whether it will be good enough for other cases (ini files -having sections into account-, YAML- like (Salt), JSON (docker), etc.).
The whole stuff makes only sense with ini style files.
Hi thorsten, I am a bit lost here. Do you mean with this statement that libeconf makes sense only with ini files? Or that whole /usr/etc is intended for ini files? Or that merging makes sense only for ini files?
Why not using augeas (or any other similar library) to parse the files instead of rolling our own? libaugeas and its lenses are quite small.
The parser is far the smallest part, and we haven't found anything which is merging the configuration files in the way we need it (the systemd developer were not willing to provide their code as standalone library or make the interface public).
As Imo write, our approach with augeas is that we get configuration tree and do operation on top of that tree like merging and we get final result and it also allows various strategy for merging as we learned, that sometimes logic is quite complicated and merging is not just about /etc and /usr/etc, but in fact also various conf.d directory that do config snippets that can overwrite. We pick augeas due to number of existing parsers and also ability to edit with minimal changes. That allows us to share with upstream various fixes and faster react to changes in configuration. I just worry that your parser is small only as long as you not handle all possible corner cases of configuration formats. Know problems even with simple ini files we face so far are windows line ending, strange characters, escaping and various format extensions. Josef
Thorsten
The YaST team has a Ruby gem[3] that offers a high level API to configuration files using augeas to parse them. As a proof of concept, we already implemented support to behave more or less like libeconf. But I would be great if we could join forces and reduce code duplication. Of course, we could rely on libeconf when possible and use augeas when needed...
Regards, Imo
[1] https://etherpad.suse.de/p/yast-usr-etc [2] https://github.com/openSUSE/libeconf/blob/master/README.md [3] https://github.com/config-files-api/config_files_api [4] https://github.com/config-files-api/config_files_api/pull/32
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg) -- Imobach González Sosa YaST Team at SUSE LINUX GmbH https://imobachgs.github.io/
-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Nov 05, Josef Reidinger wrote:
V Mon, 4 Nov 2019 13:19:29 +0100 Thorsten Kukuk <kukuk@suse.de> napsáno:
The whole stuff makes only sense with ini style files.
Hi thorsten, I am a bit lost here. Do you mean with this statement that libeconf makes sense only with ini files? Or that whole /usr/etc is intended for ini files? Or that merging makes sense only for ini files?
The merging of several snippets to one file makes only sense for some formats, most likely ini files. It's pretty hard for some other files like yaml or json. So while the /usr/etc move makes sense for all configuration files, the best solution for every single file really depends on the format of the configuration file, there is no "this library will solve all use cases". Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
V Tue, 5 Nov 2019 19:40:08 +0100 Thorsten Kukuk <kukuk@suse.de> napsáno:
On Tue, Nov 05, Josef Reidinger wrote:
V Mon, 4 Nov 2019 13:19:29 +0100 Thorsten Kukuk <kukuk@suse.de> napsáno:
The whole stuff makes only sense with ini style files.
Hi thorsten, I am a bit lost here. Do you mean with this statement that libeconf makes sense only with ini files? Or that whole /usr/etc is intended for ini files? Or that merging makes sense only for ini files?
The merging of several snippets to one file makes only sense for some formats, most likely ini files. It's pretty hard for some other files like yaml or json.
What is exactly difference in data format that prevent merging? For it is not much difference if file1: A => 1, B => 2 file2: A => 3 file3: C => 4 result: A => 3, B => 2, C => 4 depends on format. So I still do not see how merging and configuration format is related together.
So while the /usr/etc move makes sense for all configuration files, the best solution for every single file really depends on the format of the configuration file, there is no "this library will solve all use cases".
That is why I think that best approach is to provide reasonable default and allow to redefine strategy for merging when it makes sense for user. In our POC we basically do simple replace as I did above when merging multiple files, but allow user to redefine it if there is specific behavior needed ( like concating values for keys, strange precedence, etc ). This way it is easy to use for us and also does not block us when we need more complicated strategy. Josef
Thorsten
-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 06/11/2019 14.07, Josef Reidinger wrote:
What is exactly difference in data format that prevent merging? For it is not much difference if
file1: A => 1, B => 2 file2: A => 3 file3: C => 4
result: A => 3, B => 2, C => 4
depends on format. So I still do not see how merging and configuration format is related together.
yaml and json support nested structures and there can be A: B: 1 and A: C: 2 and merging them could either give A: C: 2 or A: B: 1 C: 2 and if the merge produce the latter, it is not clear how to unset values - most likely you have to explicitly include them again and set them again. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
06.11.2019 16:43, Bernhard M. Wiedemann пишет:
On 06/11/2019 14.07, Josef Reidinger wrote:
What is exactly difference in data format that prevent merging? For it is not much difference if
file1: A => 1, B => 2 file2: A => 3 file3: C => 4
result: A => 3, B => 2, C => 4
depends on format. So I still do not see how merging and configuration format is related together.
yaml and json support nested structures and there can be
A: B: 1
and A: C: 2
How is it different from [A] B=1 [A] C=2 in .ini file?
and merging them could either give A: C: 2
Can you explain how merging can give that result?
or
A: B: 1 C: 2
and if the merge produce the latter, it is not clear how to unset values
Neither is it possible to unset keys with .ini files. You can only set them to empty value and treat empty value as unset value (that is what systemd does) but then you can do it with any serialization format.
- most likely you have to explicitly include them again and set them again.
-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (6)
-
Andrei Borzenkov -
Bernhard M. Wiedemann -
Imobach González Sosa -
Jan Engelhardt -
Josef Reidinger -
Thorsten Kukuk