[zypp-devel] Augeas for libzypp and zypper?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 (orignal text at http://jniq.blogspot.com/2009/03/augeas-for-libzypp-and-zypper.html reposting for discussion) - ------copy------ The need for a configuration file for zypper (plus my wish for a nice CLI for handling it) and a visit to Raphaël Pinson's presentation at FOSDEM made me look at Augeas closely. Augeas enables you to read and edit your config files in a way which preserves comments and formatting details, allowing humans and programs coexist peacefully when it comes to editing configuration files. More than that, you can map some special structures of the file to special parts of the augeas tree, rather than using the generic INI file parser. See the zypp.conf for example (zypper.conf will have the same structure): [main] ## Option description. ## Default: etc. ## # disabledoption = value ## Another option description ## another.option = value With Augeas you can write a lens (the file contents description) which will map such file into a tree like this: /files/etc/zypp/zypp.conf/main/1 /files/etc/zypp/zypp.conf/main/1/description[1] = "Option description." /files/etc/zypp/zypp.conf/main/1/description[2] = "Default: etc." /files/etc/zypp/zypp.conf/main/1/description[3] /files/etc/zypp/zypp.conf/main/1/commented /files/etc/zypp/zypp.conf/main/1/disabledoption = "value" /files/etc/zypp/zypp.conf/main/2 /files/etc/zypp/zypp.conf/main/2/description[1] = "Another option description." /files/etc/zypp/zypp.conf/main/2/description[2] /files/etc/zypp/zypp.conf/main/2/another.option = "value" Note the commented node. Augeas has an API for looking for the nodes, getting values from them, setting the values, and finally saving the tree back to the config file. What does this mean for zypper? Implementing zypper conf --list is a piece of cake, as well as zypper conf --set another.option value. Commenting or uncommenting the option is a matter of adding or removing the commented node. Getting the option description for zypper conf - --help some.option is easy (and maybe it is possible to map the multi-line ## description to a single 'description' node!). Here is a draft of lens for zypper (it will need comments and further improvements). To try it out you can crete a /etc/zypp/zypper.conf file with contents like the above example, and use the augtool like this: $ augtool -I /the/dir/with/the/zypper.aug/file augtool> print /files/etc/zypp/zypper.conf/ /files/etc/zypp/zypper.conf /files/etc/zypp/zypper.conf/anon /files/etc/zypp/zypper.conf/anon/description[1] = "Configuration file for zypper" /files/etc/zypp/zypper.conf/anon/description[2] /files/etc/zypp/zypper.conf/anon/description[3] = "Boolean values are 0 1 yes no on off true false" /files/etc/zypp/zypper.conf/main /files/etc/zypp/zypper.conf/main/1 /files/etc/zypp/zypper.conf/main/1/description[1] = "Whether to use colors" /files/etc/zypp/zypper.conf/main/1/description[2] /files/etc/zypp/zypper.conf/main/1/description[3] = "Default value: autodetected" /files/etc/zypp/zypper.conf/main/1/description[4] /files/etc/zypp/zypper.conf/main/1/description[5] /files/etc/zypp/zypper.conf/main/1/colors = "yes" augtool> Type help for other commands, use TAB key for completion, and check also the Augeas home page for more info. What does this mean for libzypp? Basically the above, with respect to libzypp's files. Currently, you either edit your repos.d/*.repo, locks, credentials, and other files by hand or you leave it to libzypp's frontends, or you at least take care not to write comments in those files, because they will all get removed upon the next change done by libzypp (what we do there is to rewrite the entire files from in-memory data upon saving). This can be easily avoided using Augeas and we can add an editing API for zypp.conf. Plus the bonuses described above. The library package (libaugeas0) would add 300 kiB of installed size to libzypp's/zypper's dependencies, (which is not little), plus an optional 200 kiB of the official lenses (augeas-lenses), if we decide to use them. todo: Look at Augeas' error reporting. The user can of course break the structure of the file in a way that the lens can't map it to the tree anymore. If parsing of the file will fail, we need to report why, so that the user can fix it. - ------copy------ I also have some code using the draft lens for zypper ready, so unless somebody is very much against zypper depending on libaugeas0, i will soon push it. BTW, can i place system-wide zypper.conf to /etc/zypp (the user's will be ~/.zypper.conf). Also, i plan to put the zypper.aug to /usr/share/zypper, so that zypper can tell augeas to read only this .aug file and parse only the zypper.conf files so that it is quick. Whether it's worth to use augeas in libzypp, i propose for discussion. IMO, the effort to enable this is not big, but it is also not a need-to-have, but rather a nice-to-have, and we surely have more important things to do. Just wanted to let you know this is available for grabbing anytime :O) Note that just writing lenses for zypp without touching zypp itself can be beneficial, too. - -- cheers, jano Ján Kupec YaST team - ---------------------------------------------------------(PGP)--- Key ID: 637EE901 Fingerprint: 93B9 C79B 2D20 51C3 800B E09B 8048 46A6 637E E901 - ---------------------------------------------------------(IRC)--- Server: irc.freenode.net Nick: jniq Channels: #zypp #yast #suse #susecz - ---------------------------------------------------------(EOF)--- -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iEYEARECAAYFAkmu+u8ACgkQgEhGpmN+6QG4wACeO6t2McWLv8MAjK5A410LFVoi 4zQAnAsaJ/2jvmtiwRaA5DfQ2Xu51suL =oMpW -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Streda 04 March 2009 23:04:31 Jan Kupec wrote:
(orignal text at http://jniq.blogspot.com/2009/03/augeas-for-libzypp-and-zypper.html reposting for discussion)
------copy------ The need for a configuration file for zypper (plus my wish for a nice CLI for handling it) and a visit to Raphaël Pinson's presentation at FOSDEM made me look at Augeas closely.
Augeas enables you to read and edit your config files in a way which preserves comments and formatting details, allowing humans and programs coexist peacefully when it comes to editing configuration files. More than that, you can map some special structures of the file to special parts of the augeas tree, rather than using the generic INI file parser. See the zypp.conf for example (zypper.conf will have the same structure):
[main] ## Option description. ## Default: etc. ## # disabledoption = value
## Another option description ##
another.option = value
With Augeas you can write a lens (the file contents description) which will map such file into a tree like this:
/files/etc/zypp/zypp.conf/main/1 /files/etc/zypp/zypp.conf/main/1/description[1] = "Option description." /files/etc/zypp/zypp.conf/main/1/description[2] = "Default: etc." /files/etc/zypp/zypp.conf/main/1/description[3] /files/etc/zypp/zypp.conf/main/1/commented /files/etc/zypp/zypp.conf/main/1/disabledoption = "value" /files/etc/zypp/zypp.conf/main/2 /files/etc/zypp/zypp.conf/main/2/description[1] = "Another option description." /files/etc/zypp/zypp.conf/main/2/description[2] /files/etc/zypp/zypp.conf/main/2/another.option = "value"
Note the commented node. Augeas has an API for looking for the nodes, getting values from them, setting the values, and finally saving the tree back to the config file.
What does this mean for zypper? Implementing zypper conf --list is a piece of cake, as well as zypper conf --set another.option value. Commenting or uncommenting the option is a matter of adding or removing the commented node. Getting the option description for zypper conf --help some.option is easy (and maybe it is possible to map the multi-line ## description to a single 'description' node!).
Here is a draft of lens for zypper (it will need comments and further improvements). To try it out you can crete a /etc/zypp/zypper.conf file with contents like the above example, and use the augtool like this:
$ augtool -I /the/dir/with/the/zypper.aug/file augtool> print /files/etc/zypp/zypper.conf/ /files/etc/zypp/zypper.conf /files/etc/zypp/zypper.conf/anon /files/etc/zypp/zypper.conf/anon/description[1] = "Configuration file for zypper" /files/etc/zypp/zypper.conf/anon/description[2] /files/etc/zypp/zypper.conf/anon/description[3] = "Boolean values are 0 1 yes no on off true false" /files/etc/zypp/zypper.conf/main /files/etc/zypp/zypper.conf/main/1 /files/etc/zypp/zypper.conf/main/1/description[1] = "Whether to use colors" /files/etc/zypp/zypper.conf/main/1/description[2] /files/etc/zypp/zypper.conf/main/1/description[3] = "Default value: autodetected" /files/etc/zypp/zypper.conf/main/1/description[4] /files/etc/zypp/zypper.conf/main/1/description[5] /files/etc/zypp/zypper.conf/main/1/colors = "yes" augtool>
Type help for other commands, use TAB key for completion, and check also the Augeas home page for more info.
What does this mean for libzypp? Basically the above, with respect to libzypp's files. Currently, you either edit your repos.d/*.repo, locks, credentials, and other files by hand or you leave it to libzypp's frontends, or you at least take care not to write comments in those files, because they will all get removed upon the next change done by libzypp (what we do there is to rewrite the entire files from in-memory data upon saving). This can be easily avoided using Augeas and we can add an editing API for zypp.conf. Plus the bonuses described above.
The library package (libaugeas0) would add 300 kiB of installed size to libzypp's/zypper's dependencies, (which is not little), plus an optional 200 kiB of the official lenses (augeas-lenses), if we decide to use them.
todo: Look at Augeas' error reporting. The user can of course break the structure of the file in a way that the lens can't map it to the tree anymore. If parsing of the file will fail, we need to report why, so that the user can fix it. ------copy------
I also have some code using the draft lens for zypper ready, so unless somebody is very much against zypper depending on libaugeas0, i will soon push it. BTW, can i place system-wide zypper.conf to /etc/zypp (the user's will be ~/.zypper.conf). Also, i plan to put the zypper.aug to /usr/share/zypper, so that zypper can tell augeas to read only this .aug file and parse only the zypper.conf files so that it is quick.
I'm still not sure if 300kiB of a library for reading a INI file is worth it. Stano -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jan Kupec wrote:
Note that just writing lenses for zypp without touching zypp itself can be beneficial, too.
I think we should link libzypp with augeas for various reasons. - - I would like to give some love to ZConfig. Its design won't scale anymore, by adding a setter/getter to every new option, but I would like something more similar to KDE's KConfig ( config.writeBoolean("myopt", true ) ) ... with the type safety of SolvAttr ( enum -> string mapping ) plus some way of specifying declaring available options and their type, which at least could be checked at runtime... Michael, do you have any ideas on how this can be done? This class could then use the thin Augeas wrapper Jano has in his experimental zypper branch, handle stuff like caching the values.. and be the base for the specific ZYpp and zypper config classes that would declare their own options. zypper already will have a nice feature be using augeas. Augeas allow to read comments, and Jano's wrapper will allow for zypper help to show the comments of an option during help, allowing us to maintain the options in the example config itself. It could even show comments for a repository if libzypp reads those with augeas too. Duncan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iEYEARECAAYFAkm2bSUACgkQzR62qWZ+QtE8HQCgpq2yUtW+yKibzORas+0iDqh3 RUcAn1B06xxrvDYFJ/Uxn6qqXFxCkkDf =pHTr -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Tue, Mar 10, 2009 at 02:37:41PM +0100, Duncan Mac-Vicar P. wrote:
I think we should link libzypp with augeas for various reasons.
My understanding is that libaugeas was written to read/modify/write a plethora of file formats, isn't it a little overkill to use it just to parse an ini file? Preserving comments can't be the only reason to use augeas. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael Schroeder wrote:
My understanding is that libaugeas was written to read/modify/write a plethora of file formats, isn't it a little overkill to use it just to parse an ini file? Preserving comments can't be the only reason to use augeas.
It is the opposite, our config API does not support modifying the configuration because our API does not. If we added writing support (preserving comments) we would end implementing libaugeas :-) libagueas and the formats it supports are different things. Jano only uses libagueas and ignores the included lenses, and uses only one lens included in zypper itself. Duncan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iEYEARECAAYFAkm2cXYACgkQzR62qWZ+QtE5AgCdGzQVjxJ5J1+HCe1haNDIpiUg BgEAoKLMSyN+UJcgsaLML19pbokJ3giJ =Qmcl -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Michael Schroeder wrote:
On Tue, Mar 10, 2009 at 02:37:41PM +0100, Duncan Mac-Vicar P. wrote:
I think we should link libzypp with augeas for various reasons.
My understanding is that libaugeas was written to read/modify/write a plethora of file formats, isn't it a little overkill to use it just to parse an ini file? Preserving comments can't be the only reason to use augeas.
Parsing can be done without augeas, of course. But we can add editing capability to anything that libzypp writes with augeas: zypp.conf, .repo files, locks, credentials, etc. Currently we just rewrite the files from in-memory data, if we need to write them. Plus you can take advantage of augeas & the file's structure to pick up some special things, like i described in the original mail. With augeas, we can add setter methods to ZConfig, for example, pretty easily. Also for other files we use. And yes, you can use just your lenses (to avoid parsing all .aug files) and in the next release it will also be possible to parse only the config file you need (instead of parsing all files that augeas finds via its autoload feature). -- cheers, jano Ján Kupec YaST team ---------------------------------------------------------(PGP)--- Key ID: 637EE901 Fingerprint: 93B9 C79B 2D20 51C3 800B E09B 8048 46A6 637E E901 ---------------------------------------------------------(IRC)--- Server: irc.freenode.net Nick: jniq Channels: #zypp #yast #suse #susecz ---------------------------------------------------------(EOF)--- -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (4)
-
Duncan Mac-Vicar P. -
Jan Kupec -
Michael Schroeder -
Stanislav Visnovsky