Feature changed by: Michal Hrusecky (-miska-) Feature #306411, revision 20 Title: Configuration files merge openSUSE-11.2: Rejected by Michael Löffler (michl19) reject date: 2009-08-11 15:58:40 reject reason: too late for 11.2 Priority Requester: Desirable openSUSE-11.3: Evaluation Priority Requester: Desirable Requested by: Michal Hrusecky (-miska-) Description: If we want to make our distribution friendly even to advanced users, I think that we shouldn't touch users configuration files. Luckily this isn't problem as in spec file Ican state that this file is configuration and I don't want to replace it if user did some manual changes. Or I can say that I want to replace it and keep backup of original one. But what I'm missing is some tool to easily merge these files. In Gentoo there is etc-update (http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=3&chap=4#doc_chap2) or dispatch-conf (http://en.gentoo-wiki.com/wiki/Dispatch-conf) . I'll describe more precisely what feature I'm missing and why I think it's needed. Let's say that I've got some package and there was quite big change in package default configuration file which will affect most of the users. On the other hand users can have pretty complicated constructions in their configuration files and if they did something like that, it's quite possible that they don't need to update their configuration file. So it is needed to tell users that if they are experiencing problems it is most likely because they touched default configuration. I can take care of it manually, mention it in various README and such, but more easier way would be following. User installs some packages. At the end of zypper or yast installation there will be warning "You've got 42 configuration files to merge, please run etc-update/dispatch-conf/whatever". Then user will run some easy interactive tool which will state which configuration files can use some manual merging, it will allow user to see difference between them and let him choose which one he wants or let him merge them manually somehow. Same warning about unmerged configuration files should appear each time user installs something, no matter if this installation didn't add any conflict, until all conflicts are fixed. User will be warned about possible problems, hacker can play with it and keep his changes against distribution defaults and ordinary user will just click on "use distribution default for all" if he ever encounter such a problem (shouldn't happen as ordinary user wouldn't touch configuration files). I explained basic expected work flow, so now what I think should be supported: * detect configuration files, their conflicts and allow their merge - this shouldn't be a big problem as rpm already stores conflicting files as .rpmnew/.rpmold * we probably want to keep even old versions to support three way merge to achieve even better results * if we are already storing these configuration files, maybe we should let users to backup their own changes as well, add support for versioning these configuration files * warn user that he has conflicts in configuration files - will require a little bit yast/zypper integration * merging tool should support both console and graphical interface (maybe use yast as possible frontend) * support for various formats (expandable by plugins) - ordinary diff may not be the best tool for merging XML files or ini files * ordinary user who don't change anything shouldn't need to know about this tool Relations: - Debian-like dist-upgrade live system full version upgrade (feature/id: 305634) Discussion: #1: Michal Hrusecky (-miska-) (2009-06-02 09:49:38) I was thinking about this a little bit more and if we want to support some kind of versioning and we want to use plugins for getting differences, it may be a good idea to request that plugins will be capable not only of producing nice output of difference between configuration files, but it probably should be able to apply it (behave not only as a 'diff', but also as a 'patch'). And we can build some simple version control system on top of it. Advantages: * can handle non-plain text files better (via plugins) * we can implement feature to discard or merge several commits into one to save space/make commit log more readable * can handle nicelly even insane configuration files like sqlite database or compressed directory with configuration files or any other binary format Disadvantages: * probably worse handling of text files * another thing to maintain/possible source of bugs #2: Stephan Kulow (coolo) (2009-06-09 11:51:14) this sounds like a lot of work to do right and with little gain. We lived quite well for years without it and whatever tool you come up will create new problems, there won't be a perfect tool. Applications with very bizzare config format usually come with their own configuration update tool that can be called from %post So I would put very low imporance on this. #3: Andreas Jaeger (a_jaeger) (2009-06-09 14:20:14) (reply to #2) The question is what is really needed to do here. If we could take some scripts from Gentoo and those would work without changes - cool. If we would need to change each and every config file for this, I'm also for rejecting. So, for me it's not clear what the cost is. #4: Michal Hrusecky (-miska-) (2009-06-12 16:12:55) (reply to #3) Well, merging utility can be probably ported from Gentoo with few minor tweaks (although it would be console only utility) or it could be written from scratch (with possible features in mind). This shouldn't be a big problem (it will just take some time). In feature #305634 comment #19 there is even an example how to write such a simple utility. With current rpm workflow, I think there is no need for packages changes. Only thing that would need some adjustements for this feature to be fully functional is integration with yast/zypper so thay will print warning about possible conflicts. I wouldn't suggest to force people to solve, just printing that there are some and that they should solve them using this utility should be enough. So it will need just some hack/new feature in installation procedure - run a program and allow it to create some notification for user. This is probably related to feature #301175 (301175) #5: Michael Andres (mlandres) (2009-06-15 12:02:05) (reply to #3) Libzypp checks for changed configfiles, and if they differ the 'diff' output is written to /var/log/YaST2/ (I don't know if YaST still mails this to root). Notification is also /var/log/zypp/history, and AFAIK zypper displays some info as well. But it's not a big deal to have a more flexible consumer for this information. The easiest would be to check for the presence of a certain script (or a directory containing scripts), and if it exists invoke it with package names and configfiles as argument. Then you can install any handler you like. #6: Elmar Stellnberger ATK (estellnb) (2009-08-04 11:48:04) At Bug 506815, Comment #2 we have a more simple proposial simply notifying the user about changes in configuration files and selecting an appropriate .rpmnew/.rpmold strategy. This should be the first thing to be implemented in this regard before we can move on to fancier things like an interactive auto-diff&merge (https://bugzilla.novell.com/show_bug.cgi?id=506815#c2). I personally would not regard the gain as little since forgetting to re- create config files can be very tedious especially if you forget about it and especially if there are regular upgrades as for the buildservice repos. #7: Ralph Ulrich (ulenrich) (2009-08-05 13:44:25) (reply to #6) Zypper should make a summary: 1. new rpmnew,rmpold files 2. How many of rpmnew,rpmold files were sitting around before action #8: Michal Hrusecky (-miska-) (2009-08-05 16:05:07) (reply to #7) As users should be notfied about other things as well after update, notifications shouldn't be handled by zypper (we've got Yast too) but we will need some general mechanism to notify user. And probably rpm should generate message about new configurations file conflict. But this message should be one of many possible messages (sometimes some other important user action is needed after update). I think users notifications are already discussed in feature#301175 (301175) . I also created wiki (http://en.opensuse.org/Configuration_Merge) page for this feature some time ago. #9: Stanislav Brabec (sbrabec) (2009-08-06 12:43:40) (reply to #8) User notification were also topic of: http://lists.opensuse.org/zypp-devel/2009-05/msg00001.html https://features.opensuse.org/305307 may be interesting way to inform as well (especially for auto updates) #10: Rick Beauchamp (techwiz03) (2009-10-31 02:22:14) Goes a bit farther than what I had to do to resolve a Yast-network config problem. * Yast correctly identified network settings were bad in config files but didn't report which files were at fault. * Yast then asked for acknowledgement that message was read. * Yast then re-scanned and looped through this process until shutdown forced. For me resolve was to search some 19 config files and compare them till bad settings were found and manually fixed.   I fear that an implementation of your type of merge configs could potentially make such a scenerio seriously compromise the debugging process. After experiencing such a problem it prompted me to develope a multi- file examiner. This program does not directly change any settings or config files. What you do is select the catagory you are looking at (Networking) then choose the type of sub-system (wireless). You get a list of files that pertain to wireless networking. You then can select up to 7 files from the list which each in turn display in a tabbed viewer. You can then examine them and even change then with-in the browser, but what my solution lacks is the ability to directly save the changes. What my solution does allow is copying the contents to the clipboard for pasting into a suitable writer for saving. Thusly the intent of my solution is targeted at finding an error and identifying where the error'd file is. Tying this in with your topic, if a config merge is desired, the safest way is always through manual interaction. If you use a program like mine to view multiple config files, you could merge the contents between tab-windows, and then copy the contents complete to the clipboard for saving under an explicite filename. my 2 cents worth. + #11: Michal Hrusecky (-miska-) (2009-11-27 11:25:05) (reply to #10) + This sounds interesting. Didn't thought about categorization. Is your + program somewhere available? How do you get such categories? Got two + ideas - by package name or by path (as network is nicelly categorized + in direcotries). I still think that automatic merge can be quite safe. + If you changed some little things like enabling some service, you + probably don't need to know that there was some typo fixed somewhere in + comments in the same file. Of course it has to be up to user to + enable/disable if he wants to do some automatic merging, but typically + I'm satistied with system defaults and I do care only about few + configuration files. -- openSUSE Feature: https://features.opensuse.org/306411