[yast-devel] Killing YCP
Hi, as many of you know, in January 2013 we started a project which played with an idea to automatically translate YCP portion of YaST codebase into Ruby. Me and Josef Reidinger worked on this and before Hack Week 9 [1] we reached a state where we were able to completely translate and run the "sysconfig" YaST module. During the Hack Week we decided to push for more. We assembled a team of 5 people (me, jreidinger, mvidner, lslezak, vmoravec) and tried to translate as much modules as possible. At the end of the Hack Week almost 60% of YaST's 150+ modules either compile to Ruby successfully, or we determined they don't need to be compiled at all (e.g. because they don't contain any runnable YCP code). I consider this a huge success. We also packaged some translated modules (e.g. "proxy" or "runlevel"). You can test them by downloading from our testing project: https://build.opensuse.org/project/show?project=YaST%3AHead%3Aruby For more details, see a video about our Hack Week project and its wiki page: http://www.youtube.com/watch?v=Bzgs1lmSKCw http://bit.ly/YPBW2Q ----- So far, the project to kill YCP was entirely unofficial. I'd like to use this opportunity to start a discussion about making it official and getting rid of YCP in openSUSE 13.1 timeframe (yes, I am that ambitious :-) From what I see, YCP is very simple and inflexible language. It doesn't support many useful concepts like OOP or exception handling, code written in it is hard to test, there are some annoying features (like a tendency to be "robust", which really means hiding errors) and YaST developers universally hate it (at least all the ones I talked to). More importantly, YCP is a language used only by YaST. This means the cost of its development and maintenance lies purely on SUSE/openSUSE. In practice, its development stagnated. There is also no community around the language and no external tools & libraries developed. As a result, every bit of code needs to be written by us and the tooling is weak. All this combined makes the cost (both financial and mental) of YaST development higher than if any other suitable high-level language was used, which limits YaST potential. Translating the code into Ruby would negate most of these disadvantages. We would be able to rely on a healthy ecosystem of tools and libraries (which are getting only better over time). There is a great pool of accumulated knowledge we can use. We would be also able to share code more efficiently with other SUSE/openSUSE projects written in Ruby, like WebYaST or SUSE Studio. We will have more flexibility in where to take YaST next. Last but not least, YaST developers would hate their jobs less :-) I think that rewriting YaST into Ruby makes sense. The Hack Week proved this is indeed doable and viable project, and my estimate is that it can be done in few months if we'll work hard enough (I can elaborate on this if needed). What do you think? ----- For the curious, the tool called "YCP Killer", which manages the translation, is available here: https://github.com/yast/ycp-killer The tool uses Y2R, a YCP -> Ruby transpiler: https://github.com/yast/y2r You can get the idea about what and how we translate from the spec (take this with a grain of salt, we want to improve the generated code *a lot* in the next weeks/months): https://github.com/yast/y2r/blob/master/spec/y2r_spec.md All this is very much work in progress, so please forgive any rough edges and lack of polish. -- David Majda SUSE Studio developer http://susestudio.com/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 15.4.2013 14:28, David Majda napsal(a):
Hi,
as many of you know, in January 2013 we started a project which played with an idea to automatically translate YCP portion of YaST codebase into Ruby. Me and Josef Reidinger worked on this and before Hack Week 9 [1] we reached a state where we were able to completely translate and run the "sysconfig" YaST module.
Just to make it clear: the whole YaST architecture is still the same (e.g. UI/modules/SCR levels, agents, Gtk/Qt/ncurses UIs, Pkg bindings, perl/python bindings...). We only translate YCP files to Ruby and run them via ruby-bindings, the rest of Yast is unchanged.
What do you think?
IMO we should go for it, the sooner the better ;-) We already have discussed advantages of using Ruby in the past (see e.g. [1][2] threads), I don't think we need to do it again. IMHO the only thing is to get green from managers so we can officially work on it. [1] http://lists.opensuse.org/yast-devel/2011-02/msg00041.html [2] http://lists.opensuse.org/yast-devel/2011-01/msg00009.html -- Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
What do you think?
IMO we should go for it, the sooner the better ;-)
I assume that we will need to maintain ycp and ruby code for some time together (SLE SP3, SP4). So, I'd like to know if it is possible to use ycp killer for converting patches. Another note is loss of history in github.
Michal Filka -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Tue, 16 Apr 2013 09:43:15 +0200 "Michal Filka" <michal.filka@suse.cz> wrote:
What do you think?
IMO we should go for it, the sooner the better ;-)
I assume that we will need to maintain ycp and ruby code for some time together (SLE SP3, SP4). So, I'd like to know if it is possible to use ycp killer for converting patches.
I agree, that it makes hard to convert patches. But on other side, for me more important is to properly make such patch. So in ruby you can very intensive test such change and behavior and then do such change in ycp should be relative easy. Or other way when you have patch for ycp, then you should write test for problematic part that demonstrate problem in ruby and then just implement it.
Another note is loss of history in github.
It is just partially true. You don't lost history, it just add one more step, when you need to check old ycp code that match ruby code and see history in ycp.
Michal Filka
Josef -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
I assume that we will need to maintain ycp and ruby code for some time together (SLE SP3, SP4). So, I'd like to know if it is possible to use ycp killer for converting patches.
I agree, that it makes hard to convert patches. But on other side, for me more important is to properly make such patch. So in ruby you can very intensive test such change and behavior and then do such change in ycp should be relative easy. Or other way when you have patch for ycp, then you should write test for problematic part that demonstrate problem in ruby and then just implement it.
Doing a work twice is always problem ;-) I see two ways: - translate whole module again after each patch in ycp. Problem is potential loss of changes done in ruby code only. - translate the patch only (if possible) and try to integrate it somehow by hand.
Another note is loss of history in github.
It is just partially true. You don't lost history, it just add one more step, when you need to check old ycp code that match ruby code and see history in ycp.
This becames to be a problem in case of refactoring in ruby code.
Michal -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Tue, 16 Apr 2013 10:02:29 +0200 "Michal Filka" <michal.filka@suse.cz> wrote:
I assume that we will need to maintain ycp and ruby code for some time together (SLE SP3, SP4). So, I'd like to know if it is possible to use ycp killer for converting patches.
I agree, that it makes hard to convert patches. But on other side, for me more important is to properly make such patch. So in ruby you can very intensive test such change and behavior and then do such change in ycp should be relative easy. Or other way when you have patch for ycp, then you should write test for problematic part that demonstrate problem in ruby and then just implement it.
Doing a work twice is always problem ;-) I see two ways: - translate whole module again after each patch in ycp. Problem is potential loss of changes done in ruby code only.
Yes, you will loose changes in ruby. Only possible way that cames to my mind is to repeat translation, create patch again original translated ruby and try to apply it to master. There is more potential way how to do it.
- translate the patch only (if possible) and try to integrate it somehow by hand.
you cannot translate only patch, as ycp is really dependent on context, so only way I describe above is possible or do it twice.
Another note is loss of history in github.
It is just partially true. You don't lost history, it just add one more step, when you need to check old ycp code that match ruby code and see history in ycp.
This becames to be a problem in case of refactoring in ruby code.
But you have same problem in ycp, I don't know any version control tool that can track all code refactoring. It is not problem of ruby.
Michal
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Tue, Apr 16, 2013 at 10:02:29AM +0200, Michal Filka wrote:
I assume that we will need to maintain ycp and ruby code for some time together (SLE SP3, SP4). So, I'd like to know if it is possible to use ycp killer for converting patches.
I agree, that it makes hard to convert patches. But on other side, for me more important is to properly make such patch. So in ruby you can very intensive test such change and behavior and then do such change in ycp should be relative easy. Or other way when you have patch for ycp, then you should write test for problematic part that demonstrate problem in ruby and then just implement it.
Doing a work twice is always problem ;-) I see two ways: - translate whole module again after each patch in ycp. Problem is potential loss of changes done in ruby code only. - translate the patch only (if possible) and try to integrate it somehow by hand.
I think that after the conversion, the code bases will diverge, because it will be finally possible to make cleanups in the ruby code. Fixes from SLE 11 will have to be ported manually. Yes, it will be extra work, but since SLE 12 will have been out, the demand for SLE 11 patches will decrease (not overnight, but eventually). I don't see how we can make this cost smaller but I hope that the benefits are far greater.
Another note is loss of history in github.
It is just partially true. You don't lost history, it just add one more step, when you need to check old ycp code that match ruby code and see history in ycp.
This becames to be a problem in case of refactoring in ruby code.
Unlike the previous problem, we can simulate this one in advance easily and try some ways to make it easier for us. Like David said, certainly it will help to store the original YCP path. -- Martin Vidner, Cloud & Systems Management Team http://en.opensuse.org/User:Mvidner Kuracke oddeleni v restauraci je jako fekalni oddeleni v bazenu
On 04/16/2013 09:43 AM, Michal Filka wrote:
What do you think?
IMO we should go for it, the sooner the better ;-)
I assume that we will need to maintain ycp and ruby code for some time together (SLE SP3, SP4). So, I'd like to know if it is possible to use ycp killer for converting patches.
Well, I though of this yesterday as well. What you can IMO do: - convert original YCP code to Ruby - convert patched YCP code to Ruby - diff these two results I know that this is not really convenient, however, I believe that this can be scripted quite easily (so that you can get a Ruby diff for a specific YCP commit, e.g.). The question is, how much non-code changes the resulting diff will contain.
Another note is loss of history in github.
Good point. Jiri
Michal Filka
-- Regards, Jiri Srain Project Manager --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: jsrain@suse.com Lihovarska 1060/12 tel: +420 284 084 659 190 00 Praha 9 fax: +420 284 084 001 Czech Republic http://www.suse.com -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 16.4.2013 09:43, Michal Filka napsal(a):
What do you think?
Another note is loss of history in github.
We already thought about it. We plan to include a file name of the original YCP file in a comment at the top of translated Ruby file. Then it will be clear where to look for further history of the code. We even think about writing a git plugin that will be aware of the mapping and will be able to do "git log" or "git blame" equivalent on the .rb file that will actually work with the .ycp file it was created from (saving the manual lookup of the original file). -- David Majda SUSE Studio developer http://susestudio.com/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, Apr 15, 2013 at 02:28:44PM +0200, David Majda wrote:
What do you think?
I am all for it, which shouldn't be a surprise. -- Martin Vidner, Cloud & Systems Management Team http://en.opensuse.org/User:Mvidner Kuracke oddeleni v restauraci je jako fekalni oddeleni v bazenu
participants (6)
-
David Majda -
Jiri Srain -
Josef Reidinger -
Ladislav Slezak -
Martin Vidner -
Michal Filka