[yast-devel] Discussion - How should look new module written in ruby
Hi, we have now ruby and it brings a lot of features for free. I want to write example refactoring and new skeleton for new module. And I would like to discuss here what we should use. I think that now is good time, because before conversion we are forced to write it itself, but now we can choose from existing solution that is already implemented in ruby. 1) for skeleton I think that skeleton should be as small as possible, so move all repeating code to own method provided by a module (that is nice feature of rails, you can start really quickly ). And I think we should make clear difference between leaf module and module that can be reused by other modules. Because often there are Yast modules that have modules, but they use it themself exclusivelly. For such case now should be "lib" directory where is private Yast module code such us specific configuration logic or module specific dialogs. Advantage of such step is that you need not care about backward compatibility, as there is no public API and we can use all ruby features. 2) When module need to provide API for other modules ( so API should be based on requirements from other modules ) it should follow simple ruby - "Tell, don't ask" [1]. That helps with reducing of data leaks often seen in Yast ( when you change a map structure, you have problem because some modules depends on exact map structure ). The second nice benefit is that API will be easy to go via component system as there is almost no data flow. 3) No includes. Includes is relict from ycp and it should be moved to lib as common ruby class or modules. ( perl and python also doesn't have it ) 4) Exceptions. Ruby language provide us exceptions that can really simplify method code-flow. If there is error and it is reported by return value, then you need to explicitelly check it and go up if needed. Exception allow to handle it when you need it and if you don't handle it, then quickly recognize it[1b]. Current solution is that ruby-bindings catch exceptions when exception is catch between languages when calling modules ( so if both modules are in ruby, then it is passed ) and when use WFM::Call ( in case of exception, it is catched and return nil ). 5) testing framework. We definitively need new one, old one is really horrible to use and fragile. Ruby provide a lot of nice test frameworks, the well known are Test:Unit[2] ( classical test unit framework with asserts ) and RSpec that is designed to fit nice for TDD [3]. I personally use for a long time Test:Unit and try RSpec few months ago and I really like RSpec, especially how tests looks like and how nice is its output. 6) Documentation. For code documentation we translate comments to YARD[4]. But there is also documentation that sits around in docbook format. I found quite hard to write new parts of documentation and keep it up-to-date. I found tool that can convert docbook to textual based markdown format[5]. And I also thing that documentation should be together with code and also examples should live with code. YARD support it in quite nice way (support inlined markdown). See e.g. documentation of ActiveRecord::Base which lives together code and how it can look like [6]. 7) Continuous integration. We found that our jenkings node how few drawbacks, so we should also improve it. Lslezak have nice proposal how to simplify it. We can just package new commit and and call `osc build` and if it succeeds then submit new package to YaST:Head. Also if we improve our testing we can use e.g. travis and integrate it to test all of our pull request if it breaks test [7]. If we improve it, we can then verify also our maintenance branches and we can share load between more runners and dependency is mainly osc and what we need to package sources. 8) build program. YaST currently use autotools. It has few drawbacks - it is written in different language then rest of plugins ( M4 versus ruby now ), not so popular today ( so less people understand it, especially in ruby community ) and not so easy to share functionality ( currently we use Makefile.am.common and other tricks ). Common ruby build tool is Rake[8], question is if we want use it. I welcome any comments, suggestions or counter-proposal. Josef [1] http://pragprog.com/articles/tell-dont-ask [1b] http://nedbatchelder.com/text/exceptions-vs-status.html [2] http://www.ruby-doc.org/stdlib-2.0/libdoc/test/unit/rdoc/Test/Unit/Assertion... [3] http://rspec.info/ [4] http://yardoc.org/ [5] http://daringfireball.net/projects/markdown/syntax [6] http://api.rubyonrails.org/classes/ActiveRecord/Base.html [7] http://about.travis-ci.org/blog/2012-09-04-pull-requests-just-got-even-more-... [8] http://rake.rubyforge.org/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 1.8.2013 09:53, Josef Reidinger napsal(a):
1) for skeleton I think that skeleton should be as small as possible, so move all repeating code to own method provided by a module (that is nice feature of rails, you can start really quickly ). And I think we should make clear difference between leaf module and module that can be reused by other modules. Because often there are Yast modules that have modules, but they use it themself exclusivelly. For such case now should be "lib" directory where is private Yast module code such us specific configuration logic or module specific dialogs. Advantage of such step is that you need not care about backward compatibility, as there is no public API and we can use all ruby features.
2) When module need to provide API for other modules ( so API should be based on requirements from other modules ) it should follow simple ruby - "Tell, don't ask" [1]. That helps with reducing of data leaks often seen in Yast ( when you change a map structure, you have problem because some modules depends on exact map structure ). The second nice benefit is that API will be easy to go via component system as there is almost no data flow.
I am not sure this can be understood by anyone beside people who talked to Josef in person about these issues already :-) I'll try to restate the idea more clearly, because I think it's pretty important. Josef, please correct my re-statement is wrong. First, the work "module" is overloaded. It can mean 3 things: * YaST module (a set of files roughly corresponding to a package and a Git repo) * YCP module (YCP file that begins with `module "foo"` declaration) * Ruby module (Ruby syntactic construct) YCP modules were translated into Ruby. After the translation they are quite heavy-weight, require declaring of published methods/variables and they are global/singleton in nature. They are not natural citizens in RUby world. This is the price for full compatibility with YCP and YaST component system. The proposal is to use YCP-compatible modules only for public API of given YaST module. Anything internal to the module should be implemented in native Ruby way (using Ruby modules and classes) and placed in the "lib" directory. ----- I agree with the idea above and personally, I would even like to see some standardization in what goes into the "lib" directory. For example, where the models and views (in the MVC sense) are placed and how they are named.
3) No includes. Includes is relict from ycp and it should be moved to lib as common ruby class or modules. ( perl and python also doesn't have it )
Yes. No includes in new code.
5) testing framework. We definitively need new one, old one is really horrible to use and fragile. Ruby provide a lot of nice test frameworks, the well known are Test:Unit[2] ( classical test unit framework with asserts ) and RSpec that is designed to fit nice for TDD [3]. I personally use for a long time Test:Unit and try RSpec few months ago and I really like RSpec, especially how tests looks like and how nice is its output.
I would also favor RSpec. Specs written using it are very readable and can be structured nicely.
6) Documentation. For code documentation we translate comments to YARD[4]. But there is also documentation that sits around in docbook format. I found quite hard to write new parts of documentation and keep it up-to-date. I found tool that can convert docbook to textual based markdown format[5]. And I also thing that documentation should be together with code and also examples should live with code. YARD support it in quite nice way (support inlined markdown). See e.g. documentation of ActiveRecord::Base which lives together code and how it can look like [6].
I think it makes sense to use just one format for the technical documentation and YARD is a natural choice here because it's used for documentation comments already.
8) build program. YaST currently use autotools. It has few drawbacks - it is written in different language then rest of plugins ( M4 versus ruby now ), not so popular today ( so less people understand it, especially in ruby community ) and not so easy to share functionality ( currently we use Makefile.am.common and other tricks ). Common ruby build tool is Rake[8], question is if we want use it.
At least for pure Ruby YaST modules, it makes a lot of sense, but there are lot of questions. The most important ones are: * Do we want to use unified tooling for all YaST modules, or is this not strictly necessary? * If we want unified tooling, do we want autotools to wrap Rake, or Rake wrap autotools in cases where they are needed (at least C++ code)? Or can Rake do all the work? Maybe some prototyping would be the best way to explore this area. -- David Majda SUSE Studio developer http://susestudio.com/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, 01 Aug 2013 11:03:33 +0200
David Majda
Dne 1.8.2013 09:53, Josef Reidinger napsal(a):
1) for skeleton I think that skeleton should be as small as possible, so move all repeating code to own method provided by a module (that is nice feature of rails, you can start really quickly ). And I think we should make clear difference between leaf module and module that can be reused by other modules. Because often there are Yast modules that have modules, but they use it themself exclusivelly. For such case now should be "lib" directory where is private Yast module code such us specific configuration logic or module specific dialogs. Advantage of such step is that you need not care about backward compatibility, as there is no public API and we can use all ruby features.
2) When module need to provide API for other modules ( so API should be based on requirements from other modules ) it should follow simple ruby - "Tell, don't ask" [1]. That helps with reducing of data leaks often seen in Yast ( when you change a map structure, you have problem because some modules depends on exact map structure ). The second nice benefit is that API will be easy to go via component system as there is almost no data flow.
I am not sure this can be understood by anyone beside people who talked to Josef in person about these issues already :-) I'll try to restate the idea more clearly, because I think it's pretty important. Josef, please correct my re-statement is wrong.
First, the work "module" is overloaded. It can mean 3 things:
* YaST module (a set of files roughly corresponding to a package and a Git repo)
* YCP module (YCP file that begins with `module "foo"` declaration)
* Ruby module (Ruby syntactic construct)
YCP modules were translated into Ruby. After the translation they are quite heavy-weight, require declaring of published methods/variables and they are global/singleton in nature. They are not natural citizens in RUby world. This is the price for full compatibility with YCP and YaST component system.
The proposal is to use YCP-compatible modules only for public API of given YaST module. Anything internal to the module should be implemented in native Ruby way (using Ruby modules and classes) and placed in the "lib" directory.
-----
I agree with the idea above and personally, I would even like to see some standardization in what goes into the "lib" directory. For example, where the models and views (in the MVC sense) are placed and how they are named.
Thanks for making it more clear. I agree that some standardization would be nice.
3) No includes. Includes is relict from ycp and it should be moved to lib as common ruby class or modules. ( perl and python also doesn't have it )
Yes. No includes in new code.
5) testing framework. We definitively need new one, old one is really horrible to use and fragile. Ruby provide a lot of nice test frameworks, the well known are Test:Unit[2] ( classical test unit framework with asserts ) and RSpec that is designed to fit nice for TDD [3]. I personally use for a long time Test:Unit and try RSpec few months ago and I really like RSpec, especially how tests looks like and how nice is its output.
I would also favor RSpec. Specs written using it are very readable and can be structured nicely.
6) Documentation. For code documentation we translate comments to YARD[4]. But there is also documentation that sits around in docbook format. I found quite hard to write new parts of documentation and keep it up-to-date. I found tool that can convert docbook to textual based markdown format[5]. And I also thing that documentation should be together with code and also examples should live with code. YARD support it in quite nice way (support inlined markdown). See e.g. documentation of ActiveRecord::Base which lives together code and how it can look like [6].
I think it makes sense to use just one format for the technical documentation and YARD is a natural choice here because it's used for documentation comments already.
8) build program. YaST currently use autotools. It has few drawbacks - it is written in different language then rest of plugins ( M4 versus ruby now ), not so popular today ( so less people understand it, especially in ruby community ) and not so easy to share functionality ( currently we use Makefile.am.common and other tricks ). Common ruby build tool is Rake[8], question is if we want use it.
At least for pure Ruby YaST modules, it makes a lot of sense, but there are lot of questions. The most important ones are:
* Do we want to use unified tooling for all YaST modules, or is this not strictly necessary?
Well, even now we use two autotools + cmake. But I found it annoying as yast have few specifics and we need to have helpers for such specifics in both of build tools. So It is not strictly necessary, but it will be really helpful.
* If we want unified tooling, do we want autotools to wrap Rake, or Rake wrap autotools in cases where they are needed (at least C++ code)? Or can Rake do all the work?
I think that Rake can do both and we should unify it - https://github.com/joeyates/rake-builder and http://stackoverflow.com/questions/4272727/rakefile-rules-in-c-project
Maybe some prototyping would be the best way to explore this area.
Yes, we can try it. Of course first part is if we want change old code or use rake for new one or unify it, as it include a lot of file. Josef -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Hi, we have now ruby and it brings a lot of features for free. I want to write example refactoring and new skeleton for new module. And I would like to discuss here what we should use. I think that now is good time, because before conversion we are forced to write it itself, but now we can choose from existing solution that is already implemented in ruby.
1) for skeleton I think that skeleton should be as small as possible, so move all repeating code to own method provided by a module (that is nice feature of rails, you can start really quickly ). And I think we should make clear difference between leaf module and module that can be reused by other modules. Because often there are Yast modules that have modules, but they use it themself exclusivelly. For such case now should be "lib" directory where is private Yast module code such us specific configuration logic or module specific dialogs. Advantage of such step is that you need not care about backward compatibility, as there is no public API and we can use all ruby features. Does this mean we'll get an additional dir called 'lib' while the 'src'
Dne 1.8.2013 09:53, Josef Reidinger napsal(a): dir will serve for module public API? Are there some best-practice tips how to structure that 'lib' dir to keep the layout similar between yast modules?
2) When module need to provide API for other modules ( so API should be based on requirements from other modules ) it should follow simple ruby - "Tell, don't ask" [1]. That helps with reducing of data leaks often seen in Yast ( when you change a map structure, you have problem because some modules depends on exact map structure ). The second nice benefit is that API will be easy to go via component system as there is almost no data flow. It would appreciate toexpandon this a bit more as I could not extract much useful and specific information from this description and reference. 3) No includes. Includes is relict from ycp and it should be moved to lib as common ruby class or modules. ( perl and python also doesn't have it )
4) Exceptions. Ruby language provide us exceptions that can really simplify method code-flow. If there is error and it is reported by return value, then you need to explicitelly check it and go up if needed. Exception allow to handle it when you need it and if you don't handle it, then quickly recognize it[1b]. Current solution is that ruby-bindings catch exceptions when exception is catch between languages when calling modules ( so if both modules are in ruby, then it is passed ) and when use WFM::Call ( in case of exception, it is catched and return nil ).
5) testing framework. We definitively need new one, old one is really horrible to use and fragile. Ruby provide a lot of nice test frameworks, the well known are Test:Unit[2] ( classical test unit framework with asserts ) and RSpec that is designed to fit nice for TDD [3]. I personally use for a long time Test:Unit and try RSpec few months ago and I really like RSpec, especially how tests looks like and how nice is its output. In case we go with RSpec we should be aware of the plans of the rspec dev team, mainly of the DSL "should"-based syntax change which is going to be depreciated in the Rspec 3 . This link is worth of reading: http://myronmars.to/n/dev-blog/2013/07/the-plan-for-rspec-3 6) Documentation. For code documentation we translate comments to YARD[4]. But there is also documentation that sits around in docbook format. I found quite hard to write new parts of documentation and keep it up-to-date. I found tool that can convert docbook to textual based markdown format[5]. And I also thing that documentation should be together with code and also examples should live with code. YARD support it in quite nice way (support inlined markdown). See e.g. documentation of ActiveRecord::Base which lives together code and how it can look like [6]. I fully agree, but beside that what I'm missing is the basic documentation or summary shown on github index page for every yast module, it's strange not having a short README file in the repo root.
When discussing docs, will we be using the github wiki for the general FAQs or HOWTOs?
7) Continuous integration. We found that our jenkings node how few drawbacks, so we should also improve it. Lslezak have nice proposal how to simplify it. We can just package new commit and and call `osc build` and if it succeeds then submit new package to YaST:Head. Also if we improve our testing we can use e.g. travis and integrate it to test all of our pull request if it breaks test [7]. If we improve it, we can then verify also our maintenance branches and we can share load between more runners and dependency is mainly osc and what we need to package sources.
Reading about integration and testing raised in my mind ideas about yast ui testing. Would it bring some significant positives from the yast future perspective? If yes, what would be the way to go, does it make sense to use e.g. ycp-macro-player/recorder, cucumber etc as I currently do not see into the details of how the components communicate. The next topic for discussion could be test coverage. The most used ruby library here seems to be simplecov: https://github.com/colszowka/simplecov . It should be easy to setup and to use and it has lot of features. I'm going to try it out with a single yast module and then report back results.
8) build program. YaST currently use autotools. It has few drawbacks - it is written in different language then rest of plugins ( M4 versus ruby now ), not so popular today ( so less people understand it, especially in ruby community ) and not so easy to share functionality ( currently we use Makefile.am.common and other tricks ). Common ruby build tool is Rake[8], question is if we want use it.
Yes, please! Thanks. vlado
[1] http://pragprog.com/articles/tell-dont-ask [1b] http://nedbatchelder.com/text/exceptions-vs-status.html [2] http://www.ruby-doc.org/stdlib-2.0/libdoc/test/unit/rdoc/Test/Unit/Assertion... [3] http://rspec.info/ [4] http://yardoc.org/ [5] http://daringfireball.net/projects/markdown/syntax [6] http://api.rubyonrails.org/classes/ActiveRecord/Base.html [7] http://about.travis-ci.org/blog/2012-09-04-pull-requests-just-got-even-more-... [8] http://rake.rubyforge.org/
Dne 1.8.2013 15:26, Vladimir Moravec napsal(a):
1) for skeleton I think that skeleton should be as small as possible, so move all repeating code to own method provided by a module (that is nice feature of rails, you can start really quickly ). And I think we should make clear difference between leaf module and module that can be reused by other modules. Because often there are Yast modules that have modules, but they use it themself exclusivelly. For such case now should be "lib" directory where is private Yast module code such us specific configuration logic or module specific dialogs. Advantage of such step is that you need not care about backward compatibility, as there is no public API and we can use all ruby features. Does this mean we'll get an additional dir called 'lib' while the 'src'
Dne 1.8.2013 09:53, Josef Reidinger napsal(a): dir will serve for module public API?
No, the "lib" directory will be at the same level as "modules", "include", etc.: /src/lib /src/modules /src/include
Are there some best-practice tips how to structure that 'lib' dir to keep the layout similar between yast modules?
We'll need to invent some.
5) testing framework. We definitively need new one, old one is really horrible to use and fragile. Ruby provide a lot of nice test frameworks, the well known are Test:Unit[2] ( classical test unit framework with asserts ) and RSpec that is designed to fit nice for TDD [3]. I personally use for a long time Test:Unit and try RSpec few months ago and I really like RSpec, especially how tests looks like and how nice is its output. In case we go with RSpec we should be aware of the plans of the rspec dev team, mainly of the DSL "should"-based syntax change which is going to be depreciated in the Rspec 3 . This link is worth of reading: http://myronmars.to/n/dev-blog/2013/07/the-plan-for-rspec-3
Thanks a lot for this link, I didn't know about the described changes.
6) Documentation. For code documentation we translate comments to YARD[4]. But there is also documentation that sits around in docbook format. I found quite hard to write new parts of documentation and keep it up-to-date. I found tool that can convert docbook to textual based markdown format[5]. And I also thing that documentation should be together with code and also examples should live with code. YARD support it in quite nice way (support inlined markdown). See e.g. documentation of ActiveRecord::Base which lives together code and how it can look like [6]. I fully agree, but beside that what I'm missing is the basic documentation or summary shown on github index page for every yast module, it's strange not having a short README file in the repo root.
I agree, basic README.md for each module would be nice. It should at least describe the module and contain a pointer to more general development information (currently in openSUSE wiki).
When discussing docs, will we be using the github wiki for the general FAQs or HOWTOs?
AFAIK we use openSUSE wiki now. This is both good and bad. Good because it is one central point (compared to a bunch of independent repositories). Good because it is independent on GitHub. Bad because compared to GitHub wiki it is heavyweight. Bad because it is it an unexpected place from POV of someone who stumbles upon any YaST Git repo. Personally, I'd prefer to use GitHub wiki of appropriate repositories (devtools, ...). -- David Majda SUSE Studio developer http://susestudio.com/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, 05 Aug 2013 14:53:07 +0200
David Majda
Personally, I'd prefer to use GitHub wiki of appropriate repositories (devtools, ...).
You can use them, and post when there are changes. If there is some way to receive notifications about wiki changes from github.com directly let me know. I can take care that it finds way into openSUSE wiki. -- Regards, Rajko. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (4)
-
David Majda -
Josef Reidinger -
Rajko -
Vladimir Moravec