[yast-devel] Yast webservice - sharing .rake files
Hi all, I started to implement an example WebYast package and I have a yast2-devtools skeleton for REST API package ready. But I found that many things are duplicated in all REST plugins. E.g. each Rakefile contains the same copied targets (e.g. 'package'). The common targets should be shared instead of copying. If there is a bug or we want to add a new feature we have to fix a lot of packages... Here is my proposal: - Add include.rake file into rakelib[*] subdirectory in each plugin, see the attachment. (It includes the files from the current directory if it's called inside the git repository otherwise it includes system files from /srv/www/yastws/...) - Define the shared tasks in rest-service/webservice/lib/tasks/webservice subdirectory. The tasks should use webservice: namespace prefix so it's obvious that they are defined in the webservice not in the plugin or somewhere else. (See syntax_check.rake example) After that 'rake -T' in a plugin subdirectory will display the syntax check target from the shared directory, no need to copy&paste the code. What do you think about it? Any comments? I'm not a Rakefile expert... [*] I found that tasks defined in 'tasks' subdirectory in plugins are not found by rake, they must be in 'rakelib' subdirectory instead. We have to call 'git mv tasks rakelib' for all plugins and fix the .spec files... -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ # import global .rake files for plugins if File.exist? "#{File.dirname(__FILE__)}/../../../webservice/lib/tasks/webservice/" # use delopemnt files if we are in git repository Dir["#{File.dirname(__FILE__)}/../../../webservice/lib/tasks/webservice/*.rake"].each { |ext| load ext } else # use the files from yast2-webservice package Dir["/srv/www/yastws/lib/tasks/webservice/*.rake"].each { |ext| load ext } end require 'rake' namespace :webservice do desc "Check syntax of all Ruby (*.rb) files" task :syntax_check do `find . -name "*.rb" |xargs -n1 ruby -c |grep -v "Syntax OK"` puts "* Done" end end
Ladislav Slezak schrieb:
Hi all,
I started to implement an example WebYast package and I have a yast2-devtools skeleton for REST API package ready.
But I found that many things are duplicated in all REST plugins. E.g. each Rakefile contains the same copied targets (e.g. 'package'). The common targets should be shared instead of copying. If there is a bug or we want to add a new feature we have to fix a lot of packages...
I think it is an good idea. But before we are splitting the rakefiles we should check if there is an "overloading mechanism" available. E.G. "rake package". Is it possible to overload the "rake package" call defined in webservice by a "rake package" call defined in the plugin ? The other concern which I have is that we should take care that our product is quite intuitive (especially for the programmer). So we will have to find the balance between module/plugin dependencies and and readability/understandability of the code. I know that is quite subjective :-) My personal aim is to give the programmer a mainframe with which we can say: "Just begin !" without reading much documentation or having a deep insight about the module dependencies. Greetings Stefan
Here is my proposal:
- Add include.rake file into rakelib[*] subdirectory in each plugin, see the attachment. (It includes the files from the current directory if it's called inside the git repository otherwise it includes system files from /srv/www/yastws/...)
- Define the shared tasks in rest-service/webservice/lib/tasks/webservice subdirectory. The tasks should use webservice: namespace prefix so it's obvious that they are defined in the webservice not in the plugin or somewhere else. (See syntax_check.rake example)
After that 'rake -T' in a plugin subdirectory will display the syntax check target from the shared directory, no need to copy&paste the code.
What do you think about it? Any comments? I'm not a Rakefile expert...
[*] I found that tasks defined in 'tasks' subdirectory in plugins are not found by rake, they must be in 'rakelib' subdirectory instead. We have to call 'git mv tasks rakelib' for all plugins and fix the .spec files...
--
Best Regards
Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/
-- ******************************************************************************* Stefan Schubert SUSE LINUX GmbH - Maxfeldstrasse 5 - D-90409 Nuernberg, Germany e-mail: schubi@suse.de ------------------------------------------------------------------------------- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Stefan Schubert wrote:
I think it is an good idea. But before we are splitting the rakefiles we should check if there is an "overloading mechanism" available. E.G. "rake package". Is it possible to overload the "rake package" call defined in webservice by a "rake package" call defined in the plugin ?
Well, by default rake joins the same tasks together, if desc "FOO" task :foo do puts "FOO" end is defined twice 'rake -T' prints: rake foo # FOO / FOO and 'rake foo' prints: FOO FOO 'rake package' should not behave this way. It's possible to redefine a task, but it's not simple, see e.g. http://blog.jayfields.com/2008/02/rake-task-overwriting.html or http://snippets.dzone.com/posts/show/2032 But IMO this should not be a problem, majority of modules should use the predefined targets, redefining should be very rare. -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Ladislav Slezak wrote:
It's possible to redefine a task, but it's not simple, see e.g. http://blog.jayfields.com/2008/02/rake-task-overwriting.html or http://snippets.dzone.com/posts/show/2032
Or we could use another approach, including of the predefined tasks is done by this code (which must be in every plugin): Dir["/srv/www/yastws/lib/tasks/webservice/*.rake"].each { |ext| load ext } We could simply add an 'if' statement into the block and skip (not load) the unwanted tasks without need to redefine them later... This is a better solution. -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Ladislav Slezak wrote:
Ladislav Slezak wrote:
It's possible to redefine a task, but it's not simple, see e.g. http://blog.jayfields.com/2008/02/rake-task-overwriting.html or http://snippets.dzone.com/posts/show/2032
Or we could use another approach, including of the predefined tasks is done by this code (which must be in every plugin):
Dir["/srv/www/yastws/lib/tasks/webservice/*.rake"].each { |ext| load ext }
What is the final goal of this? We also will have duplication of tasks between the web service and the web client, therefore it makes more sense to have the rake tasks defined in their own gem/library, where also stuff we use across both environments can be moved to. Once the tasks are in a library we don't need to hardcode paths anymore. -- Duncan Mac-Vicar P. - Engineering Manager, YaST SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Duncan Mac-Vicar Prett wrote:
Ladislav Slezak wrote:
Dir["/srv/www/yastws/lib/tasks/webservice/*.rake"].each { |ext| load ext }
What is the final goal of this?
The goal is to load all tasks from the webservice base in plugins.
We also will have duplication of tasks between the web service and the web client, therefore it makes more sense to have the rake tasks defined in their own gem/library, where also stuff we use across both environments can be moved to.
OK, I agree, this was my first attempt to implement basic task sharing.
Once the tasks are in a library we don't need to hardcode paths anymore.
I'll try building a separate library... -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Ladislav Slezak napsal(a):
Duncan Mac-Vicar Prett wrote:
Once the tasks are in a library we don't need to hardcode paths anymore.
I'll try building a separate library...
OK, I have created yast2-webservice-tasks ruby gem in rest-service/webservice-tasks directory. The shared tasks (*.rake) are in lib/tasks subdirectory, there is webservice.rb which loads them. You don't have to specify any strange path in Rakefile anymore, the use is more straightforward, see below. Please review it, this is my first ruby gem package... I have a question, are there any conventions how to name a gem (resp. the resulting RPM package)? The gem is named 'yast2-webservice-tasks', the RPM package is 'rubygem-yast2-webservice-tasks'. Is that OK? How to build the gem -------------------- # cd rest-service/webservice-tasks # rake package How to install it ----------------- # sudo gem install yast2-webservice-tasks (or you can build the RPM package in package subdirectory) How to use it in the webservice (or plugins) -------------------------------------------- Just add require 'tasks/webservice' line to your Rakefile. The tasks will be found and loaded automatically, you can verify it with 'rake -T' command. Existing tasks -------------- git_check - checks whether the local and remote GIT repo are in sync (e.g. no uncommitted changes...) syntax_check - check syntax of all Ruby (*.rb) files We should use both tasks in the package task, similar to 'make package' in standard Yast modules. How to add a new task --------------------- Just place the .rake file into rest-service/webservice-tasks/lib/tasks subdirectory. Note: I have adapted the devtools skeleton to use this gem. -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Ladislav Slezak napsal(a):
Existing tasks --------------
git_check - checks whether the local and remote GIT repo are in sync (e.g. no uncommitted changes...)
syntax_check - check syntax of all Ruby (*.rb) files
We should use both tasks in the package task, similar to 'make package' in standard Yast modules.
I have added 'package' task (the standard package task using Rake::PackageTask class) and 'dist' task which calls 'syntax_check', 'git_check' and then builds the package using 'package'. Actually it would be better if we had 'package' and 'package-local' tasks, just like in Yast modules. Unfortunately rake doesn't allow to rename a task and Rake::PackageTask doesn't accept any parameter to change the default task name. (IMO it's a bug or missing feature, e.g. Rake::TestTask allows to name the test task as you wish.) USE 'rake dist' TO BUILD THE DISTRIBUTION PACKAGE Use 'rake package' if you want just to do a test build or for testing (non-distribution) packages. dist = package (in Yast) package = package-local (in Yast) -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Ladislav Slezak wrote:
Actually it would be better if we had 'package' and 'package-local' tasks, just like in Yast modules. Unfortunately rake doesn't allow to rename a task and Rake::PackageTask doesn't accept any parameter to change the default task name. (IMO it's a bug or missing feature, e.g. Rake::TestTask allows to name the test task as you wish.)
no but you can define a new task than uses Rake:: something to run another task, or aliasing it somehow? def alias_task(new_name, old_name) Rake::Task::TASKS[new_name.to_s] = Rake::Task::TASKS.delete(old_name.to_s) end -- Duncan Mac-Vicar P. - Engineering Manager, YaST SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Duncan Mac-Vicar Prett wrote:
no but you can define a new task than uses Rake:: something to run another task, or aliasing it somehow?
def alias_task(new_name, old_name) Rake::Task::TASKS[new_name.to_s] = Rake::Task::TASKS.delete(old_name.to_s) end
I wish it was so easy... The task name is stored in Rake::Task class and there is Rake::TaskManager module which has 'task_name' -> Rake::Task mapping Hash. The module is mixed in Rake::Application class which is used in Rake::Task class... It took me quite long time to figure it out. Moreover the tasks are specified as Symbols (like :package), but the internal hash stores them as Strings. I missed a single ".to_s" and it behaved very strangely, like 'rake -T' displayed all tasks including package task, but calling 'rake package' failed with "Don't know how to build task 'package'" error message... Very nice task for debugging rake.rb... Anyway, I managed to rename 'package' task to 'package-local' task. If you ever had to rename a rake task then look at webservice-tasks/lib/tasks/rake_rename_task.rb and package-local.rake files. Now we finally have 'package' and 'package-local' tasks which work as expected! -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Stefan Schubert wrote: [...]
I think it is an good idea. But before we are splitting the rakefiles we should check if there is an "overloading mechanism" available. E.G. "rake package". Is it possible to overload the "rake package" call defined in webservice by a "rake package" call defined in the plugin ?
I have improved the task loader, in the simple case (no need to redefine a task) just add require 'tasks/webservice' to the Rakefile. If you need to redefine a task then it must be explicitly excluded in the loading code so it is defined just once. There is an extra helper class to do this, use this instead of the above code: require 'tasks/webservice_tasks' WebserviceTasks.loadTasks(:exclude => ["package.rake"]) This example loads all tasks except :package task which can be defined later. (This code is used in rubygem-yast2-webservice-tasks package itself, it's a gem package which is packaged in a different way.) See rest-service/webservice-tasks/README for more details (it also supports wildcards...). -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
participants (3)
-
Duncan Mac-Vicar Prett -
Ladislav Slezak -
Stefan Schubert