On Thu, 15 Nov 2007, Klaus Kaempf wrote:

I scanned all files below /usr/share/YaST2/{modules,clients,include} grepping for "SCR::". Just to get statistical relevant data, not for complete coverage.

The goal is to get relevant data for a SCR redesign to be implemented in parallel to the current SCR and have modules converted over time and shut down SCR is some distant future.

May be a dumb question, but which problems are we trying to fix? Also, are you suggesting a completely different SCR(-like) approach or just "fixed" and more unified paths? Thanks, Michal -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org

-------------------------------------------------- From: "Klaus Kaempf" Sent: Thursday, November 15, 2007 12:35 PM To: "Michal Svec" Cc: "YaST" Subject: Re: [yast-devel] SCR usage statistics.

* Michal Svec [Nov 15. 2007 18:05]:

...

On Thu, 15 Nov 2007, Klaus Kaempf wrote:

...

I scanned all files below /usr/share/YaST2/{modules,clients,include} grepping for "SCR::". Just to get statistical relevant data, not for complete coverage.

The goal is to get relevant data for a SCR redesign to be implemented in parallel to the current SCR and have modules converted over time and shut down SCR is some distant future.

May be a dumb question, but which problems are we trying to fix?

I see sevaral problems

- Code cleanup / refactoring SCR is not fully understood by developers (coding all kinds of workarounds) and incomplete in some regards (developers parsing files in ycp). This should be fixed.

- No / Inconsistent modeling The pathes and data structures used in SCR effectively define a model. There is no clear documentation nor consistency in this model. Should we make the SCR model consistent or pick up an existing one (i.e. HAL for hardware probing or CIM for everything)

- Reuse of existing code For historical reasons (non-GPL license of YaST), SCR agents don't use existing implementations for instrumentation. For Code11, we have the requirement of 'friendly coexistance' with CIM instrumentation. A resonable combination would be to replace agents with existing CIM provider code.

- Separation of rights Security and 'role based' considerations recommend to separate the controller code (WFM in YaST) from the backend (SCR in YaST). The controller code should run without root rights, the backend code should assume root rights on demand (i.e. rpm -qa does not need root). This requires a redesign of the SCR backend.

...

Also, are you suggesting a completely different SCR(-like) approach or just "fixed" and more unified paths?

Thats what I'm trying to find out and get more input from developers.

The current thinking is to implement a completely different approach while keeping the current SCR in place and migrate modules one-by-one.

Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg)

--

From experience, SCRs are just plain hard to use unless you are a rocket scientist. Writing and consuming SCRs (or their potential replacement) should be a lot easier and more "fun".

-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org

Klaus Kaempf napsal(a):

* Justin Haygood [Nov 15. 2007 19:06]:

...

From experience, SCRs are just plain hard to use unless you are a rocket scientist. Writing and consuming SCRs (or their potential replacement) should be a lot easier and more "fun".

Agreed !

And I forgot one more point in my list:

- Object orientation The 'backend api' should pick up properties of object-orientation. Examples:

Old New --- --- value = SCR::Read( .foo.bar ) -> value = client.foo.bar SCR::Write( .foo.bar, value ) -> client.foo.bar = value SCR::Execute( .foo.bar, v1, v2) -> client.foo.bar( v1, v2 )

I think this fulfills the 'easy' and 'fun' requirements ;-)

Sounds great! Nevertheless I don't see *any* benefit when comparing to the current implementation. What I see are these drawbacks: * Broken backward compatibility - and we know we need backward compatibility in such important tasks as access to system is. We just can't rewrite the whole YaST overnight (OES, SLEPOS, ...). * Confuses developers - currently we have clearly written: 'Read', 'Write', 'Execute' or 'Dir'. The proposal of replacement for SCR::Read vs. SCR::Execute is almost the same: any cmd_out = client.foo.bar ( v1 ) any value = client.foo.bar any cmd_out = SCR::Execute (.foo.bar, v1) any value = SCR::Read (.foo.bar) * It's not a fun to rewrite all the existing code :) ;) and it's not easy either! ;) The current SCR implementation expects caching the changed values, all are written to disk at once to make it faster. I don't see any clear /object-oriented/ syntax to rewrite it to be still understandable. Anyway, if we are talking about SCR -> 'client', WFM would need to be 'system' or something similar. Which, on the other hand, would bring another confusedness. IMO: All in all, this is not worth changing. But please, convince me it is. Bye Lukas PS: The current documentation of SCR agents is generated from sources for every release. See these pages, for instance: SCR: http://forgeftp.novell.com/yast/doc/SL10.3/scr/index.html http://forgeftp.novell.com/yast/doc/SL10.3/scr/136.root.wgetrc.html http://forgeftp.novell.com/yast/doc/SL10.3/scr/152.sysconfig.SuSEfirewall2.h... SCR how-to document: http://forgeftp.novell.com/yast/doc/SL10.3/tdg/documentation_SCR.html -- Lukas Ocilka, YaST Developer (xn--luk-gla45d) ----------------------------------------------------------------- SUSE LINUX, s. r. o., Lihovarska 1060/12, Praha 9, Czech Republic

I agree with Klaus here, one thing is about the CIM and WBEM tools and the other is reinventing our own model. The CIM model is quite complete, and we should try to come up with something following its philosophy. About SCR, yes, creating agents is not something intuitive, specially if it is not a simple inherited ini file agent, but a full YaST component. After all the interesting we discussed with Klaus during the week, I have been thinking a lot into all those problems, and started looking for solutions. For example, playing with a little of Ruby sugar, I managed to replicate SCR in a interesting way: Lets see how do I define an agent: Plugin.define "release_agent" do author "Duncan" version "1.0.0" extends :agent run_before :other def read(path) if path == :'.system.product.edition' f = File.open('/etc/SuSE-release', 'r') f.each_line do |l| return l end return "" end end def supports_path?(path) return path == :'.system.product.edition' end end puts SCR::read(:'.system.product.edition') done. Why is it so hard with our tools, because we reinvented everything. (Note, the example implementation of a SCR engine is 26 lines of code, not shown above) Note: the above example is even compatible with the old scr, as any path is passed down to the old SCR. Require the wbem module and you can add wbem access, require drb and you can call distributed code. Now, our main problem is that we have to interact with quite a lot of code in different languages, ages, paradigms. We have our own component model, but it is quite complex. We could go our way and implement all the YaST basic core in Ruby or Python and add nice stuff like WBEM access, access to services in the web, etc, but we need a nice way to reuse and use functionality from C/C++ code, Python, Perl, etc, and the other way around. Especially if we are dealing with the system, you don't want the developers to learn how to create ruby extensions or python modules just to interface with some low level library. We depend or will depend on that (if we plan to ditch everything inhouse that is a reinvention). We have that component model right now (liby2) but try to create a component yourself, isn't that the reason why libstorage uses generated perl swig bindings, to interact to the component model via the super complex perl-bindings. I was giving a look at solutions out there. One I found was OASIS Service Component Architecture / C and C++ (SCA-C-C++) which has a implementation by Apache called Tuscany, already in the 1.0 Milestone3. Its a service model, more oriented to web services but supporting local components. The approach is pretty simple, a service in c++ is just a .h defining the interface, the implementation, and a wsdl file. You can implement them in python and ruby (services and clients) and that is supported by the runtime out of the box. There is an article at IBM DeveloperWorks [1]. Going back to SCR. I don't think the syntax sugar is very important, but how we use existing knowledge there. The CIM model is a place to start, but there are also issues that are not intuitive. The CIM model is just a OO way of looking the system, where you ask for instances of a class, and then use the objects properties and methods. If you ask the CIMOM for the instances of CIM_UnixFile, you get the root tree /, in my experiments, I had no clue how to go forward and do _SIMPLE_ stuff like reading, globing, etc. (not to mention it took 10 seconds to read 10 directories in / but probably it was ruby::wbem fault). However, stuff like getting the numbers of CPU, the installed products, etc. CIM provides a model much more consistent than the low level greps of /proc and the package bindings, but some models like accessing files are not nice. (An argument could be that we should not access files, but I am not sure about that). The big question when prototyping all the ideas floating out there is, how to move on. Now there is some momentum, and the right time for other pieces (for example, the great YaST UI being decoupled). If anyone has a concrete experiment to try next, speak on! :-) Duncan [1]: http://www.ibm.com/developerworks/webservices/library/ws-soa-capisca1/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org

On Friday 16 November 2007 10:11:49 am Lukas Ocilka wrote:

Nevertheless I don't see *any* benefit when comparing to the current implementation. What I see are these drawbacks:

I agree, we should not focus on the syntax sugar, but on the model.

* Broken backward compatibility - and we know we need backward compatibility in such important tasks as access to system is. We just can't rewrite the whole YaST overnight (OES, SLEPOS, ...).

Yes, we can. Released products use released version of YaST, that we have to maintain, but we have to maintain it anyways as much as we maintain the old package manager library which is a different approach as SLES. We should not. Rewrites aren't good ( http://www.joelonsoftware.com/articles/fog0000000069.html ) but one thing is renewing our platform, the other is rewriting. You can write new ways to use the platform, and reuse old code (like SCR agents for example).

* It's not a fun to rewrite all the existing code :) ;) and it's not easy either! ;)

We don't want to rewrite. But we don't want to rethink anything too. We have human barriers in the product. We, the developers, are so used our smelly bunch of tools and base technology, that we refuse to think on a platform other people would like to use too. Can we enable other to use YaST and at the same time keep our users saying that YaST rock? I think we can.

The current SCR implementation expects caching the changed values, all are written to disk at once to make it faster. I don't see any clear /object-oriented/ syntax to rewrite it to be still understandable.

I don't consider the syntax sugar important at this stage, and you are right, sometimes adding OO without reason can transform something simple into something verbose. I see the OO power more important in the agent development (see my other post).

Anyway, if we are talking about SCR -> 'client', WFM would need to be 'system' or something similar. Which, on the other hand, would bring another confusedness.

Lets try to brainstorm this in the other way, instead of finding out why something will not work, lets try to find a _different_ approach that would work.

IMO: All in all, this is not worth changing. But please, convince me it is.

Just try the exercise of trying to explain somebody how to create an agent which is not based on the sysconfig or ini one.

PS: The current documentation of SCR agents is generated from sources for every release. See these pages, for instance:

SCR: http://forgeftp.novell.com/yast/doc/SL10.3/scr/index.html http://forgeftp.novell.com/yast/doc/SL10.3/scr/136.root.wgetrc.html http://forgeftp.novell.com/yast/doc/SL10.3/scr/152.sysconfig.SuSEfirewall2. html

SCR how-to document: http://forgeftp.novell.com/yast/doc/SL10.3/tdg/documentation_SCR.html

You see that every agent documentation is one line, and how to create an agent is a copy paste of a ini based agent, which confirms my theory that we are unable to explain humans how to do anything more complex than that :-) Duncan -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org

Dne Thursday 15 of November 2007 18:35:23 Klaus Kaempf napsal(a):

* Michal Svec [Nov 15. 2007 18:05]:

...

On Thu, 15 Nov 2007, Klaus Kaempf wrote:

...

I scanned all files below /usr/share/YaST2/{modules,clients,include} grepping for "SCR::". Just to get statistical relevant data, not for complete coverage.

The goal is to get relevant data for a SCR redesign to be implemented in parallel to the current SCR and have modules converted over time and shut down SCR is some distant future.

May be a dumb question, but which problems are we trying to fix?

I see sevaral problems

- Code cleanup / refactoring SCR is not fully understood by developers (coding all kinds of workarounds) and incomplete in some regards (developers parsing files in ycp). This should be fixed.

- No / Inconsistent modeling The pathes and data structures used in SCR effectively define a model. There is no clear documentation nor consistency in this model. Should we make the SCR model consistent or pick up an existing one (i.e. HAL for hardware probing or CIM for everything)

- Reuse of existing code For historical reasons (non-GPL license of YaST), SCR agents don't use existing implementations for instrumentation. For Code11, we have the requirement of 'friendly coexistance' with CIM instrumentation. A resonable combination would be to replace agents with existing CIM provider code.

- Separation of rights Security and 'role based' considerations recommend to separate the controller code (WFM in YaST) from the backend (SCR in YaST). The controller code should run without root rights, the backend code should assume root rights on demand (i.e. rpm -qa does not need root). This requires a redesign of the SCR backend.

...

Also, are you suggesting a completely different SCR(-like) approach or just "fixed" and more unified paths?

Thats what I'm trying to find out and get more input from developers.

The current thinking is to implement a completely different approach while keeping the current SCR in place and migrate modules one-by-one.

This is something which we already tried with LiMaL. The actual result is that we have two "standard" ways to access system. If we decide for any new approach, we should decide for an existing standard. With the requirement for code reuse, CIM seems to me to be the way to go provided we get reasonably over the coding overhead. Jiri -- Regards, Jiri Srain YaST Team Leader --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: jsrain@suse.cz Lihovarska 1060/12 tel: +420 284 028 959 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz

* Ralf Haferkamp [Nov 16. 2007 11:29]:

One of LiMaL's original goals was to make it a layer below CIM, and other higher level management tools (YaST/SCR). Maybe you can think of it as a convenience library to ease the creation of CIM providers while additionally allowing to manage the (local) system without CIM.

Agreed.

But IMO using CIM everywhere will create a huge overhead on a local system.

Please don't confuse CIM (the model) with WBEM (talking to a CIMOM). The overhead of the latter should be avoided. Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org