* Michael Andres [Sep 02. 2011 10:49]:

We can also work on the zypp solvers API side. AFAIK the solver is basically able to resolve requests like 'install one out of a set of packages'. There's just no zypp side api for this.

The complete solver api, including policies, is available in the satsolver bindings. Exposing these in the libzypp bindings should be fairly simple and also prevent from re-inventing the wheel. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Michael Andres [Sep 02. 2011 13:30]:

On Friday 02 September 2011 10:53:30 Klaus Kaempf wrote:

...

* Michael Andres [Sep 02. 2011 10:49]:

...

We can also work on the zypp solvers API side. AFAIK the solver is basically able to resolve requests like 'install one out of a set of packages'. There's just no zypp side api for this.

The complete solver api, including policies, is available in the satsolver bindings. Exposing these in the libzypp bindings should be fairly simple and also prevent from re-inventing the wheel.

But libzypp does not grant unattended lowlevel access to the satsolver instance it is using.

Actually, I'd consider this a bug. I imagine libzypp as a collection of modules using a common base for datastructures and API design. The modules are repo mgmt, solver, package download, and transaction commit. Libzypp should not try to reinvent the wheel or hide underlying implementations only where it provides real value. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Duncan Mac-Vicar P. [Sep 20. 2011 14:19]:

I am quite surprised that you find this a bug, when you had to do exactly the same once and invent xsolvable. ZYpp just took a different path and prefered to manage one pool instead of adding one Pool pointer to each solvable.

The difference is that satsolvevr-bindings allows direct access to the underlying satsolver while libzypp does not. At least, thats my understanding.

If the sat-solver would offer a string pool shared across multiple pools, it would be cool to get rid of global objects, and IdString could have only a global string pool.

Whats the use-case for multiple pools ?

The API we have been discussing tries to hide the fact that there are global objects. Even if they are, you use your own handler.

Then we're in complete agreement as I'm not aware of any global objects used in satsolver-bindings. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

On Tuesday 20 September 2011 15:04:47 Klaus Kaempf wrote:

Oh, I forgot to mention one important property of the xsolvable approach. It gives you the flexibility to create solvables and add them to the pool during runtime. IIRC we dicussed creating patches in libzypp at runtime and the libzypp approach (resulting in a static pool) prevented that.

? The pool is not static, it's global. I.e. we use a single pool, but you are able to change it at runtime. We do this e,g, after commit when rereading the rpm database. Though the public API is focused on loading solv or helix files, you could also inject single solvables. If we have a usecase for this we can also polish the API. The problems will most probably arise at the UI, not in libzypp. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres SUSE LINUX Products GmbH, Development, ma@suse.de GF:Jeff Hawn,Jennifer Guild,Felix Imendörffer, HRB16746(AG Nürnberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

On Monday 05 September 2011 11:19:16 Duncan Mac-Vicar P. wrote:

- A C task-oriented API would allow zypper, YaST and PackageKit to deduplicate code  and if the API is good, nice bindings.

zypper, YaST and PackageKit are actuallty C++ consumers, so it might make sense to design the API also as simple C++ API which should be straight forward to wrap to C: class Foo { Foo(); ~Foo(); int method( const char * arg ); int method( std::string arg ) { return method( arg.c_str() ); } int method( Capability arg ) { return method( arg.c_str() ); } }; ---- Foo* foo_init() { return new Foo; } void foo_delete(Foo* p) { delete p; } int foo_method( Foo*p, const char * arg ) { return p->method( arg ); } [ int foo_method_2( Foo*p, Capability * arg ); ] Within libzypp many stringbased types like Capability, Arch, Edition, .. exist. Most of them are just an index into the satsolvers string pool and the methods just wrap corresponding calls in libsatsolver. But it's quite convenient to have them as different types if you have a compiler that checks. Another - more important - reason for wrapping almost all satsolver 'classes' in libzypp (C++) is the need to write exception safe code. Each resource allocation (foo_init) which requires a corresponding release action (foo_delete) will have to be wrapped into a class to prevent resource leaks if exeptions were thrown. We probably do not want zypper, YaST and PackageKit to wrap back the C task-oriented API into C++. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres SUSE LINUX Products GmbH, Development, ma@suse.de GF:Jeff Hawn,Jennifer Guild,Felix Imendörffer, HRB16746(AG Nürnberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Duncan Mac-Vicar P. [Sep 05. 2011 11:19]:

On 09/02/2011 10:53 AM, Klaus Kaempf wrote:

...

* Michael Andres [Sep 02. 2011 10:49]:

...

We can also work on the zypp solvers API side. AFAIK the solver is basically able to resolve requests like 'install one out of a set of packages'. There's just no zypp side api for this.

The complete solver api, including policies, is available in the satsolver bindings. Exposing these in the libzypp bindings should be fairly simple and also prevent from re-inventing the wheel.

While it would be cool, it has the following problems:

- satsolver is not different to libzypp. Is not a task oriented API but a model of the problem domain. Not to mention it does 30% of the problem

You have looked at the satsolver-bindings and the example code ? Can you provide an example to make it even more task-oriented ?

- As Michael mentioned, libzypp does not allow raw manipulation via the satsolver API on one side and then using libzypp on the other side See my previous mail. I think this is wrong.

- Almost every client tool where we want to reduce code duplication is not using any scripting language. We want to offer bindings for _future_ tools.

Shouldn't future tools be implemented in a scripting language ? The porcellain/plumbing model, e.g. git is using, seems very desirable to me.

- While moving away from libzypp sounds like simplifying the problem. I think is just an instance of a known mistake: http://www.joelonsoftware.com/articles/fog0000000069.html

Is anyone proposing to move away from libzypp ?

I think that having functional code in the bindings is re-inventing the wheel.

Can you give an example for this concern ?

Any code not using scripting languages would need to rewrite that code.

Which code not using scriting language are you envisioning ?

I experimented during the weeked with the small API (zpm_query_what_provides and zpm_query_search) using 2 different approaches:

- Using libzypp, and hiding its classes into this light swig friendly API - Using satsolver only. Reusing code by mls in his example solv.c (parsing and download of repos, download of packages, etc).

This compares apples with oranges.

The feeling I got doing those experiments:

- A C task-oriented API would allow zypper, YaST and PackageKit to deduplicate code and if the API is good, nice bindings.

Ah, so you're targeting a C api here. Then using SWIG is certainly the wrong approach. However, I currently cannot imagine many consumers of a C API.

- ZYpp is a great way to implement this API. Using the plain satsolver was not bad at all, but is a very raw API

Thats why satsolver-bindings provide a (C language) 'applayer', making the satsolver API much more approachable.

- Implementing all the other requirements: memory management, selectables, powerful filters and search, key management, repository management, media abstraction, etc in plain C together with the satsolver (basically the experiment I did using mls solv.c code) is a step backwards.

Of course ! And noone is questioning that. The question is how to provide bindings for the objects shared between libzypp and libsatsolver, like 'Solvable', 'Solver', 'Problem', 'Solution', 'Transaction', 'Pool', 'Repo', etc. My proposal / offer is to use the API already existing in satsolver-bindings for the shared objects. OTOH, given sufficient time and resources a completely new approach is possible. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

On 09/20/2011 07:09 PM, Michael Andres wrote:

- In satsolver by convention all patch names start with 'patch:' and all pattern names with 'pattern:', and every name that has no known prefix is a package. Among all packages, source packages are identified by having architecture 'src' or 'nosrc'.

- Within zypp all solvable names are unprefixed. A solvable attribute named 'kind' exists, telling whether it's a package, patch, pattern... Sourcepackages are of kind 'srcpackage' and their architecture is 'noarch'. A 'src' nor 'nosrc' architecture does not exist within zypp.

Yup. And that is why I have never seen sat-solver as an "API". I understand that the bindings hide its drawbacks, but I am talking as a C API here (which we need). Prefixes, Id math with non-opaque structs, etc. Nothing for application developers. Great for building APIs on top of it. There is stuff that can be improved though. And the current draft of the ZYpp C api does indeed only offer one opaque ptr called "zypp_solvable". For the C++ part, I actually do like the object hierarchy, which is really useful for building user interfaces, which usually means holding base classes in ListViews. -- Duncan Mac-Vicar P. - http://www.suse.com/ SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Michael Andres [Sep 20. 2011 19:09]:

On Tuesday 20 September 2011 14:20:21 Klaus Kaempf wrote:

...

* Duncan Mac-Vicar P. [Sep 05. 2011 11:19]: The question is how to provide bindings for the objects shared between libzypp and libsatsolver, like 'Solvable', 'Solver', 'Problem', 'Solution', 'Transaction', 'Pool', 'Repo', etc.

But unfortunately libzypp is not an applayer on top of satsolver. You remember the "evolution" of zypp - we ripped out the old solver and plugged in a new one. At the same time the API towards the zypper, YaST and PK remained as unchanged as possible.

Most of the time this API prevents us from being more 'satsolverish'. The satsolver objects are implementation details, sometimes more, sometimes less exposed.

I couldn't agree more. And that's why satsolver-bindings do not just grab some satsolver include files and let SWIG generate bindings from this. Instead, the satsolver-bindings design starts from the problem domain (package dependency solving) with sample code in the target language. It all revolves around "how should 'dependency solving' look like when I use Perl/Python/Ruby". Satsolver internals are exposed where it adds value (i.e. to enable id comparison vs. string comparison) or where it doesn't alienate the target language.

But there are several contrains libzypp enforces and translations it applies and which prevent us from offering unattended access to the satsolver objects.

That's fine and needs to be documented. However, adding another layer of abstraction increases the effort for the developer and the consumer of bindings.

Zypp and Sat sometimes speak a different language.

They shouldn't. Zypp should extend the language of the problem domain instead of translating it to a different language.

A quite obvious difference is the solvable 'name' and the solvable 'architecture':

- In satsolver by convention all patch names start with 'patch:' and all pattern names with 'pattern:', and every name that has no known prefix is a package. Among all packages, source packages are identified by having architecture 'src' or 'nosrc'.

- Within zypp all solvable names are unprefixed. A solvable attribute named 'kind' exists, telling whether it's a package, patch, pattern... Sourcepackages are of kind 'srcpackage' and their architecture is 'noarch'. A 'src' nor 'nosrc' architecture does not exist within zypp.

And its the task of the bindings to hide such implementation details.

Also the tight linkage between a 'product' and it's -release package is something which is asserted by zypp and must not be broken by direct satsolver access.

Its a question of effort and resources if libzypp should completely prevent this or add a simple "you might shoot yourself in the foot" warning. ;-)

I'm not sure whether common bindings are a low hanging fruit.

The alternative seems to be a refactored libzypp, a useful C API, and new bindings on top. Looks like a lot of work to me. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Duncan Mac-Vicar P. [Sep 21. 2011 15:19]:

On 09/21/2011 09:30 AM, Klaus Kaempf wrote:

...

The alternative seems to be a refactored libzypp, a useful C API, and new bindings on top. Looks like a lot of work to me.

What is the other alternative?

Refactor and adapt whats already existing. Make libzypp an extension of the underlying solver library, both from an API and a bindings POV. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org