[zypp-devel] shutting down zypp (Fwd)

Hi, Benji has asked me about the zypp lock: "Product.ycp obtains a lock on the package database, and there doesn't seem to be a way to release it, I've worked around the problem for now." So I remembered the old discussion about the lock and I'm forwarding it from the old closed list. ----- Forwarded message from Martin Vidner ----- Date: Thu, 30 Mar 2006 13:24:45 +0200 From: Martin Vidner To: zypp-devel@suse.de User-Agent: Mutt/1.5.9i Subject: [zypp-devel] shutting down zypp Hi, during the installation, suse_register is called form yast and it calls zmd to manipulate whatever. Therefore we need to let go of zypp. We introduced ZYpp::reset and SourceManager::reset for this on Mar 3. Now, I also need to release the lock. One part of doing it is allocating the lock object dynamically. But it also makes me think that the reset interface needs to be fixed. Like, why is a separate SourceManager::reset needed at all? Why not have it all encapsulated in ZYpp and simply destroy the main interface object? I do not know the library well enough to change it myself. Please help or guide me. -- Martin Vidner, YaST developer SuSE CR, s.r.o. e-mail: mvidner@suse.cz Drahobejlova 27 tel:+420-296542390 190 00 Praha 9, Czech Republic http://www.suse.cz ----- End forwarded message ----- ----- Forwarded message from Klaus Kaempf ----- Date: Thu, 30 Mar 2006 14:26:19 +0200 From: Klaus Kaempf To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp Cc: * Martin Vidner [Mar 30. 2006 13:24]:

Hi,

during the installation, suse_register is called form yast and it calls zmd to manipulate whatever. Therefore we need to let go of zypp. We introduced ZYpp::reset and SourceManager::reset for this on Mar 3.

Magically, this works in the current builds. I do see a "can't aquire lock" in the first call to a libzypp-zmd-backend helper from zmd but the next one succeeds. The initial failure of lock aquisition is probably due to an (too) early start of zmd.

Now, I also need to release the lock. One part of doing it is allocating the lock object dynamically. But it also makes me think that the reset interface needs to be fixed. Like, why is a separate SourceManager::reset needed at all? Why not have it all encapsulated in ZYpp and simply destroy the main interface object?

Right. I also don't see the need for a separate, public SourceManager::reset().

I do not know the library well enough to change it myself. Please help or guide me.

Michael, this is probably your call ;-) Klaus ----- End forwarded message ----- ----- Forwarded message from Martin Vidner ----- Date: Thu, 30 Mar 2006 14:31:12 +0200 From: Martin Vidner To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp User-Agent: Mutt/1.5.9i On Thu, Mar 30, 2006 at 02:26:19PM +0200, Klaus Kaempf wrote:

* Martin Vidner [Mar 30. 2006 13:24]:

...

Hi,

during the installation, suse_register is called form yast and it calls zmd to manipulate whatever. Therefore we need to let go of zypp. We introduced ZYpp::reset and SourceManager::reset for this on Mar 3.

Magically, this works in the current builds.

I do see a "can't aquire lock" in the first call to a libzypp-zmd-backend helper from zmd but the next one succeeds.

The initial failure of lock aquisition is probably due to an (too) early start of zmd.

Possibly, but it may be that you're seeing a bug (no buzgilla) where the lock was deleted even when not owned. Duncan fixed it in r2749 yesterday at 2 pm. -- Martin Vidner, YaST developer SuSE CR, s.r.o. e-mail: mvidner@suse.cz Drahobejlova 27 tel:+420-296542390 190 00 Praha 9, Czech Republic http://www.suse.cz ----- End forwarded message ----- ----- Forwarded message from Michael Andres ----- Date: Thu, 30 Mar 2006 16:23:20 +0200 From: Michael Andres To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp Organization: SuSE GmbH, Nuernberg, Germany User-Agent: Mutt/1.5.9i On Thu, Mar 30, Martin Vidner wrote:

Hi,

during the installation, suse_register is called form yast and it calls zmd to manipulate whatever. Therefore we need to let go of zypp. We introduced ZYpp::reset and SourceManager::reset for this on Mar 3.

reset will introduce more problems, than it solves, because we loose information (e.g. about availbale languages). And every change we may apply to zypp will require someone to remember to check and think about how to handle it on reset. The correct way would be to destroy the ZYpp object. And recreate a new one later. Both via ZYppFactory. No ZYpp object, no lock. -- cu, Michael (ma@suse.de) +------------------------------------------------------------------+ ----- End forwarded message ----- ----- Forwarded message from Martin Vidner ----- Date: Thu, 30 Mar 2006 17:52:21 +0200 From: Martin Vidner To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp User-Agent: Mutt/1.5.9i On Thu, Mar 30, 2006 at 04:23:20PM +0200, Michael Andres wrote:

The correct way would be to destroy the ZYpp object. And recreate a new one later. Both via ZYppFactory. No ZYpp object, no lock.

OK. So, I should - create ZYppFactory::killZYpp () { _instance = 0; } - in pkg-bindings, call it and assign 0 also to the pkg-binding copy of ZYpp::Ptr - make sure I cleared all other references, otherwise zypp survives - move the unlocking from ~ZYppGlobalLock to ~ZYpp. For debugging, how can I ask ZYpp::Ptr how many references it has? -- Martin Vidner, YaST developer SuSE CR, s.r.o. e-mail: mvidner@suse.cz Drahobejlova 27 tel:+420-296542390 190 00 Praha 9, Czech Republic http://www.suse.cz ----- End forwarded message ----- ----- Forwarded message from Michael Andres ----- Date: Thu, 30 Mar 2006 22:41:17 +0200 From: Michael Andres To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp Organization: SuSE GmbH, Nuernberg, Germany User-Agent: Mutt/1.5.9i On Thu, Mar 30, Martin Vidner wrote:

On Thu, Mar 30, 2006 at 04:23:20PM +0200, Michael Andres wrote:

...

The correct way would be to destroy the ZYpp object. And recreate a new one later. Both via ZYppFactory. No ZYpp object, no lock.

OK. So, I should - create ZYppFactory::killZYpp () { _instance = 0; }

No 'kill', call it release.

- in pkg-bindings, call it and assign 0 also to the pkg-binding copy of ZYpp::Ptr - make sure I cleared all other references, otherwise zypp survives - move the unlocking from ~ZYppGlobalLock to ~ZYpp.

Despite all possible problems: ZYpp::Impl is the class of interest, not Zypp. ZYpp is just an interface, ZYpp::Impl is the object. And the one who applies the lock, has to release it, so either both in ZYppFactory or both in ZYpp::Impl. not mixed. But,.. No matter what you do, the solution will be weak. Due to the limited ammout of time, we had to drop the assumption, that ZYpp might not be available. Thus all classes that need ZYpp store a ZYpp::Ptr or call getZYpp (as having multiple ZYpp instances is another assumption we had to drop). AFIK no class is using ZYpp is realy prepared to face the fact, that the controling ZYpp might become unavailable or changes. But there is currently no way to unload ZYpp and the objects using it. Esp. Sources and Resolvables created will not go out of scope, even if ZYpp does. So even if no external reference (outside libzyp) to those objects exists, or is used after the ZYpp shutdown (what may or may not lead to abort), we will still have the memory problem. - release() should be removed from the public ZYpp interface (or at least be deprecated, as it does not work). - If release() is removed, ZYppFactory should store a ZYpp::Impl_Ptr instead of ZYpp::Ptr _instance. That way ZYppFactory is still able to class ZYpp::Impl::release(). (better than nothing) - In ZYppFactory::releaseZYpp, call ZYpp::Impl::release(), then set _instance to NULL, and unlock. Everything else is wasted time, as the problem is far larger than making ZYpp go out of scope. -- cu, Michael (ma@suse.de) +------------------------------------------------------------------+ ----- End forwarded message ----- ----- Forwarded message from Martin Vidner ----- Date: Fri, 31 Mar 2006 11:27:01 +0200 From: Martin Vidner To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp User-Agent: Mutt/1.5.9i I must admit that you got me more confused than I expected at first reading :-/ Here are just the obvious unclear points: On Thu, Mar 30, 2006 at 10:41:17PM +0200, Michael Andres wrote:

- release() should be removed from the public ZYpp interface (or at least be deprecated, as it does not work).

It it possible at all? It is inherited from zypp::base::ReferenceCounted

- If release() is removed, ZYppFactory should store a ZYpp::Impl_Ptr instead of ZYpp::Ptr _instance. That way ZYppFactory is still able to class ZYpp::Impl::release(). (better than nothing)

ZYpp::Impl::release() does not exist. I suppose it is because the impl is a shared_ptr instead of an intrusive_ptr.

- In ZYppFactory::releaseZYpp, call ZYpp::Impl::release(), then set _instance to NULL, and unlock.

Everything else is wasted time, as the problem is far larger than making ZYpp go out of scope.

-- Martin Vidner, YaST developer SuSE CR, s.r.o. e-mail: mvidner@suse.cz Drahobejlova 27 tel:+420-296542390 190 00 Praha 9, Czech Republic http://www.suse.cz ----- End forwarded message ----- ----- Forwarded message from Michael Andres ----- Date: Fri, 31 Mar 2006 13:01:28 +0200 From: Michael Andres To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp (different approach) Organization: SuSE GmbH, Nuernberg, Germany User-Agent: Mutt/1.5.9i On Fri, Mar 31, Martin Vidner wrote:

I must admit that you got me more confused than I expected at first reading :-/ Here are just the obvious unclear points:

We'll provide a special (otherwise deprecated) ZYppFactory call, that returns a ZYpp which must not be used for other purposes than read_only access to data (as it does not check the global lock). The helper needed for suse_register will use this ZYppFactory call. pkg-bindings need not reset/invalidate ZYpp during suse_register. You can proceed as if libzypp had a prefect locking. -- cu, Michael (ma@suse.de) +------------------------------------------------------------------+ ----- End forwarded message ----- ----- Forwarded message from Michael Andres ----- Date: Fri, 31 Mar 2006 13:02:47 +0200 From: Michael Andres To: zypp-devel@suse.de Subject: Re: [zypp-devel] shutting down zypp (different approach) Organization: SuSE GmbH, Nuernberg, Germany User-Agent: Mutt/1.5.9i On Fri, Mar 31, Michael Andres wrote:

On Fri, Mar 31, Martin Vidner wrote:

...

I must admit that you got me more confused than I expected at first reading :-/ Here are just the obvious unclear points:

We'll provide a special (otherwise deprecated) ZYppFactory call, that returns a ZYpp which must not be used for other purposes than read_only access to data (as it does not check the global lock).

The helper needed for suse_register will use this ZYppFactory call.

pkg-bindings need not reset/invalidate ZYpp during suse_register.

You can proceed as if libzypp had a prefect locking.

And ZYpp::reset will be removed from the interface (in case pkg-bindings already use it). -- cu, Michael (ma@suse.de) +------------------------------------------------------------------+ ----- End forwarded message ----- (afterwards the thread continued to discuss details of the read-only lock) -- Martin Vidner, YaST developer http://en.opensuse.org/User:Mvidner Kuracke oddeleni v restauraci je jako fekalni oddeleni v bazenu -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org