[zypp-devel] [REVIEW] granular inter process lock
Hi guys, I had this in my queue since Easter. to make it short, our current global lock sucks, and it will suck more when we get packagekit, zypper and yast running at the same time :-) My prototype has passed the two initial test cases. The idea is a scoped object, that is, to make it work you create it, and once it is out of scope, it releases the lock. Locks are by name. And you can create writer or readers mutex for a named region. The name you give to the mutex will determine which region it will protect, and if you are entering it as reader or writer, and the behavior: wait until free, wait with timeout, or abort if locked. The different behaviors are that for example some operations, like enabling or disabling repos are really short, but probably we don't want to wait for the rpm commit lock to end and abort immediately. So you use like this: process A, the pool creates a reader lock for the pool: InterProcessMutex mutex( "pool", Reader); The repomanager wants to rewrite the solv files, which would screw up if some process have the global pool (usually zypper for some seconds, or YaST while the user has it opened), so the repomanager could do: InterProcessMutex mutex( "pool", Writer, 10); Repomanager will not be able to write while the first mutex is on scope, and will sleep for a total of 10 seconds on 1 seconds intervals, and then will throw if the lock is still there. However if another process just want to query data from the solv files, like a second process accessing the global pool, it will take a second reader lock on "pool" which will coexist nicely with the first one. Reference counting, cleaning etc is done by fcntl locks, so the kernel takes care of most stuff. (it is a direct mapping of the named resources over locks on a file per-resource, plus cleaning an waiting logic) It is not correct, if we for example test a lock for not being locked, and then try to acquire a lock, and just in the middle another process take it first, what we do is we abort mission and re enter the loop to try to acquire again, so it works reasonable, but it does not guarantee order on serving the resource, which is reasonable for the use case I guess. There are some improvements suggested by Michael Andres already, specially in the code path, and I also need more test cases covering more combinations. But I post it here for review / comments. http://svn.opensuse.org/svn/zypp/trunk/libzypp/zypp/base/InterProcessMutex.h (and .cc ) Testcases: http://svn.opensuse.org/svn/zypp/trunk/libzypp/tests/zypp/base/InterProcessM... http://svn.opensuse.org/svn/zypp/trunk/libzypp/tests/zypp/base/InterProcessM... Cheers Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Duncan Mac-Vicar Prett wrote:
Hi guys, I had this in my queue since Easter. to make it short, our current global lock sucks, and it will suck more when we get packagekit, zypper and yast running at the same time :-)
+cim providers :O) Great stuff, i look forward to this working! jano -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Hi, On Thu, 27 Mar 2008, Duncan Mac-Vicar Prett wrote:
InterProcessMutex mutex( "pool", Reader);
The base is sound. I see problems with the hardcoding of paths to /var/run/zypp-$NAME.pid . Users can't write to /var/run. It would also be nice if the path was configurable like the other paths. Ciao, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Thu, Mar 27, Michael Matz wrote:
Hi,
On Thu, 27 Mar 2008, Duncan Mac-Vicar Prett wrote:
InterProcessMutex mutex( "pool", Reader);
The base is sound. I see problems with the hardcoding of paths to /var/run/zypp-$NAME.pid . Users can't write to /var/run. It would also be nice if the path was configurable like the other paths.
Esp. as we should't lock /var/run/zypp-$NAME.pid while we are committing to a target with root /mnt. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Thu, Mar 27, 2008 at 05:40:39PM +0100, Duncan Mac-Vicar Prett wrote:
Hi guys, I had this in my queue since Easter. to make it short, our current global lock sucks, and it will suck more when we get packagekit, zypper and yast running at the same time :-)
My prototype has passed the two initial test cases.
The idea is a scoped object, that is, to make it work you create it, and once it is out of scope, it releases the lock.
Locks are by name. And you can create writer or readers mutex for a named region. The name you give to the mutex will determine which region it will protect, and if you are entering it as reader or writer, and the behavior: wait until free, wait with timeout, or abort if locked.
How about some hierarchy between the locks? Is it needed? Can it be implemented? My idea is to have locks like "pool" "cache" "repos.d" all covered by "GLOBAL" or something like that. -- Martin Vidner, YaST developer http://en.opensuse.org/User:Mvidner Kuracke oddeleni v restauraci je jako fekalni oddeleni v bazenu -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Dňa Thursday 27 March 2008 17:40:39 Duncan Mac-Vicar Prett ste napísal:
Hi guys, I had this in my queue since Easter. to make it short, our current global lock sucks, and it will suck more when we get packagekit, zypper and yast running at the same time :-)
My prototype has passed the two initial test cases.
The idea is a scoped object, that is, to make it work you create it, and once it is out of scope, it releases the lock.
Locks are by name. And you can create writer or readers mutex for a named region. The name you give to the mutex will determine which region it will protect, and if you are entering it as reader or writer, and the behavior: wait until free, wait with timeout, or abort if locked.
We will need a list of pre-defined locks to avoid typos in names leading to races. Good stuff! Stano -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (6)
-
Duncan Mac-Vicar Prett
-
Jan Kupec
-
Martin Vidner
-
Michael Andres
-
Michael Matz
-
Stanislav Visnovsky