[zypp-devel] SAT: speedup zypper lu
Hi,
the same problem as with dist upgrade also hits with zypper lu, it does a
linear walk over all poolitems, for each calling the ByName iterator,
which in turn also walks over the entire pool. Not So Clever, except if
you want to test how the machine behaves with O(N^2) algorithms :)
This of course could be fixed in libzypp by building a name->poolitem
index itself and use it in the ByName iterator. But in the old libzypp a
considerable amount of space and time was used for just building several
dozen indices, and ideally we don't repeat this again. For one or two
ByName filters (e.g. when done in reaction to a user action) such index is
not necessary.
In this case of course it is. The question is, if we want to support such
use case in libzypp nicely, and if yes in which way. We could add an
ByNameFast filter which builds an index, or just rely on the callers being
fixed (like I do here).
What the SAT library can provide without much overhead is an index for a
ByKindName filter (doesn't exist yet), so _that_ one could be made fast
without taking many resources. Interestingly that happens to be the
usecase of zypper too.
Anyway, for the time being for people testing zypper of the sat branch,
this patch makes one usecase (which happens to be my primary performance
testcase) fast.
Ciao,
Michael.
Index: zypper-misc.cc
===================================================================
--- zypper-misc.cc (revision 8599)
+++ zypper-misc.cc (working copy)
@@ -1,6 +1,7 @@
#include <fstream>
#include <sstream>
#include
* Michael Matz
Hi,
the same problem as with dist upgrade also hits with zypper lu, it does a linear walk over all poolitems, for each calling the ByName iterator, which in turn also walks over the entire pool. Not So Clever, except if you want to test how the machine behaves with O(N^2) algorithms :)
Huh ? libzypp (the old one ;-)) created a couple of maps for the solver and should've used those.
What the SAT library can provide without much overhead is an index for a ByKindName filter (doesn't exist yet), so _that_ one could be made fast without taking many resources. Interestingly that happens to be the usecase of zypper too.
My current thinking (to be implemented during hackweek ...) is to separate different kinds into different repos. So the solver doesn't ever see kinds, but it needs the ability to restrict solving to a set of repos. Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Hi, On Mon, 11 Feb 2008, Klaus Kaempf wrote:
* Michael Matz
[Feb 11. 2008 08:49]: Hi,
the same problem as with dist upgrade also hits with zypper lu, it does a linear walk over all poolitems, for each calling the ByName iterator, which in turn also walks over the entire pool. Not So Clever, except if you want to test how the machine behaves with O(N^2) algorithms :)
Huh ? libzypp (the old one ;-)) created a couple of maps for the solver and should've used those.
Now, guess why I prefixed my Subject with "SAT" :-) I was speaking only about the sat branch (everything else is Schnee von gestern :) )
My current thinking (to be implemented during hackweek ...) is to separate different kinds into different repos. So the solver doesn't ever see kinds, but it needs the ability to restrict solving to a set of repos.
I don't see the value in really forcing diffenrent kinds into different repos, _if_ we have a possibility to solve on a subset of packages. It's sort of convenient if e.g. patches and packages are in the same solv file. Ciao, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
* Michael Matz
I don't see the value in really forcing diffenrent kinds into different repos, _if_ we have a possibility to solve on a subset of packages. It's sort of convenient if e.g. patches and packages are in the same solv file.
As long as we can separate patches from packages, I don't really care where they're stored ;-) Klaus -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Mon, Feb 11, Michael Matz wrote:
you want to test how the machine behaves with O(N^2) algorithms :)
This of course could be fixed in libzypp by building a name->poolitem index itself and use it in the ByName iterator. But in the old libzypp a considerable amount of space and time was used for just building several dozen indices, and ideally we don't repeat this again. For one or two ByName filters (e.g. when done in reaction to a user action) such index is not necessary.
In most cases the applications use the byName iterator and filter by kind (or vice versa). This was ok as we mainatined a name index. But ideally we'll provide (and the the applications should use) a combined (kind,name) iterator. Unlike byName, we can process by(kind,name) without any string operations. (libsatsolver does not provide individual name and kind values). And if it's still to slow, we can index it. by(kind,name) is more frequently used than the individual ones. (e.g. to build a selectable). -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (3)
-
Klaus Kaempf
-
Michael Andres
-
Michael Matz