Mailinglist Archive: zypp-devel (75 mails)
| < Previous | Next > |
Re: [zypp-devel] zypper performance gone bad, bad, bad
- From: Michael Matz <matz@xxxxxxx>
- Date: Sat, 17 May 2008 00:27:13 +0200 (CEST)
- Message-id: <Pine.LNX.4.64.0805170009020.20583@xxxxxxxxxxxxx>
Hi,
On Fri, 16 May 2008, Michael Matz wrote:
I know coolos problem I think. He had a lock file written with the old
attribute names still, like:
kind: package
solvable_name: build
string_type: glob
When parsing this in, "kind" and "string_type" are parsed as normal
attributes to match, because they meanwhile are called "type" and
"match_type". This makes it a three-attribute query (where two of them
never will match anything because they are special attributes that aren't
recognized anymore). And queries with more than one attribute are not
matched by the dataiterator but by libzypp itself, which is _very_
inefficient (it basically retrieves _all_ attributes of all solvables and
tries to match them against the query).
To show you the difference, with my locks file using the old attribute
names:
% time zypper in fate
real 0m42.440s
user 0m41.127s
sys 0m1.204s
When I edit the locks file by hand to use the meanwhile correct attribute
names (but otherwise the same):
% time zypper in fate
real 0m2.161s
user 0m1.312s
sys 0m0.252s
So, coolo: do this on your locks file:
sed -e 's/^kind:/type:/' -e 's/^string_type:/match_type:/'
Josef: it would be worthwhile I think to still recognize and parse the old
names (without exposing them in the API) to avoid this problem with lock
files written in the intermediate form.
Ciao,
Michael.
--
To unsubscribe, e-mail: zypp-devel+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: zypp-devel+help@xxxxxxxxxxxx
On Fri, 16 May 2008, Michael Matz wrote:
Nah. The problem seems to be that right now PoolQuery wants to match
strings on it's own instead of letting it be done by the DataIterator,
so it has to iterate over all solvables itself, and somewhere there must
be a bug that leads to actually viting all solvables again leading to
quadraticness. There's clearly something going very wrong in PoolQuerys
use of the DataIterator, I see it in gdb.
I know coolos problem I think. He had a lock file written with the old
attribute names still, like:
kind: package
solvable_name: build
string_type: glob
When parsing this in, "kind" and "string_type" are parsed as normal
attributes to match, because they meanwhile are called "type" and
"match_type". This makes it a three-attribute query (where two of them
never will match anything because they are special attributes that aren't
recognized anymore). And queries with more than one attribute are not
matched by the dataiterator but by libzypp itself, which is _very_
inefficient (it basically retrieves _all_ attributes of all solvables and
tries to match them against the query).
To show you the difference, with my locks file using the old attribute
names:
% time zypper in fate
real 0m42.440s
user 0m41.127s
sys 0m1.204s
When I edit the locks file by hand to use the meanwhile correct attribute
names (but otherwise the same):
% time zypper in fate
real 0m2.161s
user 0m1.312s
sys 0m0.252s
So, coolo: do this on your locks file:
sed -e 's/^kind:/type:/' -e 's/^string_type:/match_type:/'
Josef: it would be worthwhile I think to still recognize and parse the old
names (without exposing them in the API) to avoid this problem with lock
files written in the intermediate form.
Ciao,
Michael.
--
To unsubscribe, e-mail: zypp-devel+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: zypp-devel+help@xxxxxxxxxxxx
| < Previous | Next > |