[zypp-devel] Some numbers of the ported zypper
dmacvicar@piscola:/space/build/zypp/trunk/zypper> time sudo src/zypper sl # | Enabled | Refresh | Type | Name | URI --+---------+---------+--------+----------------+------------------------------------------------------------------------ 1 | No | No | rpm-md | filesharing | http://software.opensuse.org/download/filesharing/SUSE_Factory/ 1 | Yes | No | rpm-md | home:dmacvicar | http://software.opensuse.org/download/home:/dmacvicar/openSUSE_Factory/ 1 | Yes | No | rpm-md | ruby | http://software.opensuse.org/download/ruby/openSUSE_Factory/ real 0m0.034s user 0m0.028s sys 0m0.008s dmacvicar@piscola:/space/build/zypp/trunk/zypper> time sudo src/zypper -v --non-interactive install kopete-floor Verbosity 1 Entering non-interactive mode. Initializing Target loading repo resolvables... DONE \ RPM database[100%] Establishing status of aggregates Resolving dependencies... Summary: <install> [S0:0][package]kopete-floor-0.1.0-17.2.i586 Continue? [y/n] y committing Downloading: [S0:0][package]kopete-floor-0.1.0-17.2.i586, -1 B(-1 B unpacked) Downloading: http://software.opensuse.org/download/home:/dmacvicar/openSUSE_Factory/i586/... - Downloading[100%] Installing: [S0:0][package]kopete-floor-0.1.0-17.2.i586 | Installing [100%] CommitResult 1 (errors 0, remaining 0, srcremaining 0) real 0m8.925s user 0m7.880s sys 0m0.688s Note: from the 8 seconds, mostly is reading the rpm database. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
* Duncan Mac-Vicar Prett <dmacvicar@suse.de> [Jun 19. 2007 18:04]:
real 0m8.925s user 0m7.880s sys 0m0.688s
Note: from the 8 seconds, mostly is reading the rpm database.
Looks like the refactoring effort is a huge success, congratulations !! Can we do reading the rpm database on demand (if changed externally) only ? Klaus -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wednesday 20 June 2007 11:17:29 Klaus Kaempf wrote:
Can we do reading the rpm database on demand (if changed externally) only ?
You need to read it at least once to add the resolvables in the pool. -- Duncan Mac-Vicar Prett Novell :: SUSE R&D, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
* Duncan Mac-Vicar Prett <dmacvicar@suse.de> [Jun 20. 2007 12:27]:
On Wednesday 20 June 2007 11:17:29 Klaus Kaempf wrote:
Can we do reading the rpm database on demand (if changed externally) only ?
You need to read it at least once to add the resolvables in the pool.
Which seems to take more time than reading resolvables from the (sqlite) cache. So how about 'parsing' (resp. caching) the rpm database to the same database all other resolvables are ? (Together with a time stamp or whatever is needed to detect changes). This way, only the 'refresh' part of libzypp needs to distinguish between repositories with metadata and the rpm database. Applications just init the Repositories (and Pool) without knowing the difference. Klaus -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Klaus Kaempf wrote:
* Duncan Mac-Vicar Prett <dmacvicar@suse.de> [Jun 20. 2007 12:27]:
On Wednesday 20 June 2007 11:17:29 Klaus Kaempf wrote:
Can we do reading the rpm database on demand (if changed externally) only ? You need to read it at least once to add the resolvables in the pool.
Which seems to take more time than reading resolvables from the (sqlite) cache.
So how about 'parsing' (resp. caching) the rpm database to the same database all other resolvables are ? (Together with a time stamp or whatever is needed to detect changes).
This would be the first step. And probably easy to do. Another step would be not just simply appending target resolvables to the database, but *merging* them with repository resolvables. That means marking those that match as installed and appending only those that don't match (are installed and are not found in repos). That would speed up search significantly (and maybe solving as well?). But this is something for which we don't have time for 10.3. But what about a feature request for 10.3++?
This way, only the 'refresh' part of libzypp needs to distinguish between repositories with metadata and the rpm database. Applications just init the Repositories (and Pool) without knowing the difference. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On 6/19/07, Duncan Mac-Vicar Prett <dmacvicar@suse.de> wrote:
real 0m8.925s user 0m7.880s sys 0m0.688s
These results sound really great! Would it perform similarly if there was a lot more metadata to deal with (say, if Factory was added)? Kind thoughts, -- Francis Giannaros http://francis.giannaros.org -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Dňa St 20. Jún 2007 13:45 Francis Giannaros napísal:
On 6/19/07, Duncan Mac-Vicar Prett <dmacvicar@suse.de> wrote:
real 0m8.925s user 0m7.880s sys 0m0.688s
These results sound really great! Would it perform similarly if there was a lot more metadata to deal with (say, if Factory was added)?
This is using Factory, see the original mail's source configured. Stano -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Stanislav Visnovsky wrote:
Dňa St 20. Jún 2007 13:45 Francis Giannaros napísal:
On 6/19/07, Duncan Mac-Vicar Prett <dmacvicar@suse.de> wrote:
real 0m8.925s user 0m7.880s sys 0m0.688s These results sound really great! Would it perform similarly if there was a lot more metadata to deal with (say, if Factory was added)?
This is using Factory, see the original mail's source configured.
Not really. That was just ruby and filesharing subset (Duncan, right?) I'm getting 20+ secs for the whole factory (see below). Which is also good compared to 40+ like before. But there is still room for optimization. Although i can't estimate what we will make to 10.3. Jano $ ./zypper sl # | Enabled | Refresh | Type | Name | URI --+---------+---------+--------+--------------+---------------------------------------------------------------------------------- 1 | Yes | No | rpm-md | packman | http://packman.iu-bremen.de/suse/10.3/ 1 | No | No | yast2 | factory-yast | http://ftp.opensuse.org/pub/opensuse/distribution/SL-OSS-factory/inst-source 1 | Yes | No | rpm-md | factory-yum | http://ftp.opensuse.org/pub/opensuse/distribution/SL-OSS-factory/inst-source... $ time ./zypper search zypper S | Catalog | Type | Name | Version | Arch --+---------+---------+--------+------------+------- i | | patch | zypper | 2513-0 | noarch i | | atom | zypper | 0.6.15-0.1 | x86_64 i | | package | zypper | 0.7.1-3 | x86_64 v | | package | zypper | 0.7.2-7 | i586 v | | package | zypper | 0.7.2-7 | x86_64 real 0m24.721s user 0m22.817s sys 0m1.148s $ time ./zypper install zypper Reading RPM database... Summary: <uninstall> [S0:0][package]yast2-perl-bindings-2.14.0-33.x86_64 <uninstall> [S0:0][package]zypper-0.7.1-3.x86_64 <uninstall> [S0:0][package]yast2-ncurses-2.15.14-3.x86_64 <install> [S0:0][package]libzypp-3.2.2-15.x86_64 <install> [S0:0][package]yast2-perl-bindings-2.15.1-8.x86_64 <install> [S0:0][package]libstdc++42-4.2.1_20070604-8.x86_64 <install> [S0:0][package]yast2-ncurses-2.15.19-2.x86_64 <install> [S0:0][package]yast2-qt-2.15.5-32.x86_64 <install> [S0:0][package]yast2-pkg-bindings-2.15.20-12.x86_64 <install> [S0:0][package]zypper-0.7.2-7.x86_64 <uninstall> [S0:0][package]yast2-qt-2.15.5-3.x86_64 <uninstall> [S0:0][package]libstdc++41-4.1.3_20070302-3.x86_64 <uninstall> [S0:0][package]libzypp-3.2.0-3.x86_64 <uninstall> [S0:0][package]yast2-pkg-bindings-2.15.15-3.x86_64 Continue? [y/n] n real 0m27.877s user 0m24.082s sys 0m1.308s -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Also to note: - the search there in Jano's numbers is implemented loading and then iterating. Searching will take almost no time when we implement the query from zypper. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wednesday 20 June 2007 01:45:55 pm Francis Giannaros wrote:
On 6/19/07, Duncan Mac-Vicar Prett <dmacvicar@suse.de> wrote:
real 0m8.925s user 0m7.880s sys 0m0.688s
These results sound really great! Would it perform similarly if there was a lot more metadata to deal with (say, if Factory was added)?
No. But it wont perform as bad as before. What takes time is to build the cache. Like 2 min in my slow machine with 3 repos plus factory. But you can add repos without refreshing inmediately, and you can install without refresh (if you have old metadata). This was the most annoying bug, that sources refreshed when starting up, when deleting, when listing. That is gone. Once the cache is there, with factory it goes up from 8 to 15 seconds to load the resolvables, read rpm and solve plus the time to install the rpms. Note: we haven't done any optimization yet. Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (6)
-
Duncan Mac-Vicar P. -
Duncan Mac-Vicar Prett
-
Francis Giannaros -
Jan Kupec -
Klaus Kaempf -
Stanislav Visnovsky