Pascal Bleser <pascal.bleser@skynet.be> writes:
Ulrich Windl wrote:
On 23 May 2006 at 14:09, jdd wrote:
If I follow well the thread it seems this is a meta-data download problem.
If it's a download problem, why would the CPU be at 100%?
That sounds a lot like XML repository metadata parsing. libzypp/ZMD is most probably not parsing the data stream as it is being downloaded, so I presume it's download everything (for one repository) first, then parse.
Would be interesting to do some profiling on parse-metadata. Anything available for Mono ?
parse-metadata is in C++.
What XML parsing model is being used there, SAX, DOM, StAX ?
Most probably not DOM...
libxml2's most appropriate one for this, don't know what it is.
When I look at my ("guru") RPM-MD repository for 10.0 (which is large, but a lot smaller than the FTP tree): == compressed: primary.xml.gz = 1,058,326 (bytes) filelists.xml.gz = 1,029,756 other.xml.gz = 497,642
== uncompressed: primary.xml = 6,174,750 filelists.xml = 11,032,393 other.xml = 2,950,989
== compression ratio: primary.xml = 5.83 filelists.xml = 10.42 other.xml = 5.92
Now when I look at SL-10.1/inst-source/suse/repodata: == compressed: primary.xml.gz = 8,056,136 (bytes) filelists.xml.gz = 17,474,199 other.xml.gz = 53,265,854
BTW, other.xml.gz is *huge* (contains %changelog information) - I don't know whether libzypp/zmd download and/or use "other.xml.gz" though. smart (http://smartpm.org) doesn't.
It does not use it anymore. That was one of the early fixes ;-)
== uncompressed: primary.xml = 47,127,500 filelists.xml = 211,884,423 other.xml = 206,534,422
== compression ratio: primary.xml = 5.83 filelists.xml = 12.12 other.xml = 3.87
Assuming that other.xml is not being used by libzypp/ZMD, I would guess the following memory usage with DOM:
- primary.xml: 47MB on disk => 150-200MB RAM
- filelists.xml: 210MB on disk => 600-800MB RAM
Hmm.. after all... maybe it _is_ DOM ;)
No, definitely not, Andreas -- Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj/ SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126