Mailinglist Archive: opensuse-factory (757 mails)

< Previous Next >
Re: [opensuse-factory] Bug 177758
  • From: Andreas Jaeger <aj@xxxxxxx>
  • Date: Tue, 23 May 2006 18:39:55 +0200
  • Message-id: <ho64jw1yqs.fsf@xxxxxxxxxxxxx>
Pascal Bleser <pascal.bleser@xxxxxxxxx> writes:

> Ulrich Windl wrote:
>> On 23 May 2006 at 14:09, jdd wrote:
>>
>>> If I follow well the thread it seems this is a meta-data
>>> download problem.
>>
>> If it's a download problem, why would the CPU be at 100%?
>
> That sounds a lot like XML repository metadata parsing.
> libzypp/ZMD is most probably not parsing the data stream as it is
> being downloaded, so I presume it's download everything (for one
> repository) first, then parse.
>
> Would be interesting to do some profiling on parse-metadata.
> Anything available for Mono ?

parse-metadata is in C++.

>
> What XML parsing model is being used there, SAX, DOM, StAX ?
>
> Most probably not DOM...

libxml2's most appropriate one for this, don't know what it is.

>
> When I look at my ("guru") RPM-MD repository for 10.0 (which is large,
> but a lot smaller than the FTP tree):
> == compressed:
> primary.xml.gz = 1,058,326 (bytes)
> filelists.xml.gz = 1,029,756
> other.xml.gz = 497,642
>
> == uncompressed:
> primary.xml = 6,174,750
> filelists.xml = 11,032,393
> other.xml = 2,950,989
>
> == compression ratio:
> primary.xml = 5.83
> filelists.xml = 10.42
> other.xml = 5.92
>
> Now when I look at SL-10.1/inst-source/suse/repodata:
> == compressed:
> primary.xml.gz = 8,056,136 (bytes)
> filelists.xml.gz = 17,474,199
> other.xml.gz = 53,265,854
>
> BTW, other.xml.gz is *huge* (contains %changelog information) - I
> don't know whether libzypp/zmd download and/or use "other.xml.gz"
> though. smart (http://smartpm.org) doesn't.

It does not use it anymore. That was one of the early fixes ;-)

>
> == uncompressed:
> primary.xml = 47,127,500
> filelists.xml = 211,884,423
> other.xml = 206,534,422
>
> == compression ratio:
> primary.xml = 5.83
> filelists.xml = 12.12
> other.xml = 3.87
>
> Assuming that other.xml is not being used by libzypp/ZMD, I would
> guess the following memory usage with DOM:
> - primary.xml: 47MB on disk => 150-200MB RAM
> - filelists.xml: 210MB on disk => 600-800MB RAM
>
> Hmm.. after all... maybe it _is_ DOM ;)

No, definitely not,
Andreas
--
Andreas Jaeger, aj@xxxxxxx, http://www.suse.de/~aj/
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 N├╝rnberg, Germany
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
< Previous Next >
This Thread
Follow Ups