Mailinglist Archive: opensuse-features (291 mails)

< Previous Next >
[openFATE 309167] lzma compression for primary.xml
  • From: fate_noreply@xxxxxxx
  • Date: Thu, 6 May 2010 22:57:13 +0200 (CEST)
  • Message-id: <feature-309167-5@xxxxxxxxxxxxxx>
Feature changed by: Jan Engelhardt (jengelh)
Feature #309167, revision 5
Title: lzma compression for primary.xml

Buildservice: Evaluation
Priority
Requester: Mandatory

openSUSE-11.3: Evaluation
Priority
Requester: Mandatory

Requested by: Michal Marek (michal-m)

Description:
Providing all exported kernel symbols as individual dependencies (fate
305945) resulted in a DoS against download.o.o, because the primary.xml.
gz file in the update repository grows with each kernel update (bnc
582907 and 586960). The least intrusive fix to this seems to be: Sort
the entries in primary.xml and provide a lzma compressed version in
addition to primary.xml.gz. Testing shows that this brings the metadata
growth back to the 11.1 symset approach (about 100k for each subsequent
kernel update) and also reduces the size, so it would be beneficial
also for normal repositories. The repomd.xml format extension could
look like:
<data type="primary">
<location href="repodata/primary.xml.gz"/>
<checksum type="sha">1233456...</checksum>
<timestamp>1268305225</timestamp>
<open-checksum type="sha">abcdef...</open-checksum>
</data>
<data type="primary-lzma">
<location href="repodata/primary.xml.lzma"/>
<checksum type="sha">56789a...</checksum>
<timestamp>1268305245</timestamp>
<open-checksum type="sha">abcdef...</open-checksum>
</data>

With the meaning that if there is a primary.xml.lzma with the same open-
checksum, download this one instead of primary.xml.gz, decompress it
with lzma (in default install since 11.1) and then continue as usual.
Old clients would continue working with the primary.xml.gz file. The
feature primarily targets libzypp, but also extending yum should not be
hard.

Relations:
- Get rid of "symsets" in kernel package dependencies (feature/id:
305945)
- zsync support for update channel refresh (feature/id: 309088)
- fate #305945 causes extreme update channel metadata overhead
(novell/bugzilla/id: 586960)
https://bugzilla.novell.com/show_bug.cgi?id=586960

Discussion:
#2: Michal Marek (michal-m) (2010-05-05 11:05:36)
patches for createrepo:
http://gitorious.org/mmarek-misc/createrepo/commits/lzma-support libzypp
patches:
http://gitorious.org/~mmarek/opensuse/libzypp-lzma/commits/lzma-support
sat-solver patches:
http://gitorious.org/~mmarek/opensuse/sat-solver-lzma/commits/lzma-support
createrepo is submitted to system:packagemanager as sr 39243. Libzypp and sat-
solver patches were posted to the zypp-devel mailing list:
http://lists.opensuse.org/zypp-devel/2010-05/msg00005.html
(http://lists.opensuse.org/zypp-devel/2010-05/msg00005.html)
.

+ #3: Jan Engelhardt (jengelh) (2010-05-06 22:57:05)
+ Should use xz these days.



--
openSUSE Feature:
https://features.opensuse.org/309167

< Previous Next >
This Thread
References