Mailinglist Archive: opensuse-features (365 mails)

< Previous Next >
[openFATE 305945] Get rid of "symsets" in kernel package dependencies
  • From: fate_noreply@xxxxxxx
  • Date: Mon, 1 Mar 2010 19:19:13 +0100 (CET)
  • Message-id: <feature-305945-32@xxxxxxxxxxxxxx>
Feature changed by: Dirk Mueller (dirkmueller)
Feature #305945, revision 32
Title: Get rid of "symsets" in kernel package dependencies

openSUSE-11.2: Done
Priority
Requester: Important

Requested by: Michal Marek (michal-m)

Description:
Currently, the kernel rpm packages group the exported kernel and module
symbols in find-provides and provide these "symsets". It has been done
this way to reduce the amount of data the solver has to process. It has
a number of problems though:
* Moving modules around or adding symbols changes the symset checksums,
so the rpm build needs symsets of the previous kernel(s) to provide
both old and new checksums
* changing or removing one unused or unimportant symbol means that each
kmp that uses any other symbol from the same symset needs to be rebuilt
Now that we have a much better solver, it's time to get rid of this and
provide just the individual symbols. I created a test project in the
build service: home:michal-m:kill-symsets. which rebuilds all 11.1 kmps
against a normal kernel and against a kernel with symsets turned off
(this kernel is built in home:michal-m:kill-symsets:changes). The size
difference is rather low, total size of all the rpms on i586 and x86_64
increased by about 1.3M, installed size decreased by 2.6M (because of
the removed /boot/symsets* files). Size of yum metadata increased by 1.4
M.
What needs to be done is to make sure that the solver can really handle
this fine (-> Michael). I'm waiting for the test repos to appear on
download.o.o.

Relations:
+ - Bugreport tracking metadata size regression (url:
+ http://bugzilla.novell.com/show_bug.cgi?id=582907)

Test Case:
All kernel and kmp packages have ksym(...) provides/requires instead of
kernel(...). KMPs must install without dependency errors.

Discussion:
#1: Michal Marek (michal-m) (2009-03-03 16:01:08)
Added attachment table of size changes (sizes.txt)

#2: Michal Marek (michal-m) (2009-03-17 16:26:51)
I tried installing packages from
http://download.opensuse.org/repositories/home:/michal-m:/kill-symsets/ and
I didn't notice any prerformance issues with zypper, so I hope there
won't be any problems with 4k additional rpm provides.
I pushed this to master, let's hope it will be possible to do this
change also in SLE11-SP1.

#3: Andreas Gruenbacher (agruen) (2009-03-17 16:59:51) (reply to #2)
Thanks -- hopefully you were not too fast with this: the issue to watch
out for here is not the rpm package size. Rather, the real issues are:
* The additional size of the repository metadata,
* The speed of the resolver for complex tasks like system updates.
* The time it will take rpm to resolve dependencies during package
installation and updates. This may affect all packages, not just the
kernels and kmps. (Our resolver sure performs much better than rpm
itself.)
Did you try out these things as well? Thanks!
(Also see feature 303641, which is a duplicate of this feature.)

#4: Michal Marek (michal-m) (2009-03-17 17:30:39) (reply to #3)
The yum repodata changed by about 1.5MB, see the attachment. I didn't
really test yet how rpm itself copes with the change.

#5: Andreas Jaeger (a_jaeger) (2009-06-09 14:55:32)
Michael, could you comment on this, please?

#6: Michael Schröder (mlschroe) (2009-06-09 16:26:42) (reply to #5)
How much bigger is primary.xml.gz?
I don't think our solver will have any problems, but I don't know about
rpm. Please do some tests with the rpms, i.e. installung multiple
kernel versions and flavors, updating kernels...

#7: Michal Marek (michal-m) (2009-08-12 12:22:14) (reply to #6)
I copied matching kernel flavors from SLE11_BRANCH and master kotd to
two directories and ran createrepo in each of them. The difference is:
$ ls -1sh */repodata/primary.xml.gz
1,4M factory/repodata/primary.xml.gz
60K sle11/repodata/primary.xml.gz

which matches my earlier tests. When I installed the factory kernel-
default on a 11.1/x86_64, /var/lib/rpm grew from 61MB (after rpm --
rebuilddb) to 63MB. When installing/updating kernels and KMPs, I
couldn't observe any slowdown, most time was spent in mkintrd and the
weak-modules2 script.
Interesting observation: the kernel exports over 9k symbols, but only
about 800 are actually used by KMPs.

#19: Alexander Orlovskyy (aorlovskyy) (2009-10-27 10:59:40) (reply to
#6)
Michael, on openSUSE 11.2 RC1 when i instal multiple kmp modules i always get
an
message of dependencies error is that a bug ?

#20: Michal Marek (michal-m) (2009-10-27 12:52:20) (reply to #19)
Which kmps and which errors? The kabi is not frozen yet, so make sure
you have kmps matching your kernel. And please report problems like
this one in bugzilla, this feature is done.

#8: Michal Marek (michal-m) (2009-08-12 12:22:44)
Andreas - you rejected the feature for 11.2, but in fact this has been
in factory since March (see comment #2). And I haven't seen any reports
about slow kernel / kmp installation, so I think the impact is
relatively low.

#9: Andreas Jaeger (a_jaeger) (2009-08-12 13:31:38) (reply to #8)
Then why was this not marked as done? I expected there was further work
needed. Michal, you can set features yourself as done.

#21: Marcus Meissner (msmeissn) (2010-02-24 21:44:50)
We have today found out that is detrimental to the openSUSE 11.2
infrastructure.
Every kernel-flavour RPM takes 800KB, kernel-flavour-base 400KB
uncompressed primary size.
As we keep all kernels in the repo.
2 kernel updates for 11.2 so far have caused a primary.xml size
increase of 33MB.
Yes. 16 MB more per kernel updates and we have 2 architectures only on
11.2.
This is going to be quite unmanagabkle very soon.



--
openSUSE Feature:
https://features.opensuse.org/305945

< Previous Next >
This Thread
  • No further messages