Feature changed by: Dirk Mueller (dirkmueller) Feature #305945, revision 32 Title: Get rid of "symsets" in kernel package dependencies openSUSE-11.2: Done Priority Requester: Important Requested by: Michal Marek (michal-m) Description: Currently, the kernel rpm packages group the exported kernel and module symbols in find-provides and provide these "symsets". It has been done this way to reduce the amount of data the solver has to process. It has a number of problems though: * Moving modules around or adding symbols changes the symset checksums, so the rpm build needs symsets of the previous kernel(s) to provide both old and new checksums * changing or removing one unused or unimportant symbol means that each kmp that uses any other symbol from the same symset needs to be rebuilt Now that we have a much better solver, it's time to get rid of this and provide just the individual symbols. I created a test project in the build service: home:michal-m:kill-symsets. which rebuilds all 11.1 kmps against a normal kernel and against a kernel with symsets turned off (this kernel is built in home:michal-m:kill-symsets:changes). The size difference is rather low, total size of all the rpms on i586 and x86_64 increased by about 1.3M, installed size decreased by 2.6M (because of the removed /boot/symsets* files). Size of yum metadata increased by 1.4 M. What needs to be done is to make sure that the solver can really handle this fine (-> Michael). I'm waiting for the test repos to appear on download.o.o. Relations: + - Bugreport tracking metadata size regression (url: + http://bugzilla.novell.com/show_bug.cgi?id=582907) Test Case: All kernel and kmp packages have ksym(...) provides/requires instead of kernel(...). KMPs must install without dependency errors. Discussion: #1: Michal Marek (michal-m) (2009-03-03 16:01:08) Added attachment table of size changes (sizes.txt) #2: Michal Marek (michal-m) (2009-03-17 16:26:51) I tried installing packages from http://download.opensuse.org/repositories/home:/michal-m:/kill-symsets/ and I didn't notice any prerformance issues with zypper, so I hope there won't be any problems with 4k additional rpm provides. I pushed this to master, let's hope it will be possible to do this change also in SLE11-SP1. #3: Andreas Gruenbacher (agruen) (2009-03-17 16:59:51) (reply to #2) Thanks -- hopefully you were not too fast with this: the issue to watch out for here is not the rpm package size. Rather, the real issues are: * The additional size of the repository metadata, * The speed of the resolver for complex tasks like system updates. * The time it will take rpm to resolve dependencies during package installation and updates. This may affect all packages, not just the kernels and kmps. (Our resolver sure performs much better than rpm itself.) Did you try out these things as well? Thanks! (Also see feature 303641, which is a duplicate of this feature.) #4: Michal Marek (michal-m) (2009-03-17 17:30:39) (reply to #3) The yum repodata changed by about 1.5MB, see the attachment. I didn't really test yet how rpm itself copes with the change. #5: Andreas Jaeger (a_jaeger) (2009-06-09 14:55:32) Michael, could you comment on this, please? #6: Michael Schröder (mlschroe) (2009-06-09 16:26:42) (reply to #5) How much bigger is primary.xml.gz? I don't think our solver will have any problems, but I don't know about rpm. Please do some tests with the rpms, i.e. installung multiple kernel versions and flavors, updating kernels... #7: Michal Marek (michal-m) (2009-08-12 12:22:14) (reply to #6) I copied matching kernel flavors from SLE11_BRANCH and master kotd to two directories and ran createrepo in each of them. The difference is: $ ls -1sh */repodata/primary.xml.gz 1,4M factory/repodata/primary.xml.gz 60K sle11/repodata/primary.xml.gz which matches my earlier tests. When I installed the factory kernel- default on a 11.1/x86_64, /var/lib/rpm grew from 61MB (after rpm -- rebuilddb) to 63MB. When installing/updating kernels and KMPs, I couldn't observe any slowdown, most time was spent in mkintrd and the weak-modules2 script. Interesting observation: the kernel exports over 9k symbols, but only about 800 are actually used by KMPs. #19: Alexander Orlovskyy (aorlovskyy) (2009-10-27 10:59:40) (reply to #6) Michael, on openSUSE 11.2 RC1 when i instal multiple kmp modules i always get an message of dependencies error is that a bug ? #20: Michal Marek (michal-m) (2009-10-27 12:52:20) (reply to #19) Which kmps and which errors? The kabi is not frozen yet, so make sure you have kmps matching your kernel. And please report problems like this one in bugzilla, this feature is done. #8: Michal Marek (michal-m) (2009-08-12 12:22:44) Andreas - you rejected the feature for 11.2, but in fact this has been in factory since March (see comment #2). And I haven't seen any reports about slow kernel / kmp installation, so I think the impact is relatively low. #9: Andreas Jaeger (a_jaeger) (2009-08-12 13:31:38) (reply to #8) Then why was this not marked as done? I expected there was further work needed. Michal, you can set features yourself as done. #21: Marcus Meissner (msmeissn) (2010-02-24 21:44:50) We have today found out that is detrimental to the openSUSE 11.2 infrastructure. Every kernel-flavour RPM takes 800KB, kernel-flavour-base 400KB uncompressed primary size. As we keep all kernels in the repo. 2 kernel updates for 11.2 so far have caused a primary.xml size increase of 33MB. Yes. 16 MB more per kernel updates and we have 2 architectures only on 11.2. This is going to be quite unmanagabkle very soon. -- openSUSE Feature: https://features.opensuse.org/305945