Feature changed by: Michael Meeks (michael_meeks) Feature #308837, revision 3 Title: performance: jail creation ... Buildservice: New Priority Requester: Mandatory Requested by: Michael Meeks (michael_meeks) Description: For leaf GNOME packages - partly due to the granular packaging (probably) it takes a long time - sometimes 30%+ of build time, just to populate the jail from clean. Furthermore, the distinction between 'clean' and 'unclean' jails is highly problematic - with local builds often failing mid-way due to jail un-cleanness. It would be ideal to make jail construction lightening fast, such that cleaning is never necessary. This should be possible with a stackable FUSE filing system, keyed on the RPMs that are installed: ultimately, given a set of packages to install - we should end up with a know-able / cache-able state. Better - this should allow multiple local builds to be performed, concurrently on the local host without the excessive resourcing (for leaf packages) of installing two massive, duplicate chroot jails, with the problems of shared-nothing in memory. Discussion: #1: Jan Engelhardt (jengelh) (2010-01-19 01:50:16) Removing --clean from the argument list to build is a nice shortcut. Additionally, you want to make sure to use an fs that is reasonably fast for lots of metadata updates, such as it happens with buildroot initialization. /tmp, /var/tmp (latter could be a symlink) could be made btrfs. I am not convinced a stackable filesystem helps much, as each buildroot may have package sets that can almost totally differ. Beyond the Preinstalls, I do not expect any real package sharing. + #2: Michael Meeks (michael_meeks) (2010-01-19 11:17:49) (reply to #1) + Well - when we get to leaf packages; there is a lot of sharing possible + - at least conceptually. + As an example, the entire GNOME stack - will require all the + dependencies of gtk+, gconf, dbus, X11 - and so on; there is a huge + common set of packages that that will pull in. KDE will be similar I + guess, likewise for Moblin: everything up to clutter-devel :-) Even for + simple stuff - we tend to need a few compilers, perl, cmake, autotools + (etc.) installed to build things. Consider OpenOffice_org package + builds - we have 20+ source packages, all of which require the same + base stack - base-system+gnome+kde+X11+OpenOffice_org-bootstrap. + Clearly the win accumulates to the leaf packages though. + So - sure, while providing a perfect file-system image with the right + packages pre-installed is prolly impossible; having a set of stacked + layered images - that include our common development stacks, should + save a large number of individual package installs - all of which take + significant time & space. Naturally, a few more packages will be + required to be installed on top, but we can cut out 90% of the package + installation phase [ I hope ]. That should give enough file-system + space to easily allow multiple parallel package builds on the same + laptop as a bonus ;-) -- openSUSE Feature: https://features.opensuse.org/308837