[opensuse-kernel] Bashing my head against the desk re kernel-flavors
I can't take it anymore. Development on a new kernel flavor is just so complicated and rife with little things to "take care of" which are simply hindering development and making life a misery of little mistakes, and forgetful actions, and even when something comes out right, building an RPM includes untarring 250MB of source code just to tell me the config is bad within the first 4 files compiled or so. The one thing that is really tiring me, though, is the layout of /usr/src/packages. In the way SUSE has it, kernel-source, kernel-flavor and every other thing installed from an SRPM install into exactly the same directory. This has some effects that are annoying me to some degree I am not comfortable with 1) broken packages which maintainers are unwilling to fix which blithely untar every *.tar.bz2 in /usr/src/packages/SOURCES (xorg-x11-libs!) 2) kernel-source, kernel-flavor SRPMS overwrite each other if they are installed. ARGH! 3) kernel-flavor .spec file is actually in SOURCES and not SPECS when it gets installed. WHAT? I really need to know, what kind of design decision is it that all RPMs get dumped into the same directory? Now, in a sane system like the /usr/share/doc/packages directory each package has it's basename in there. Each package is separate and self-contained. Docs for one do not go in the folder for docs for another.. :) Why was this never brought across to RPM builds? Is it because people are used to using build chroots and so on? Even in that scenario building a source RPM sort of requires the whole shebang to be in place right? What am I doing wrong here and how can I stop using /usr/src/packages for RPM development, and if this IS possible AND recommended, why does this method still even exist? If not doing anything wrong, why on earth can't we split RPM sources into their own package directories in /usr/src/packages like /usr/src/packages/kernel-source /usr/src/packages/kernel-whatever (yes of course it will duplicate files with the above but it's all compressed) /usr/src/packages/xorg-x11-libs etc.? -- Matt Sealey, Genesi, Manager Developer Relations -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Matt Sealey wrote:
I can't take it anymore. Development on a new kernel flavor is just so complicated and rife with little things to "take care of" which are simply hindering development and making life a misery of little mistakes, and forgetful actions, and even when something comes out right, building an RPM includes untarring 250MB of source code just to tell me the config is bad within the first 4 files compiled or so.
Try the attached spec files as a base instead. kernel-source should work fine without alteration. kernel-default just needs the build_flavor macro changed to whatever you want to call it. These spec files, or something close to them, are going to be used in the next release.
The one thing that is really tiring me, though, is the layout of /usr/src/packages.
[snipped]
If not doing anything wrong, why on earth can't we split RPM sources into their own package directories in /usr/src/packages like
/usr/src/packages/kernel-source /usr/src/packages/kernel-whatever (yes of course it will duplicate files with the above but it's all compressed) /usr/src/packages/xorg-x11-libs
On my system, I use the following ~/.rpmmacros:
%_sourcedir /home/jeffm/src/packages/%{name}
%_specdir /home/jeffm/src/packages/%{name}
%_builddir /home/jeffm/src/packages/BUILD
This is an RPM thing, not a kernel package thing. Your best bet is to
file a bug report. It should be easy to change the macros in
/usr/lib/rpm/macros to something else - but that's not a decision the
kernel teams make.
-Jeff
--
Jeff Mahoney
SUSE Labs
#
# spec file for package kernel-default (Version 2.6.29)
#
# Copyright (c) 2008 SUSE LINUX Products GmbH, Nuernberg, Germany.
# This file and all modifications and additions to the pristine
# package are under the same license as the package itself.
#
# Please submit bugfixes or comments via http://bugs.opensuse.org/
#
# norootforbuild
%define using_buildservice 0%{?opensuse_bs}
%define variant
%if %using_buildservice
# Strip off the build number ("y") from the "x.y" release number
%define source_rel %(release=%release; echo ${release%.*})
%else
# We don't have build numbers internally
%define source_rel %release
%endif
# Don't use shell commands in build macros, this won't work outside of rpm
%define build_flavor default
%define build_kdump 0
%define build_xen 0
%define build_vanilla 0
%define build_ps3 0
%define srcversion 2.6.28
%define patchversion 2.6.29-rc1
# This works fine, but the build service won't expand it for security reasons
#%define variant %(case %build_flavor in (rt|rt_*) echo -rt ;; (vanilla) echo -vanilla ;; esac)
%if %{build_flavor} == "kdump"
%define build_kdump 1
%endif
%if %{build_flavor} == "xen"
%define build_xen 1
%endif
%if %{build_flavor} == "vanilla"
%define build_vanilla 1
%endif
%if %{build_flavor} == "ps3"
%define build_ps3 1
%endif
%(chmod +x %_sourcedir/{arch-symbols,guards,config-subst,check-for-config-changes,check-supported-list,built-in-where,modversions,symsets.pl})
%define arch_symbols %(%_sourcedir/arch-symbols %_target_cpu)
%define symbols %(set -- %name kernel-%build_flavor $(case %build_flavor in (rt|rt_*) echo RT ;; esac) $([ -e %_sourcedir/extra-symbols ] && cat %_sourcedir/extra-symbols) ; echo $*)
%define cpu_arch_flavor %(%_sourcedir/guards %symbols %arch_symbols < %_sourcedir/config.conf | grep '/%build_flavor$')
# Define some CONFIG variables as rpm macros as well. (rpm cannot handle
# defining them all at once.)
%define config_vars CONFIG_MODULES
%{expand:%(eval "$(test -n "%cpu_arch_flavor" && tar xfj %_sourcedir/config.tar.bz2 --to-stdout config/%cpu_arch_flavor)"; for config in %config_vars; do echo "%%global $config ${!config:-n}"; done)}
%ifarch %ix86 x86_64
%define install_vdso 1
%else
%define install_vdso 0
%endif
%if %build_vanilla || %build_kdump || %CONFIG_MODULES != "y"
%define split_packages 0
%else
%define split_packages 1
%endif
Name: kernel-%build_flavor
Summary: Dummy summary
Version: 2.6.29
%if %using_buildservice
Release: rc1.<RELEASE>
BuildRequires: kernel-source
%else
%define kernel_source_release %(rpm -q kernel-source%variant-%version --qf "%{RELEASE}")
Release: %kernel_source_release
BuildRequires: kernel-source%variant = %version-%kernel_source_release
%endif
License: GPL
Group: System/Kernel
Url: http://www.kernel.org/
AutoReqProv: on
BuildRequires: coreutils module-init-tools sparse
BuildRequires: fdupes
Provides: %{name}_%_target_cpu = %version-%release
%if %split_packages
Requires: %name-base_%_target_cpu = %version-%release
%endif
Requires(pre): coreutils awk
Requires(post): module-init-tools
# This Requires is wrong, because the post/postun scripts have a
# test -x update-bootloader, having perl-Bootloader is not a hard requirement.
# But, there is no way to tell rpm or yast to schedule the installation
# of perl-Bootloader before kernel-binary.rpm if both are in the list of
# packages to install/update. Likewise, this is true for mkinitrd.
# A specific version of perl-Bootloader is not required, because the post/postun
# scripts handle the two API versions of 10.1/SLES10 GA and 10.2/SLES10 SP1
Requires(post): perl-Bootloader
Requires(post): mkinitrd
#!BuildIgnore: perl-Bootloader mkinitrd
%if ! %using_buildservice
BuildRequires: kernel-dummy
%endif
%ifarch ia64
# arch/ia64/scripts/unwcheck.py
BuildRequires: python
%endif
%ifarch s390 s390x
BuildRequires: dwarfextract
%endif
%if %build_xen
%ifarch %ix86
Provides: kernel-xenpae = %version
Obsoletes: kernel-xenpae <= %version
%endif
#!BuildIgnore: xen
%endif
Provides: %name-nongpl
Obsoletes: %name-nongpl
%if %build_vanilla
# force bzip2 instead of lzma compression to allow install on older dist versions
%define _binary_payload w9.bzdio
%endif
# dead network if installed on SLES10, otherwise it will work (mostly)
Conflicts: sysfsutils < 2.0
%if ! %build_vanilla
Conflicts: apparmor-profiles <= 2.1
Conflicts: apparmor-parser < 2.3
# root-lvm only works with newer udevs
Conflicts: udev < 118
Conflicts: lvm2 < 2.02.33
%endif
%ifarch %ix86
Conflicts: libc.so.6()(64bit)
%endif
Provides: kernel = %version-%source_rel
%ifarch %ix86
Provides: k_athlon k_debug k_deflt k_deflt_22 k_deflt_24 k_eide k_laptop k_orig k_pentiu k_pos_ibm k_psmp k_smp k_smp_22 k_smp_24 smp kernel-smp
Obsoletes: k_athlon k_debug k_deflt k_deflt_22 k_deflt_24 k_eide k_laptop k_orig k_pentiu k_pos_ibm k_psmp k_smp k_smp_22 k_smp_24 smp kernel-smp
%else
%ifarch ia64
Provides: k_debug k_deflt k_itanium2 k_itanium2-smp k_smp kernel-sn2
Obsoletes: k_debug k_deflt k_itanium2 k_itanium2-smp k_smp kernel-sn2
%else
%ifarch ppc
Provides: k_chrp k_chrps k_deflt k_pmac k_pmacs k_prep k_preps
Obsoletes: k_chrp k_chrps k_deflt k_pmac k_pmacs k_prep k_preps
%else
%ifarch ppc64
%else
%ifarch s390x
Provides: kernel-64bit k_deflt
Obsoletes: kernel-64bit k_deflt
%else
%ifarch x86_64
Provides: k_deflt k_numa k_smp smp kernel-smp
Obsoletes: k_deflt k_numa k_smp smp kernel-smp
%endif
%endif
%endif
%endif
%endif
%endif
Source10: preun.sh
Source11: postun.sh
Source12: pre.sh
Source13: post.sh
Source20: series.conf
Source21: config.conf
Source22: supported.conf
Source30: arch-symbols
Source31: guards
Source32: config-subst
Source33: check-for-config-changes
Source34: check-supported-list
Source40: build-source-timestamp
Source41: built-in-where
Source44: find-provides
Source45: module-renames
Source46: modversions
Source47: symsets.pl
Source100: config.tar.bz2
Source101: kabi.tar.bz2
%define my_builddir %_builddir/%{name}-%{version}
BuildRoot: %{_tmppath}/%{name}-%{version}-build
ExclusiveArch: %ix86 ia64 ppc ppc64 s390x x86_64
# These files are found in the kernel-source package:
NoSource: 100
NoSource: 101
# The following KMPs have been integrated into the kernel package.
Obsoletes: iwlwifi-kmp
Obsoletes: ipw3945-kmp
Obsoletes: adm8211-kmp
Obsoletes: rt2x00-kmp
Obsoletes: rfswitch-kmp
Obsoletes: uvcvideo-kmp
Obsoletes: atl2-kmp
Obsoletes: wlan-ng-kmp
Obsoletes: et131x-kmp
Obsoletes: ivtv-kmp
Obsoletes: at76_usb-kmp
Obsoletes: pcc-acpi-kmp
Obsoletes: uvcvideo-kmp
Obsoletes: ralink-rt2860-kmp
# Build with bash instead of sh as the shell: this turns on bash
# extensions like <(...).
%define _buildshell /bin/bash
# Provide the exported symbols as "ksym(symbol) = hash"
%define __find_provides %my_builddir/find-provides %name
# Will modules not listed in supported.conf abort the kernel build (0/1)?
%define supported_modules_check 0
%define tolerate_unknown_new_config_options 0
# kABI change tolerance (default in maintenance should be 4, 6, 8 or 15,
# 31 is the maximum; see scripts/kabi-checks)
%define tolerate_kabi_changes 31
%description
Dummy description.
%prep
%setup -q -c -T -a 100 -a 101
supported_conf() {
%_sourcedir/guards %symbols $* < %_sourcedir/supported.conf | sort -u
}
# Generate the list of modules to be marked as supported
{ supported_conf base
for how in external; do
comm -2 -3 <(supported_conf base $how) <(supported_conf base) \
| sed -e 's:$: '"$how"':'
done
} | sed -e 's,.*/,,' -e 's,\.ko$,,' > %my_builddir/Module.supported
# Create grep pattern file for the modules to end up in the base package
comm -2 -3 <(supported_conf base) <(supported_conf) \
| sed -e 's:.*/::' -e 's:^:\\/:' -e 's:$:\.ko$:' \
> %my_builddir/grep-for-base-modules
# Release number without the EXTRAVERSION
RELEASE=%source_rel
while [ "$RELEASE" != "${RELEASE#[^0-9]*.}" ]; do
RELEASE=${RELEASE#[^0-9]*.}
done
if [ -f %_sourcedir/localversion ] ; then
cat %_sourcedir/localversion > %my_builddir/localversion
fi
KERNELRELEASE="%patchversion-$RELEASE-%build_flavor"
SRCDIR="/usr/src/linux-%patchversion-$RELEASE%variant"
BUILDDIR="%buildroot$SRCDIR-obj/%cpu_arch_flavor"
mkdir -p "$BUILDDIR"
cat config/%cpu_arch_flavor \
| %_sourcedir/config-subst CONFIG_LOCALVERSION '"'-$RELEASE-%build_flavor'"' \
| %_sourcedir/config-subst CONFIG_SUSE_KERNEL y \
%if 0%{?__debug_package:1}
| %_sourcedir/config-subst CONFIG_DEBUG_INFO y \
%endif
> %my_builddir/.config
cpu_arch_flavor="%cpu_arch_flavor"
cat >> .rpm-defs <
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeff Mahoney wrote:
Matt Sealey wrote:
I can't take it anymore. Development on a new kernel flavor is just so complicated and rife with little things to "take care of" which are simply hindering development and making life a misery of little mistakes, and forgetful actions, and even when something comes out right, building an RPM includes untarring 250MB of source code just to tell me the config is bad within the first 4 files compiled or so.
Try the attached spec files as a base instead. kernel-source should work fine without alteration. kernel-default just needs the build_flavor macro changed to whatever you want to call it. These spec files, or something close to them, are going to be used in the next release.
Oh, actually, you can remove this chunk entirely from kernel-default.spec. It takes the release number from the installed kernel-source package. %if ! %using_buildservice BuildRequires: kernel-dummy %endif - -Jeff - -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iEYEARECAAYFAkluPNIACgkQLPWxlyuTD7J9NgCfQMGZw1QZW1cykBH4NJmKDbB8 T+YAn1q3AWuAQEAm1SmsRSlhjuVzoLIb =FFTa -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Jeff Mahoney wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jeff Mahoney wrote:
Oh, actually, you can remove this chunk entirely from kernel-default.spec. It takes the release number from the installed kernel-source package.
%if ! %using_buildservice BuildRequires: kernel-dummy %endif
I was already doing this in my current stuff, although I was told by some SUSE guys that this was required for BS (obviously it's not). The whole kernel-dummy thing is really weird and I don't really understand why it's needed anyway. I am going to back everything up and start this kernel thing all over again, because it is driving me seven ways to insanity at the moment. I should be asleep recovering from food poisoning but... -- Matt -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matt Sealey wrote:
Jeff Mahoney wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jeff Mahoney wrote:
Oh, actually, you can remove this chunk entirely from kernel-default.spec. It takes the release number from the installed kernel-source package.
%if ! %using_buildservice BuildRequires: kernel-dummy %endif
I was already doing this in my current stuff, although I was told by some SUSE guys that this was required for BS (obviously it's not).
The whole kernel-dummy thing is really weird and I don't really understand why it's needed anyway.
The confusion arises because there are multiple build systems now. There used to be only one internal build system. Then, a few years ago, the openSUSE Build Service was built. Recently, we have an internal version of that as well. The "new" build systems can handle release number synchronization between related packages and two-factor release numbers, like "1.14". The "old' build system couldn't handle synchronization and only supported single-factor release numbers, like "13". kernel-dummy is a way to "fake" synchronization. The kernel-dummy package is built first, and then *all* of the other packages inherit its release number via some black magic you don't want to know about. The distributions are now build using the "new" build system, but we still use the old one for building kernels for testing by customers. So, it's still needed for that. Eventually it will go away. Without the release number synchronization, you could end up with things like: kernel-source-2.6.27-184 kernel-default-2.6.27-139 kernel-syms-2.6.27-167 They would all be based on the same code, but you'd have no way of knowing that. You could also install a new kernel-source, but have no obvious way of knowing that they're NOT based on the same code. So, we do a bit of juggling to make sure all the release numbers match up and make those relationships obvious. - -Jeff - -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iEYEARECAAYFAkluRUQACgkQLPWxlyuTD7Lv7wCgm48S7pMxH+DKWO/EiZ3+UrIp gv4AniSqhbdR6ctoU9Guo6gJ1GYD9VSq =fJZF -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Jeff Mahoney wrote:
Matt Sealey wrote:
On my system, I use the following ~/.rpmmacros: %_sourcedir /home/jeffm/src/packages/%{name} %_specdir /home/jeffm/src/packages/%{name} %_builddir /home/jeffm/src/packages/BUILD
:O So if I install an RPM it will go into that directory all the time?
This is an RPM thing, not a kernel package thing.
Indeed but..
Your best bet is to file a bug report.
I think I will just be told the above and have it marked "WONTFIX".. as usual :D
/usr/lib/rpm/macros to something else - but that's not a decision the kernel teams make.
Okay. I will have a go during the week or weekend. Writing it on the whiteboard now. -- Matt -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Hi, On Wed, 14 Jan 2009, Matt Sealey wrote:
I can't take it anymore. Development on a new kernel flavor is just so complicated and rife with little things to "take care of" which are simply hindering development and making life a misery of little mistakes, and forgetful actions, and even when something comes out right, building an RPM includes untarring 250MB of source code just to tell me the config is bad within the first 4 files compiled or so.
The one thing that is really tiring me, though, is the layout of /usr/src/packages.
If you use 'build' (the package and program in there) to build your packages, not rpm -b* or rpmbuild directly it doesn't matter where your sources are, hence the (unfortunate) layout of /usr/src/packages/ doesn't enter the picture. If you nevertheless want to have a different layout of %_sourcedir, simply add the following to your ~/.rpmmacros (or root, as whomever you start building): %_sourcedir %{_topdir}/SOURCES/%{name}-%{version} And yes, the current flat layout is an artifact of us using build roots all the time, so nobody cared to ever change it to something else after dozens of years (remember to us internally its layout and existence is irrelevant) :-) As more than a few people are annoyed by that it might change in the future. I bet there's a bug report about that layout already, in case there isn't there should be one I guess, you might want to check. Ciao, Michael. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Michael Matz wrote:
Hi,
And yes, the current flat layout is an artifact of us using build roots all the time, so nobody cared to ever change it to something else after dozens of years (remember to us internally its layout and existence is irrelevant) :-) As more than a few people are annoyed by that it might change in the future. I bet there's a bug report about that layout already, in case there isn't there should be one I guess, you might want to check.
https://bugzilla.novell.com/show_bug.cgi?id=466222 I couldn't really find one so I made one that is a dependent of the xorg-x11-libs thing I tripped up on when first trying this stuff. I couldn't CC either of you guys.. :/ -- Matt -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
* Matt Sealey (matt@genesi-usa.com) [20090114 23:29]:
I couldn't CC either of you guys.. :/
Replace suse.de in the address by novell.com and it will work. Philipp -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Wed, Jan 14, 2009 at 01:13:00PM -0600, Matt Sealey wrote:
I can't take it anymore. Development on a new kernel flavor is just so complicated and rife with little things to "take care of" which are simply hindering development and making life a misery of little mistakes, and forgetful actions, and even when something comes out right, building an RPM includes untarring 250MB of source code just to tell me the config is bad within the first 4 files compiled or so.
The one thing that is really tiring me, though, is the layout of /usr/src/packages.
In the way SUSE has it, kernel-source, kernel-flavor and every other thing installed from an SRPM install into exactly the same directory. This has some effects that are annoying me to some degree I am not comfortable with
1) broken packages which maintainers are unwilling to fix which blithely untar every *.tar.bz2 in /usr/src/packages/SOURCES (xorg-x11-libs!)
2) kernel-source, kernel-flavor SRPMS overwrite each other if they are installed. ARGH!
3) kernel-flavor .spec file is actually in SOURCES and not SPECS when it gets installed. WHAT?
I really need to know, what kind of design decision is it that all RPMs get dumped into the same directory? Now, in a sane system like the /usr/share/doc/packages directory each package has it's basename in there. Each package is separate and self-contained. Docs for one do not go in the folder for docs for another.. :)
kernel-source SRPM is sufficient to install for _all_ kernel flavours. You do not need the kernel-<flavour> SRPMs. Ciao, Marcus -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Wednesday 14 January 2009 20:13:00 Matt Sealey wrote:
The one thing that is really tiring me, though, is the layout of /usr/src/packages.
As pointed out already, it is straightforward to change rpm's defaults in ~/.rpmmacros. This still only gives different per-user settings, and different packages can still step on their toes (and they will still create a big mess in the SOURCES directory). People are led to include %name in all patch names to somewhat reduce the mess, for example. When using rpm directly without a chroot, I like to keep everything in the same sub-directory instead. For building, I use the attached script. This would make much saner defaults for rpm IMO. This doesn't seem to be a popular opinion, however. Andreas
participants (6)
-
Andreas Gruenbacher
-
Jeff Mahoney
-
Marcus Meissner
-
Matt Sealey
-
Michael Matz
-
Philipp Thomas