Mailinglist Archive: opensuse-buildservice (311 mails)

< Previous Next >
Re: [opensuse-buildservice] build 2009.07.27 doesn't detect pre-existing Ubuntu installs [PATCH]
I've modified our local copy of init_buildsystem to detect debs that are
already installed. It is based on the 2009.07.27 version of the build
rpm. The patch is attached.

I've tried to match up the dpkg/dpkg-query/debsums commands as closely
to the equivalent rpm commands as I could, but I'm not that familiar
with using any of them since I use Synaptic and apt-get to handle
package management on Ubuntu. debsums is not part of the preinstalls in
the Ubuntu:9.10 prjconf, so you'll need to add that for the rpm --verify
equivalent (debsums) to work.

On Mon, 2009-11-09 at 12:47 -0600, Adrian Schröter wrote:
Am Montag, 9. November 2009 19:16:32 schrieb Adrian Schröter:
Am Montag, 9. November 2009 18:52:11 schrieb Adrian Schröter:
Am Montag, 9. November 2009 17:47:11 schrieb Luke Imhoff:
When building against xUbuntu_9.10 using build-2009.07.27, it keeps
reinstall all the .debs. Do .deb based workareas not have detection
that packages are already installed in the chroot like .rpm based
chroots do? Or do I need to upgrade to a newer build? We're still
using OBS 1.5 for the server, so I didn't want to upgrade build if it
might break compatibility.

It worked latetly for me. You may try to upgrade osc and esp. the build
script first on your workstation to test again.

okay, I was wrong here. This is currently not supported by the build
script. A partial fix is in svn now, but it is not complete.

And another sorry, there is currently to much missing to make this cleanly
working. This is not that easy to fix for me, because I have not enough clue
about debs.

If you like to implement support for this, have a look into the
init_buildsystem script of build package.

You need to extend the validate_cache_file funcaiton in first place, to
generate also .id files from the deb meta data. I dunno what makes sense as
unique identifier for debs repositories.

bye
adrian


Index: init_buildsystem
===================================================================
--- init_buildsystem (revision 2049)
+++ init_buildsystem (working copy)
@@ -31,6 +31,7 @@
TMPFILE=$BUILD_ROOT/tmpfile
#buildhost removed so that id can be generated from repo files
#RPMIDFMT="%{NAME}-%{VERSION}-%{RELEASE} %{BUILDHOST}-%{BUILDTIME}\n"
+DEBIDFMT='${Package}_${Version}\n'
RPMIDFMT="%{NAME}-%{VERSION}-%{RELEASE} %{BUILDTIME}\n"

PREPARE_XEN=
@@ -404,7 +405,7 @@
rm -rf $BUILD_ROOT/.init_b_cache
cleanup_and_exit 0
fi
-
+
PACKAGES_TO_INSTALL=
PACKAGES_TO_PREINSTALL=
PACKAGES_TO_RUNSCRIPTS=
@@ -449,21 +450,26 @@
#
# now test if there is already a build dir.
#
-if test ! -f $BUILD_ROOT/var/lib/rpm/packages.rpm -a ! -f
$BUILD_ROOT/var/lib/rpm/Packages ; then
+if test ! -f $BUILD_ROOT/var/lib/rpm/packages.rpm -a ! -f
$BUILD_ROOT/var/lib/rpm/Packages -a ! -f $BUILD_ROOT/var/lib/dpkg/status ; then
+
mkdir -p $BUILD_ROOT/var/lib/rpm || cleanup_and_exit 1
mkdir -p $BUILD_ROOT/usr/src/packages/SOURCES || cleanup_and_exit 1
mkdir -p $BUILD_ROOT/etc || cleanup_and_exit 1
mkdir -p $BUILD_ROOT/proc || cleanup_and_exit 1
test -f $BUILD_ROOT/etc/HOSTNAME || hostname -f > $BUILD_ROOT/etc/HOSTNAME
+ # deb based system
if test $PSUF = deb ; then
- mkdir -p $BUILD_ROOT/var/lib/dpkg
- mkdir -p $BUILD_ROOT/var/log
- mkdir -p $BUILD_ROOT/etc/default
+ mkdir -p $BUILD_ROOT/var/lib/dpkg || cleanup_and_exit 1
+ mkdir -p $BUILD_ROOT/var/log || cleanup_and_exit 1
+ mkdir -p $BUILD_ROOT/etc/default || cleanup_and_exit 1
:> $BUILD_ROOT/var/lib/dpkg/status
:> $BUILD_ROOT/var/lib/dpkg/available
:> $BUILD_ROOT/var/log/dpkg.log
:> $BUILD_ROOT/etc/ld.so.conf
:> $BUILD_ROOT/etc/default/rcS
+ # rpm based system
+ else
+ mkdir -p $BUILD_ROOT/var/lib/rpm || cleanup_and_exit 1
fi
for PKG in $PACKAGES_TO_RUNSCRIPTS ; do
: > $BUILD_ROOT/.init_b_cache/scripts/$PKG.run
@@ -531,17 +537,34 @@
fi

#
-# get list and ids of already installed rpms
+# get list and ids of already installed packages
#
mkdir -p $BUILD_ROOT/.init_b_cache/alreadyinstalled
+ALREADYINSTALLED_CMD=""
+ALREADYINSTALLED_FMT=""
+# if rpm based system
if test -f $BUILD_ROOT/var/lib/rpm/packages.rpm -o -f
$BUILD_ROOT/var/lib/rpm/Packages ; then
- chroot $BUILD_ROOT rpm -qa --qf "%{NAME} $RPMIDFMT" | (
- while read pp ii; do
- echo "$ii" > "$BUILD_ROOT/.init_b_cache/alreadyinstalled/$pp"
- done
- )
+ ALREADYINSTALLED_CMD="rpm -qa --qf"
+ ALREADYINSTALLED_FMT="%{NAME} $RPMIDFMT"
+# if deb based system
+elif test -f $BUILD_ROOT/var/lib/dpkg/status ; then
+ # dpkg-query show format also uses $ for substitution, so need to escape it
+ # for bash.
+ ALREADYINSTALLED_CMD="dpkg-query --show --showformat"
+ ALREADYINSTALLED_FMT="\${Package} $DEBIDFMT"
fi

+if test -n "$ALREADYINSTALLED_CMD" -a -n "$ALREADYINSTALLED_CMD" ; then
+ echo "Calculating installed packages..."
+ chroot $BUILD_ROOT $ALREADYINSTALLED_CMD "$ALREADYINSTALLED_FMT" | (
+ while read package_name package_id; do
+ echo "$package_id" >
"$BUILD_ROOT/.init_b_cache/alreadyinstalled/$package_name"
+ done
+ )
+else
+ echo "$BUILD_ROOT does not have an rpm or deb database. No installed
package can be determined."
+fi
+
#
# reorder packages (already done in XEN continuation)
#
@@ -556,20 +579,30 @@
# delete all packages we don't want
#
mkdir -p $BUILD_ROOT/.init_b_cache/todelete
+# mark all packages that are already installed as to be deleted
for PKG in $BUILD_ROOT/.init_b_cache/alreadyinstalled/* ; do
PKG=${PKG##*/}
test "$PKG" = "*" && continue
ln $BUILD_ROOT/.init_b_cache/alreadyinstalled/$PKG
$BUILD_ROOT/.init_b_cache/todelete/$PKG
done
+# mark all packages that are already installed and would be reinstalled as not
to be deleted
for PKG in $PACKAGES_TO_INSTALL_FIRST $PACKAGES_TO_INSTALL ; do
rm -f $BUILD_ROOT/.init_b_cache/todelete/$PKG
done
+# delete packages marked for deletion
for PKG in $BUILD_ROOT/.init_b_cache/todelete/* ; do
PKG=${PKG##*/}
test "$PKG" = "*" && continue
echo deleting `sed -e 's/ .*//' < $BUILD_ROOT/.init_b_cache/todelete/$PKG`
- chroot $BUILD_ROOT rpm --nodeps -e $PKG 2>&1 | \
- grep -v -e "^r.*failed: No such file or directory" -e "^error: failed
to stat .*: No such file or directory"
+
+ # debs
+ if test $PSUF = deb ; then
+ chroot $BUILD_ROOT dpkg --force all --remove $PKG 2>&1
+ # rpms
+ else
+ chroot $BUILD_ROOT rpm --nodeps -e $PKG 2>&1 | \
+ grep -v -e "^r.*failed: No such file or directory" -e "^error:
failed to stat .*: No such file or directory"
+ fi
done
rm -rf $BUILD_ROOT/.init_b_cache/todelete

@@ -601,25 +634,12 @@
test -f $BUILD_ROOT/installed-pkg/$PKG && continue

if test $PSUF = deb ; then
- # debian world, install deb files
- test -L $BUILD_ROOT/.init_b_cache/rpms/$PKG.deb || continue
- if ! test "$BUILD_ROOT/.init_b_cache/rpms/$PKG.deb" -ef
"$BUILD_ROOT/.init_b_cache/$PKG.deb" ; then
- rm -f $BUILD_ROOT/.init_b_cache/$PKG.deb
- cp $BUILD_ROOT/.init_b_cache/rpms/$PKG.deb
$BUILD_ROOT/.init_b_cache/$PKG.deb || cleanup_and_exit 1
- fi
- PKGID=`readlink $BUILD_ROOT/.init_b_cache/rpms/$PKG.deb`
- PKGID="${PKGID##*/}"
- PKGID="${PKGID%.deb}"
- echo "installing ${PKGID%_*}"
- ( chroot $BUILD_ROOT dpkg -i --force all .init_b_cache/$PKG.deb 2>&1 ||
touch $BUILD_ROOT/exit ) | \
- perl -ne '$|=1;/^(Configuration file|Installing new config
file|Selecting previously deselected|\(Reading database|Unpacking |Setting
up|Creating config file|Preparing to replace dpkg)/||/^$/||print'
- test -e $BUILD_ROOT/exit && cleanup_and_exit 1
- echo "$PKGID debian" > $BUILD_ROOT/installed-pkg/$PKG
- continue
+ EXT="deb"
+ else
+ EXT="rpm"
fi
-
- test -L $BUILD_ROOT/.init_b_cache/rpms/$PKG.rpm || continue
-
+
+ test -L $BUILD_ROOT/.init_b_cache/rpms/$PKG.$EXT || continue
if test -f $BUILD_ROOT/.init_b_cache/rpms/$PKG.id -a -f
$BUILD_ROOT/.init_b_cache/alreadyinstalled/$PKG ; then
read PKGID < $BUILD_ROOT/.init_b_cache/rpms/$PKG.id
read OLDPKGID < $BUILD_ROOT/.init_b_cache/alreadyinstalled/$PKG
@@ -629,27 +649,50 @@
continue
fi
fi
-
- PKGID=`rpm -qp --qf "$RPMIDFMT" $RPMCHECKOPTS_HOST
$BUILD_ROOT/.init_b_cache/rpms/$PKG.rpm`
-
+
+ if test $PSUF = deb ; then
+ PKGNAME=`dpkg --field $BUILD_ROOT/.init_b_cache/rpms/$PKG.$EXT Package`
+ PKGVERSION=`dpkg --field $BUILD_ROOT/.init_b_cache/rpms/$PKG.$EXT
Version`
+ PKGID="${PKGNAME}_${PKGVERSION}"
+ else
+ PKGID=`rpm -qp --qf "$RPMIDFMT" $RPMCHECKOPTS_HOST
$BUILD_ROOT/.init_b_cache/rpms/$PKG.$EXT`
+ fi
+
if test -f $BUILD_ROOT/.init_b_cache/alreadyinstalled/$PKG ; then
+ KEEP=true
read OLDPKGID < $BUILD_ROOT/.init_b_cache/alreadyinstalled/$PKG
if test "$PKGID" != "$OLDPKGID" ; then
echo deleting unwanted ${OLDPKGID%% *}
- chroot $BUILD_ROOT rpm --nodeps -e $PKG 2>&1 | \
- grep -v -e "^r.*failed: No such file or directory" -e "^error:
failed to stat .*: No such file or directory"
+
+ if test $PSUF = deb ; then
+ # TODO filter acceptable errors as is done with rpms
+ chroot $BUILD_ROOT dpkg --force all --remove $PKG 2>&1
+ else
+ chroot $BUILD_ROOT rpm --nodeps -e $PKG 2>&1 | \
+ grep -v -e "^r.*failed: No such file or directory" -e
"^error: failed to stat .*: No such file or directory"
+ fi
elif test "$VERIFY_BUILD_SYSTEM" = true ; then
- chroot $BUILD_ROOT rpm --verify $PKG 2>&1 | tee $TMPFILE
- if grep ^missing $TMPFILE > /dev/null ; then
- echo deleting incomplete ${PKGID%% *}
- chroot $BUILD_ROOT rpm --nodeps -e $PKG 2>&1 | \
- grep -v -e "^r.*failed: No such file or directory" -e
"^error: failed to stat .*: No such file or directory"
- else
- echo "keeping ${PKGID%% *}"
- echo "$PKGID" > $BUILD_ROOT/installed-pkg/$PKG
- continue
- fi
- else
+ echo verifying $PKG
+ if test $PSUF = deb ; then
+ chroot $BUILD_ROOT debsums $PKG 2>&1 | tee $TMPFILE
+ if grep -e 'FAILED$\|(No such file or directory)' $TMPFILE >
/dev/null ; then
+ echo deleting incomplete ${PKGID%% *}
+ # TODO filter acceptable errors as is done with rpms
+ chroot $BUILD_ROOT dpkg --force all --remove $PKG 2>&1
+ KEEP=false
+ fi
+ else
+ chroot $BUILD_ROOT rpm --verify $PKG 2>&1 | tee $TMPFILE
+ if grep ^missing $TMPFILE > /dev/null ; then
+ echo deleting incomplete ${PKGID%% *}
+ chroot $BUILD_ROOT rpm --nodeps -e $PKG 2>&1 | \
+ grep -v -e "^r.*failed: No such file or directory" -e
"^error: failed to stat .*: No such file or directory"
+ KEEP=false
+ fi
+ fi
+ fi
+
+ if $KEEP ; then
echo "keeping ${PKGID%% *}"
echo "$PKGID" > $BUILD_ROOT/installed-pkg/$PKG
continue
@@ -664,16 +707,22 @@
export ADDITIONAL_PARAMS="$ADDITIONAL_PARAMS --force"
fi
echo "installing ${PKGID%% *}"
- if ! test "$BUILD_ROOT/.init_b_cache/rpms/$PKG.rpm" -ef
"$BUILD_ROOT/.init_b_cache/$PKG.rpm" ; then
- rm -f $BUILD_ROOT/.init_b_cache/$PKG.rpm
- cp $BUILD_ROOT/.init_b_cache/rpms/$PKG.rpm
$BUILD_ROOT/.init_b_cache/$PKG.rpm || cleanup_and_exit 1
+ if ! test "$BUILD_ROOT/.init_b_cache/rpms/$PKG.$EXT" -ef
"$BUILD_ROOT/.init_b_cache/$PKG.$EXT" ; then
+ rm -f $BUILD_ROOT/.init_b_cache/$PKG.$EXT
+ cp $BUILD_ROOT/.init_b_cache/rpms/$PKG.$EXT
$BUILD_ROOT/.init_b_cache/$PKG.$EXT || cleanup_and_exit 1
fi
- ( chroot $BUILD_ROOT rpm --ignorearch --nodeps -U --oldpackage
--ignoresize $RPMCHECKOPTS \
- $ADDITIONAL_PARAMS .init_b_cache/$PKG.rpm 2>&1 || \
- touch $BUILD_ROOT/exit ) | \
- grep -v "^warning:.*saved as.*rpmorig$"
+ if test $PSUF = deb ; then
+ ( chroot $BUILD_ROOT dpkg -i --force all .init_b_cache/$PKG.deb 2>&1 ||
touch $BUILD_ROOT/exit ) | \
+ perl -ne '$|=1;/^(Configuration file|Installing new config
file|Selecting previously deselected|\(Reading database|Unpacking |Setting
up|Creating config file|Preparing to replace dpkg)/||/^$/||print'
+ test -e $BUILD_ROOT/exit && cleanup_and_exit 1
+ else
+ ( chroot $BUILD_ROOT rpm --ignorearch --nodeps -U --oldpackage
--ignoresize $RPMCHECKOPTS \
+ $ADDITIONAL_PARAMS .init_b_cache/$PKG.rpm 2>&1 || \
+ touch $BUILD_ROOT/exit ) | \
+ grep -v "^warning:.*saved as.*rpmorig$"
+ fi
# delete link so package is only installed once
- rm -f $BUILD_ROOT/.init_b_cache/$PKG.rpm
+ rm -f $BUILD_ROOT/.init_b_cache/$PKG.$EXT
test -e $BUILD_ROOT/exit && cleanup_and_exit 1
echo "$PKGID" > $BUILD_ROOT/installed-pkg/$PKG

< Previous Next >