[opensuse-buildservice] [PATCH 0/3] Fixes to obs-service-tar_scm
Three minor fixes to obs-service-tar_scm to work better when an existing source tarball is updated. The third patch requires obs-server v2.3 (more specifically commit 9647a4ed in upstream) to work correctly. Markus Lehtonen (3): [obs-service-tar_scm] Checkout master before git update [obs-service-tar_scm] create tarball even if version does not change [obs-service-tar_scm] Grab (possibly) existing tar from .old subdir tar_scm | 24 +++--------------------- 1 files changed, 3 insertions(+), 21 deletions(-) -- 1.7.3.4 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Check out master branch before doing git update so that merge succeeds and the files are updated. --- tar_scm | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/tar_scm b/tar_scm index 257b794..34b50f9 100644 --- a/tar_scm +++ b/tar_scm @@ -175,6 +175,8 @@ elif [ "$MYSCM" == "git" ]; then # update existing content for speed/bandwidth reasons cd "$TAR_DIRECTORY" OLDVERSION=`git show --pretty=%at | head -n 1` + # switch to default branch so that merge succeeds + git checkout master || exit 1 git pull $MYGITARGS || exit 1 if [ -n "$MYREVISION" ]; then git checkout "$MYREVISION" || exit 1 -- 1.7.3.4 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Freitag, 26. August 2011, 08:43:05 schrieb Markus Lehtonen:
Check out master branch before doing git update so that merge succeeds and the files are updated.
Would this still work when you actually want to checkout a non-master branch ? eg. when MYREVISION is 2.3 for build-service repository ? We may should do fetch checkout merge here instead ?
--- tar_scm | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/tar_scm b/tar_scm index 257b794..34b50f9 100644 --- a/tar_scm +++ b/tar_scm @@ -175,6 +175,8 @@ elif [ "$MYSCM" == "git" ]; then # update existing content for speed/bandwidth reasons cd "$TAR_DIRECTORY" OLDVERSION=`git show --pretty=%at | head -n 1` + # switch to default branch so that merge succeeds + git checkout master || exit 1 git pull $MYGITARGS || exit 1 if [ -n "$MYREVISION" ]; then git checkout "$MYREVISION" || exit 1 -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 2011/08/26 at 09:36:52, Adrian Schröter wrote:
Am Freitag, 26. August 2011, 08:43:05 schrieb Markus Lehtonen:
Check out master branch before doing git update so that merge succeeds and the files are updated.
Would this still work when you actually want to checkout a non-master branch ? eg. when MYREVISION is 2.3 for build-service repository ?
Yes it should. My patch just tries to make sure that the merge in pull succeeds. This matters if no revision is specified in _service - i.e. always use the latest git head in the tarball. Problems arise e.g. if you first define MYREVISION 2.3 (which "checkouts" 2.3 in to a non-branch), and then later, want to get latest HEAD and remove MYREVISION. In this case the merge in git pull would fail and you'd still get version 2.3 from that non-branch that was previously checked out.
We may should do
fetch checkout merge
here instead ?
Hmm, shouldn't matter, AFAIU. I see one potential problem, though. If the git HEAD on the remote server points to something else than refs/heads/master, let's say refs/heads/upstream. Then git clone would create branch 'upstream' and there'd be no master branch and checkout would fail. I don't then know how to nicely checkout the "default branch", i.e. how to determine the name of that branch. Something like? git fetch DEFAULTBRANCH=`sed s'!^ref.*/!!' .git/refs/remotes/origin/HEAD` git checkout $DEFAULTBRANCH git merge What do you think?
--- tar_scm | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/tar_scm b/tar_scm index 257b794..34b50f9 100644 --- a/tar_scm +++ b/tar_scm @@ -175,6 +175,8 @@ elif [ "$MYSCM" == "git" ]; then # update existing content for speed/bandwidth reasons cd "$TAR_DIRECTORY" OLDVERSION=`git show --pretty=%at | head -n 1` + # switch to default branch so that merge succeeds + git checkout master || exit 1 git pull $MYGITARGS || exit 1 if [ -n "$MYREVISION" ]; then git checkout "$MYREVISION" || exit 1
-- Markus -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Freitag, 26. August 2011, 12:42:06 schrieb Markus Lehtonen:
On 2011/08/26 at 09:36:52, Adrian Schröter wrote:
Am Freitag, 26. August 2011, 08:43:05 schrieb Markus Lehtonen:
Check out master branch before doing git update so that merge
succeeds
and the files are updated.
Would this still work when you actually want to checkout a non-master branch ? eg. when MYREVISION is 2.3 for build-service repository ?
Yes it should. My patch just tries to make sure that the merge in pull succeeds. This matters if no revision is specified in _service - i.e. always use the latest git head in the tarball. Problems arise e.g. if you first define MYREVISION 2.3 (which "checkouts" 2.3 in to a non-branch), and then later, want to get latest HEAD and remove MYREVISION. In this case the merge in git pull would fail and you'd still get version 2.3 from that non-branch that was previously checked out.
ic
We may should do
fetch checkout merge
here instead ?
Hmm, shouldn't matter, AFAIU. I see one potential problem, though. If the git HEAD on the remote server points to something else than refs/heads/master, let's say refs/heads/upstream. Then git clone would create branch 'upstream' and there'd be no master branch and checkout would fail. I don't then know how to nicely checkout the "default branch", i.e. how to determine the name of that branch. Something like? git fetch DEFAULTBRANCH=`sed s'!^ref.*/!!' .git/refs/remotes/origin/HEAD` git checkout $DEFAULTBRANCH git merge
What do you think?
Hm, I would avoid to handle the DEFAULTBRANCH at all. It is just important for us to handle OUR branch. So my code would be this: if [ -n "$MYREVISION" ]; then git fetch $MYGITARGS || exit 1 git checkout "$MYREVISION" || exit 1 git merge "origin/$MYREVISION" HEAD || exit 1 else git pull $MYGITARGS || exit 1 fi and leave it to "git pull" to handle the default branch if needed. Does this make sense ? bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 2011/08/29 at 15:06:06, Adrian Schröter wrote:
Am Freitag, 26. August 2011, 12:42:06 schrieb Markus Lehtonen:
On 2011/08/26 at 09:36:52, Adrian Schröter wrote:
We may should do
fetch checkout merge
here instead ?
Hmm, shouldn't matter, AFAIU. I see one potential problem, though. If the git HEAD on the remote server points to something else than refs/heads/master, let's say refs/heads/upstream. Then git clone would create branch 'upstream' and there'd be no master branch and checkout would fail. I don't then know how to nicely checkout the "default branch", i.e. how to determine the name of that branch. Something like? git fetch DEFAULTBRANCH=`sed s'!^ref.*/!!' .git/refs/remotes/origin/HEAD` git checkout $DEFAULTBRANCH git merge
What do you think?
Hm, I would avoid to handle the DEFAULTBRANCH at all. It is just important for us to handle OUR branch. So my code would be this:
if [ -n "$MYREVISION" ]; then git fetch $MYGITARGS || exit 1 git checkout "$MYREVISION" || exit 1 git merge "origin/$MYREVISION" HEAD || exit 1 else git pull $MYGITARGS || exit 1 fi
and leave it to "git pull" to handle the default branch if needed. Does this make sense ?
Mm, it basically makes sense. However, the pull fails if I first define MYREVISION but later remove it. The repo is in the "detached head" state (i.e. non-branch) which is not updated at all as pull fails. AFAIK, pull doesn't know how to handle "a default branch" it only fetches and tries to merge for the current branch. I don't know how relevant corner case this is, though. -- Markus -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Don't skip tarball creation because otherwise the tarball is completely deleted as bs_service deletes all old _service* files. --- tar_scm | 20 -------------------- 1 files changed, 0 insertions(+), 20 deletions(-) diff --git a/tar_scm b/tar_scm index 34b50f9..9278569 100644 --- a/tar_scm +++ b/tar_scm @@ -142,11 +142,6 @@ if [ "$MYSCM" == "svn" ]; then fi NEWVERSION=`LC_ALL=C svn info | sed -n 's,^Last Changed Rev: \(.*\),\1,p'` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout @@ -183,11 +178,6 @@ elif [ "$MYSCM" == "git" ]; then fi NEWVERSION=`git show --pretty=%at | head -n 1` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout @@ -212,11 +202,6 @@ elif [ "$MYSCM" == "hg" ]; then hg pull || exit 1 NEWVERSION=`hg id -i -rtip` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout @@ -245,11 +230,6 @@ elif [ "$MYSCM" == "bzr" ]; then fi NEWVERSION=`bzr revno` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout -- 1.7.3.4 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Freitag, 26. August 2011, 08:43:06 schrieb Markus Lehtonen:
Don't skip tarball creation because otherwise the tarball is completely deleted as bs_service deletes all old _service* files.
Ups, right. I tried to solve this differently in openSUSE:Tools now, because I want to avoid that the same tar ball gets re-generated and may get stored twice in source server due to a different time stamp (for example in gzip header). again, please test :) adrian
--- tar_scm | 20 -------------------- 1 files changed, 0 insertions(+), 20 deletions(-)
diff --git a/tar_scm b/tar_scm index 34b50f9..9278569 100644 --- a/tar_scm +++ b/tar_scm @@ -142,11 +142,6 @@ if [ "$MYSCM" == "svn" ]; then fi NEWVERSION=`LC_ALL=C svn info | sed -n 's,^Last Changed Rev: \(.*\),\1,p'` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout @@ -183,11 +178,6 @@ elif [ "$MYSCM" == "git" ]; then fi NEWVERSION=`git show --pretty=%at | head -n 1` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout @@ -212,11 +202,6 @@ elif [ "$MYSCM" == "hg" ]; then hg pull || exit 1 NEWVERSION=`hg id -i -rtip` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout @@ -245,11 +230,6 @@ elif [ "$MYSCM" == "bzr" ]; then fi NEWVERSION=`bzr revno` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi mv "$TAR_DIRECTORY" "${FILE}" || exit 1 else # new checkout -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Freitag, 26. August 2011, 08:47:28 schrieb Adrian Schröter:
Am Freitag, 26. August 2011, 08:43:06 schrieb Markus Lehtonen:
Don't skip tarball creation because otherwise the tarball is completely deleted as bs_service deletes all old _service* files.
Ups, right. I tried to solve this differently in openSUSE:Tools now, because I want to avoid that the same tar ball gets re-generated and may get stored twice in source server due to a different time stamp (for example in gzip header).
Again new version. Removed the code as you suggested, but fixed gzip handling in recompress service instead. bye adrian
again, please test :) adrian
---
tar_scm | 20 -------------------- 1 files changed, 0 insertions(+), 20 deletions(-)
diff --git a/tar_scm b/tar_scm index 34b50f9..9278569 100644 --- a/tar_scm +++ b/tar_scm @@ -142,11 +142,6 @@ if [ "$MYSCM" == "svn" ]; then
fi
NEWVERSION=`LC_ALL=C svn info | sed -n 's,^Last Changed Rev: \(.*\),\1,p'` cd - - if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi
mv "$TAR_DIRECTORY" "${FILE}" || exit 1
else
# new checkout
@@ -183,11 +178,6 @@ elif [ "$MYSCM" == "git" ]; then
fi NEWVERSION=`git show --pretty=%at | head -n 1` cd -
- if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi
mv "$TAR_DIRECTORY" "${FILE}" || exit 1
else
# new checkout
@@ -212,11 +202,6 @@ elif [ "$MYSCM" == "hg" ]; then
hg pull || exit 1 NEWVERSION=`hg id -i -rtip` cd -
- if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi
mv "$TAR_DIRECTORY" "${FILE}" || exit 1
else
# new checkout
@@ -245,11 +230,6 @@ elif [ "$MYSCM" == "bzr" ]; then
fi NEWVERSION=`bzr revno` cd -
- if [ "$OLDVERSION" == "$NEWVERSION" ]; then - # cleanup and skip - rm -rf "$TAR_DIRECTORY" - exit 0 - fi
mv "$TAR_DIRECTORY" "${FILE}" || exit 1
else
# new checkout -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 2011/08/26 at 09:47:28, Adrian Schröter wrote:
Am Freitag, 26. August 2011, 08:43:06 schrieb Markus Lehtonen:
Don't skip tarball creation because otherwise the tarball is completely deleted as bs_service deletes all old _service* files.
Ups, right. I tried to solve this differently in openSUSE:Tools now, because I want to avoid that the same tar ball gets re-generated and may get stored twice in source server due to a different time stamp (for example in gzip header).
again, please test :)
Tested it briefly. Seems to work. -- Markus -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
--- tar_scm | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/tar_scm b/tar_scm index 9278569..f35310d 100644 --- a/tar_scm +++ b/tar_scm @@ -115,7 +115,7 @@ if [ -z "$FILE" -a "$MYSCM" == "hg" ]; then FILE="${FILE##*/}" fi -existing_tar=$(echo $SRCDIR/_service:*tar_scm:${FILE}-*.tar*) +existing_tar=$(echo $SRCDIR/.old/_service:*tar_scm:${FILE}-*.tar*) if [ -e "$existing_tar" ]; then UNCOMPRESS="cat" if [ "${existing_tar%.tar.gz}" != "$existing_tar" ]; then -- 1.7.3.4 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
A version which should work also with OBS < 2.3 is submitted thanks a lot for all your work on this! :) adrian Am Freitag, 26. August 2011, 08:43:07 schrieb Markus Lehtonen:
--- tar_scm | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tar_scm b/tar_scm index 9278569..f35310d 100644 --- a/tar_scm +++ b/tar_scm @@ -115,7 +115,7 @@ if [ -z "$FILE" -a "$MYSCM" == "hg" ]; then FILE="${FILE##*/}" fi
-existing_tar=$(echo $SRCDIR/_service:*tar_scm:${FILE}-*.tar*) +existing_tar=$(echo $SRCDIR/.old/_service:*tar_scm:${FILE}-*.tar*) if [ -e "$existing_tar" ]; then UNCOMPRESS="cat" if [ "${existing_tar%.tar.gz}" != "$existing_tar" ]; then -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (2)
-
Adrian Schröter -
Markus Lehtonen