Hello,
On Monday, December 19, 2011 11:21:31 AM Robert Schiele wrote:
[...]
So, to sum this up: What I observed in 5-7 seems to be a bug that the
source service results are not always copied correctly, picking an
outdated result for copying instead of the most recent one. Especially
the fact that the problem even persists for newly created projects
makes me a bit nervous about the reliability here.
Anyone with an idea, what could be causing that, which places in the
sources I should focus on for searching for the cause? Alternatively
does someone have a good idea for a suitable workaround that is
smarter than my current approach to just explicitly copy in all
packages that suffer from this behavior?
[...]
reproduced the issue and it seems to be that not all meta data gets copied to the linking project. handleservice and other are looking for xservicemd5 metadata and others.
Since it does not found those meta data it triggers a source service run. But the source files actually got correctly copied, so the source service run could and should be
avoided.
I made a pull request for a patch which copies all known meta data:
https://github.com/openSUSE/open-build-service/pull/23
This patch only avoids avoidable serviceruns, e.g. if a linkedproject=all get created.
But there is still at least one open issue or thing to discuss we have to look into:
- retrigger of servicerun of the origin project causes that all linkedprojects trigger a servicerun:
this might could be avoided as well by coping tree meta data and sources to linkedprojects
instead of doing multiple sourceservice runs.
- other cases which might be checked for similiar linkedproject issues?
Best Regards,
Daniel
---
commit cd82c3d903fb2a4e8eba3e1ff9aced46edf7e53c
Author: Daniel Gollub
Date: Fri Jan 20 09:10:13 2012 +0100
[backend] copy all known tree meta data
Commit 97c1d6c61 introduce only a partial copy of the required meta
data. This commit copies additionally to the srcmd5 metadata, also
the meta data of: lsrcmd5, xservicemd5, lservicemd5 - if available.
This is required for linkproject=all projects to avoid redudant source
service triggers. As the meta data did not get copied, handleservice
always triggered a source service run for linkedprojects. Even when the
sources already got copied with copyfiles().
diff --git a/src/backend/bs_srcserver b/src/backend/bs_srcserver
index d82fff2..902112d 100755
--- a/src/backend/bs_srcserver
+++ b/src/backend/bs_srcserver
@@ -680,6 +680,20 @@ sub copyfiles {
}
}
+sub copytreefile {
+ my ($treedir, $lprojid, $packid, $md5) = @_;
+ return unless $md5;
+
+ my $ltreedir = "$treesdir/$lprojid/$packid";
+ $ltreedir = "$srcrep/$packid" if $BSConfig::nosharedtrees == 2 && ! -e "$ltreedir/$md5-MD5SUMS";
+
+ if (-e "$ltreedir/$md5-MD5SUMS") {
+ my $meta = readstr("$ltreedir/$md5-MD5SUMS");
+ mkdir_p($treedir);
+ writestr("$uploaddir/$$", "$treedir/$md5-MD5SUMS", $meta);
+ }
+}
+
sub getrev_git {
my ($projid, $packid, $rev) = @_;
my $gitdir = "$projectsdir/$projid.pkg/$packid.git";
@@ -723,20 +737,32 @@ sub getrev {
if ($lrev) {
# make sure that we may access the sources of this package
checksourceaccess($lprojid, $packid);
- my $files = lsrev($lrev);
+
+ my $linkinfo = {};
+ my $files = lsrev($lrev, $linkinfo);
+
copyfiles($projid, $packid, $lprojid, $packid, $files);
+
my $srcmd5 = $lrev->{'srcmd5'};
+ my $lsrcmd5 = $linkinfo->{'lsrcmd5'};
+ my $xservicemd5 = $linkinfo->{'xservicemd5'};
+ my $lservicemd5 = $linkinfo->{'lservicemd5'};
+
if ($BSConfig::nosharedtrees && $srcmd5 ne $emptysrcmd5) {
# copy the tree
my $treedir = "$treesdir/$projid/$packid";
if (! -e "$treedir/$srcmd5-MD5SUMS") {
my $ltreedir = "$treesdir/$lprojid/$packid";
$ltreedir = "$srcrep/$packid" if $BSConfig::nosharedtrees == 2 && ! -e "$ltreedir/$srcmd5-MD5SUMS";
- if (-e "$ltreedir/$srcmd5-MD5SUMS") {
- my $meta = readstr("$ltreedir/$srcmd5-MD5SUMS");
- mkdir_p($treedir);
- writestr("$uploaddir/$$", "$treedir/$srcmd5-MD5SUMS", $meta);
- } else {
+
+ # try to copy all known tree files, otherwise this might trigger
+ # avoidable runservice calls. Goal: avoid all avoidable runservices.
+ copytreefile($treedir, $lprojid, $packid, $lsrcmd5);
+ copytreefile($treedir, $lprojid, $packid, $lservicemd5);
+ copytreefile($treedir, $lprojid, $packid, $xservicemd5);
+
+ copytreefile($treedir, $lprojid, $packid, $srcmd5);
+ if (! -e "$ltreedir/$srcmd5-MD5SUMS") {
addmeta($projid, $packid, $files); # last resort...
}
}
--
Daniel Gollub
Linux Consultant & Developer
Tel.: +49-160 47 73 970
Mail: gollub@b1-systems.de
B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org
To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Daniel Gollub
Robert Schiele