Hello, On Monday, December 19, 2011 11:21:31 AM Robert Schiele wrote: [...]

So, to sum this up: What I observed in 5-7 seems to be a bug that the source service results are not always copied correctly, picking an outdated result for copying instead of the most recent one. Especially the fact that the problem even persists for newly created projects makes me a bit nervous about the reliability here.

Anyone with an idea, what could be causing that, which places in the sources I should focus on for searching for the cause? Alternatively does someone have a good idea for a suitable workaround that is smarter than my current approach to just explicitly copy in all packages that suffer from this behavior? [...]

reproduced the issue and it seems to be that not all meta data gets copied to the linking project. handleservice and other are looking for xservicemd5 metadata and others. Since it does not found those meta data it triggers a source service run. But the source files actually got correctly copied, so the source service run could and should be avoided. I made a pull request for a patch which copies all known meta data: https://github.com/openSUSE/open-build-service/pull/23 This patch only avoids avoidable serviceruns, e.g. if a linkedproject=all get created. But there is still at least one open issue or thing to discuss we have to look into: - retrigger of servicerun of the origin project causes that all linkedprojects trigger a servicerun: this might could be avoided as well by coping tree meta data and sources to linkedprojects instead of doing multiple sourceservice runs. - other cases which might be checked for similiar linkedproject issues? Best Regards, Daniel --- commit cd82c3d903fb2a4e8eba3e1ff9aced46edf7e53c Author: Daniel Gollub <gollub@b1-systems.de> Date: Fri Jan 20 09:10:13 2012 +0100 [backend] copy all known tree meta data Commit 97c1d6c61 introduce only a partial copy of the required meta data. This commit copies additionally to the srcmd5 metadata, also the meta data of: lsrcmd5, xservicemd5, lservicemd5 - if available. This is required for linkproject=all projects to avoid redudant source service triggers. As the meta data did not get copied, handleservice always triggered a source service run for linkedprojects. Even when the sources already got copied with copyfiles(). diff --git a/src/backend/bs_srcserver b/src/backend/bs_srcserver index d82fff2..902112d 100755 --- a/src/backend/bs_srcserver +++ b/src/backend/bs_srcserver @@ -680,6 +680,20 @@ sub copyfiles { } } +sub copytreefile { + my ($treedir, $lprojid, $packid, $md5) = @_; + return unless $md5; + + my $ltreedir = "$treesdir/$lprojid/$packid"; + $ltreedir = "$srcrep/$packid" if $BSConfig::nosharedtrees == 2 && ! -e "$ltreedir/$md5-MD5SUMS"; + + if (-e "$ltreedir/$md5-MD5SUMS") { + my $meta = readstr("$ltreedir/$md5-MD5SUMS"); + mkdir_p($treedir); + writestr("$uploaddir/$$", "$treedir/$md5-MD5SUMS", $meta); + } +} + sub getrev_git { my ($projid, $packid, $rev) = @_; my $gitdir = "$projectsdir/$projid.pkg/$packid.git"; @@ -723,20 +737,32 @@ sub getrev { if ($lrev) { # make sure that we may access the sources of this package checksourceaccess($lprojid, $packid); - my $files = lsrev($lrev); + + my $linkinfo = {}; + my $files = lsrev($lrev, $linkinfo); + copyfiles($projid, $packid, $lprojid, $packid, $files); + my $srcmd5 = $lrev->{'srcmd5'}; + my $lsrcmd5 = $linkinfo->{'lsrcmd5'}; + my $xservicemd5 = $linkinfo->{'xservicemd5'}; + my $lservicemd5 = $linkinfo->{'lservicemd5'}; + if ($BSConfig::nosharedtrees && $srcmd5 ne $emptysrcmd5) { # copy the tree my $treedir = "$treesdir/$projid/$packid"; if (! -e "$treedir/$srcmd5-MD5SUMS") { my $ltreedir = "$treesdir/$lprojid/$packid"; $ltreedir = "$srcrep/$packid" if $BSConfig::nosharedtrees == 2 && ! -e "$ltreedir/$srcmd5-MD5SUMS"; - if (-e "$ltreedir/$srcmd5-MD5SUMS") { - my $meta = readstr("$ltreedir/$srcmd5-MD5SUMS"); - mkdir_p($treedir); - writestr("$uploaddir/$$", "$treedir/$srcmd5-MD5SUMS", $meta); - } else { + + # try to copy all known tree files, otherwise this might trigger + # avoidable runservice calls. Goal: avoid all avoidable runservices. + copytreefile($treedir, $lprojid, $packid, $lsrcmd5); + copytreefile($treedir, $lprojid, $packid, $lservicemd5); + copytreefile($treedir, $lprojid, $packid, $xservicemd5); + + copytreefile($treedir, $lprojid, $packid, $srcmd5); + if (! -e "$ltreedir/$srcmd5-MD5SUMS") { addmeta($projid, $packid, $files); # last resort... } } -- Daniel Gollub Linux Consultant & Developer Tel.: +49-160 47 73 970 Mail: gollub@b1-systems.de B1 Systems GmbH Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org