[opensuse-buildservice] build-compare forces republish for new commit and new rpm

Olaf Hering

12 Nov 2014 12 Nov '14

08:57

I wonder why its important to republish a package which just had a new commit. If the resulting binary package is identical there is no need to republish, just because the rpm %RELEASE string happens to differ. Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages. Thats the (untested) change I propose to handle both cases and leave the previous package on the mirror: Index: functions.sh =================================================================== --- functions.sh (revision 222a61b1a991885b87a3e16ef27c22bb) +++ functions.sh (working copy) @@ -59,7 +59,7 @@ QF="$QF %{VENDOR} %{DISTRIBUTION} %{DISTURL}" QF="$QF %{LICENSE} %{LICENSE}\\n" QF="$QF %{GROUP} %{URL} %{EXCLUDEARCH} %{EXCLUDEOS} %{EXCLUSIVEARCH}\\n" - QF="$QF %{EXCLUSIVEOS} %{RPMVERSION} %{PLATFORM}\\n" + QF="$QF %{EXCLUSIVEOS} %{PLATFORM}\\n" QF="$QF %{PAYLOADFORMAT} %{PAYLOADCOMPRESSOR} %{PAYLOADFLAGS}\\n" @@ -91,13 +91,6 @@ # Remember to quote the . which is in release release1=`$RPM --qf "%{RELEASE}" "$oldrpm"|sed -e 's/\./\\\./g'` release2=`$RPM --qf "%{RELEASE}" "$newrpm"|sed -e 's/\./\\\./g'` - # This might happen with a forced rebuild of factory - if [ "${release1%.*}" != "${release2%.*}" ] ; then - echo "release prefix mismatch" - if test -z "$check_all"; then - return 1 - fi - fi check_provides $oldrpm $release1 > $file1 check_provides $newrpm $release2 > $file2 Olaf -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Show replies by date

Adrian Schröter

12 Nov 12 Nov

09:05

On Mittwoch, 12. November 2014, 09:57:13 wrote Olaf Hering:

...

I wonder why its important to republish a package which just had a new commit. If the resulting binary package is identical there is no need to republish, just because the rpm %RELEASE string happens to differ.

Just a guess, we have packages with multiple spec files, but which rely on %versiion-CI_CNT . If one would build and the other skipped the dependencies can not be met.

...

Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages.

It is a matter of taste, it is in any case cleaner to have only rpm packages inside of a distro which are build all by the same rpm version.

...

Thats the (untested) change I propose to handle both cases and leave the previous package on the mirror:

Index: functions.sh =================================================================== --- functions.sh (revision 222a61b1a991885b87a3e16ef27c22bb) +++ functions.sh (working copy) @@ -59,7 +59,7 @@ QF="$QF %{VENDOR} %{DISTRIBUTION} %{DISTURL}" QF="$QF %{LICENSE} %{LICENSE}\\n" QF="$QF %{GROUP} %{URL} %{EXCLUDEARCH} %{EXCLUDEOS} %{EXCLUSIVEARCH}\\n" - QF="$QF %{EXCLUSIVEOS} %{RPMVERSION} %{PLATFORM}\\n" + QF="$QF %{EXCLUSIVEOS} %{PLATFORM}\\n" QF="$QF %{PAYLOADFORMAT} %{PAYLOADCOMPRESSOR} %{PAYLOADFLAGS}\\n"

@@ -91,13 +91,6 @@ # Remember to quote the . which is in release release1=`$RPM --qf "%{RELEASE}" "$oldrpm"|sed -e 's/\./\\\./g'` release2=`$RPM --qf "%{RELEASE}" "$newrpm"|sed -e 's/\./\\\./g'` - # This might happen with a forced rebuild of factory - if [ "${release1%.*}" != "${release2%.*}" ] ; then - echo "release prefix mismatch" - if test -z "$check_all"; then - return 1 - fi - fi

check_provides $oldrpm $release1 > $file1 check_provides $newrpm $release2 > $file2

Olaf

-- Adrian Schroeter email: adrian@suse.de SUSE LINUX GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Andreas Schwab

09:12

New subject: [opensuse-buildservice] Re: build-compare forces republish for new commit and new rpm

Adrian Schröter writes:

...

On Mittwoch, 12. November 2014, 09:57:13 wrote Olaf Hering:

...
Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages.

It is a matter of taste, it is in any case cleaner to have only rpm packages inside of a distro which are build all by the same rpm version.

This is only guaranteed if you force a rebuild of all packages after a version update of rpm. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Adrian Schröter

09:13

New subject: [opensuse-buildservice] Re: build-compare forces republish for new commit and new rpm

On Mittwoch, 12. November 2014, 10:12:47 wrote Andreas Schwab:

...

Adrian Schröter writes:

...
On Mittwoch, 12. November 2014, 09:57:13 wrote Olaf Hering:

...
Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages.

It is a matter of taste, it is in any case cleaner to have only rpm packages inside of a distro which are build all by the same rpm version.

This is only guaranteed if you force a rebuild of all packages after a version update of rpm.

complete different topic, but we use to build final distros with transient build mode anyway. And rpm-build is a required base package. -- Adrian Schroeter email: adrian@suse.de SUSE LINUX GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Andreas Schwab

09:38

New subject: [opensuse-buildservice] Re: build-compare forces republish for new commit and new rpm

Adrian Schröter writes:

...

On Mittwoch, 12. November 2014, 10:12:47 wrote Andreas Schwab:

...
Adrian Schröter writes:

...
On Mittwoch, 12. November 2014, 09:57:13 wrote Olaf Hering:

...
Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages.

It is a matter of taste, it is in any case cleaner to have only rpm packages inside of a distro which are build all by the same rpm version.

This is only guaranteed if you force a rebuild of all packages after a version update of rpm.

complete different topic, but we use to build final distros with transient build mode anyway. And rpm-build is a required base package.

Do you mean "used to build"? That would mean we no longer do that. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Olaf Hering

09:16

On Wed, Nov 12, Adrian Schröter wrote:

...

On Mittwoch, 12. November 2014, 09:57:13 wrote Olaf Hering:

...
I wonder why its important to republish a package which just had a new commit. If the resulting binary package is identical there is no need to republish, just because the rpm %RELEASE string happens to differ.

Just a guess, we have packages with multiple spec files, but which rely on %versiion-CI_CNT . If one would build and the other skipped the dependencies can not be met.

What packages? Do you mean the kernel packages?

...

...
Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages.

It is a matter of taste, it is in any case cleaner to have only rpm packages inside of a distro which are build all by the same rpm version.

For Factory etc it wont matter, and once a new release is branched the version will remain the same. Olaf -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Stephan Kulow

10:14

On 12.11.2014 09:57, Olaf Hering wrote:

...

I wonder why its important to republish a package which just had a new commit. If the resulting binary package is identical there is no need to republish, just because the rpm %RELEASE string happens to differ.

The list of checked tags is not complete and so you never know what the commit was about.

...

Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages.

We don't test all header values so you just don't know if the rpm is complete or not. And to avoid that problem we flush the build-compare for new rpm versions. I think you're overoptimizing. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Olaf Hering

14 Nov 14 Nov

18:05

On Wed, Nov 12, Stephan Kulow wrote:

...

On 12.11.2014 09:57, Olaf Hering wrote:

...
I wonder why its important to republish a package which just had a new commit. If the resulting binary package is identical there is no need to republish, just because the rpm %RELEASE string happens to differ.

The list of checked tags is not complete and so you never know what the commit was about.

If the source changed there is some URL in some tag. The release itself looks meaningless. But there are likely packages that rely on a certain release number.

...

...
Similar for a new rpm version in the build tree. Why would that require a republish of the entire tree? I mean the install stack has to cope with old rpm packages anyway, no matter which version of rpm was used to create these packages.

We don't test all header values so you just don't know if the rpm is complete or not. And to avoid that problem we flush the build-compare for new rpm versions.

What do you mean with "if the rpm is complete"? There is an existing package, and its fine and usable. If the new rpm does something new to a package, who cares? If its important a to-be-added check in build-compare may catch it.

...

I think you're overoptimizing.

Just trying to reduce the noise. Olaf -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Stephan Kulow

15 Nov 15 Nov

10:22

Am 14.11.2014 19:05, schrieb Olaf Hering:

...

If the source changed there is some URL in some tag. The release itself looks meaningless. But there are likely packages that rely on a certain release number. So your use case is a commit that didn't change the sources but just the checkin counter? I'm worried about the kernels where kernel-syms requires that all kernel flavors have the same checkin counter.

...

What do you mean with "if the rpm is complete"? There is an existing package, and its fine and usable. If the new rpm does something new to a package, who cares? If its important a to-be-added check in build-compare may catch it. Well, me as factory manager cares that that Factory actually gets new features of new rpm versions. It might be an important improvement.

Greetings, Stephan -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Olaf Hering

17 Nov 17 Nov

08:48

On Sat, Nov 15, Stephan Kulow wrote:

...

Am 14.11.2014 19:05, schrieb Olaf Hering:

...
If the source changed there is some URL in some tag. The release itself looks meaningless. But there are likely packages that rely on a certain release number. So your use case is a commit that didn't change the sources but just the checkin counter?

I think the ultimate check if the sources change is DISTURL. How is the release number handled anyway? I'm using cicount="copy" (for which docu is hard to find via google). Last week I noticed imlib2 was republished after a new rpm checkin in PMBS 11.4: [ 154s] compare /.build.oldpackages/imlib2-1.4.5-49.1.src.rpm /usr/src/packages/SRPMS/imlib2-1.4.5-51.1.src.rpm [ 154s] release prefix mismatch The source in OBS:graphics/imlib2 was unchanged. Why is it two releases ahead now? Does the _link update trigger a source change? But OBS also has rev 51 since the last checkin.

...

I'm worried about the kernels where kernel-syms requires that all kernel flavors have the same checkin counter.

The current code is bogus and not documented at all. What is that check supposed to catch? 94 # This might happen with a forced rebuild of factory 95 if [ "${release1%.*}" != "${release2%.*}" ] ; then 96 echo "release prefix mismatch" 97 if test -z "$check_all"; then 98 return 1 99 fi 100 fi To make sure all kernel packages have the same RLEASE string they should be handled here. Either with a wildcard "kernel-*" like this: Index: functions.sh =================================================================== --- functions.sh (revision cf456e3c2908ec9d5cc7332940bb1ae1) +++ functions.sh (working copy) @@ -91,12 +91,19 @@ # Remember to quote the . which is in release release1=$($RPM --qf "%{RELEASE}" "$oldrpm"|sed -e 's/\./\\./g') release2=$($RPM --qf "%{RELEASE}" "$newrpm"|sed -e 's/\./\\./g') - # This might happen with a forced rebuild of factory + # This might happen when?! if [ "${release1%.*}" != "${release2%.*}" ] ; then - echo "release prefix mismatch" - if test -z "$check_all"; then - return 1 - fi + case $($RPM --qf '%{NAME}' "$newrpm") + kernel-*) + # Make sure all kernel packages have the same %RELEASE + echo "release prefix mismatch" + if test -z "$check_all"; then + return 1 + fi + ;; + # Everything other package is allowed to have a different RELEASE + *) ;; + esac fi check_provides $oldrpm $release1 > $file1 Or each affected package ships a /usr/lib/rpm/build-compare/release-mismatch.*.conf and the code does something like "grep ^%{NAME}$ release-mismatch.*.conf" to decide if a given package should be republished if the %RELEASE differes. That way only the kernel packages would match, not every package that is named kernel-*.rpm.

...

...
What do you mean with "if the rpm is complete"? There is an existing package, and its fine and usable. If the new rpm does something new to a package, who cares? If its important a to-be-added check in build-compare may catch it. Well, me as factory manager cares that that Factory actually gets new features of new rpm versions. It might be an important improvement.

I will leave that alone for now because it happens rarely. Olaf -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Marcus Meissner

08:53

On Mon, Nov 17, 2014 at 09:48:37AM +0100, Olaf Hering wrote:

...

On Sat, Nov 15, Stephan Kulow wrote:

...
Am 14.11.2014 19:05, schrieb Olaf Hering:

...
If the source changed there is some URL in some tag. The release itself looks meaningless. But there are likely packages that rely on a certain release number. So your use case is a commit that didn't change the sources but just the checkin counter?

I think the ultimate check if the sources change is DISTURL. How is the release number handled anyway? I'm using cicount="copy" (for which docu is hard to find via google). Last week I noticed imlib2 was republished after a new rpm checkin in PMBS 11.4:

[ 154s] compare /.build.oldpackages/imlib2-1.4.5-49.1.src.rpm /usr/src/packages/SRPMS/imlib2-1.4.5-51.1.src.rpm [ 154s] release prefix mismatch

The source in OBS:graphics/imlib2 was unchanged. Why is it two releases ahead now? Does the _link update trigger a source change? But OBS also has rev 51 since the last checkin.

I changed the meta data of imlib2 at least (adding SLES12), but not the sources. Ciao, MArcus -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Olaf Hering

09:04

On Mon, Nov 17, Marcus Meissner wrote:

...

On Mon, Nov 17, 2014 at 09:48:37AM +0100, Olaf Hering wrote:

...
On Sat, Nov 15, Stephan Kulow wrote:

...
Am 14.11.2014 19:05, schrieb Olaf Hering:

...
If the source changed there is some URL in some tag. The release itself looks meaningless. But there are likely packages that rely on a certain release number. So your use case is a commit that didn't change the sources but just the checkin counter?

I think the ultimate check if the sources change is DISTURL. How is the release number handled anyway? I'm using cicount="copy" (for which docu is hard to find via google). Last week I noticed imlib2 was republished after a new rpm checkin in PMBS 11.4:

[ 154s] compare /.build.oldpackages/imlib2-1.4.5-49.1.src.rpm /usr/src/packages/SRPMS/imlib2-1.4.5-51.1.src.rpm [ 154s] release prefix mismatch

The source in OBS:graphics/imlib2 was unchanged. Why is it two releases ahead now? Does the _link update trigger a source change? But OBS also has rev 51 since the last checkin.

I changed the meta data of imlib2 at least (adding SLES12), but not the sources.

This appearently did not cause a rebuild of it. Not sure why the release jumped in PMBS. But the point is really why it would matter in a non-kernel package. Olaf -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Andreas Schwab

09:42

New subject: [opensuse-buildservice] Re: build-compare forces republish for new commit and new rpm

Olaf Hering writes:

...

I think the ultimate check if the sources change is DISTURL. How is the release number handled anyway? I'm using cicount="copy" (for which docu is hard to find via google). Last week I noticed imlib2 was republished after a new rpm checkin in PMBS 11.4:

[ 154s] compare /.build.oldpackages/imlib2-1.4.5-49.1.src.rpm /usr/src/packages/SRPMS/imlib2-1.4.5-51.1.src.rpm [ 154s] release prefix mismatch

The source in OBS:graphics/imlib2 was unchanged.

Doesn't look like. 2014-11-03 13:22:37 f7643ef68995631f0a7025e4c066d984 31 1.4.5-49.1 2014-11-07 20:22:12 3e552432d6989973c8ceaaeb2f33ff22 32 1.4.5-51.1 Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Stephan Kulow

09:52

New subject: [opensuse-buildservice] Re: build-compare forces republish for new commit and new rpm

Am 17.11.2014 um 10:42 schrieb Andreas Schwab:

...

Olaf Hering writes:

...
I think the ultimate check if the sources change is DISTURL. How is the release number handled anyway? I'm using cicount="copy" (for which docu is hard to find via google). Last week I noticed imlib2 was republished after a new rpm checkin in PMBS 11.4:

[ 154s] compare /.build.oldpackages/imlib2-1.4.5-49.1.src.rpm /usr/src/packages/SRPMS/imlib2-1.4.5-51.1.src.rpm [ 154s] release prefix mismatch

The source in OBS:graphics/imlib2 was unchanged.

Doesn't look like.

2014-11-03 13:22:37 f7643ef68995631f0a7025e4c066d984 31 1.4.5-49.1 2014-11-07 20:22:12 3e552432d6989973c8ceaaeb2f33ff22 32 1.4.5-51.1

The sources changed as the _link was updated - but the verifymd5 didn't change: coolo@gertrude#imlib2>osc api "/source/graphics/imlib2?view=info&rev=3e552432d6989973c8ceaaeb2f33ff22" <sourceinfo package="imlib2" rev="3e552432d6989973c8ceaaeb2f33ff22" srcmd5="3e552432d6989973c8ceaaeb2f33ff22" verifymd5="eafef7de147eda854be00c846e178a8b"> <filename>imlib2.spec</filename> </sourceinfo> coolo@gertrude#imlib2>osc api "/source/graphics/imlib2?view=info&rev=f7643ef68995631f0a7025e4c066d984" <sourceinfo package="imlib2" rev="f7643ef68995631f0a7025e4c066d984" srcmd5="f7643ef68995631f0a7025e4c066d984" verifymd5="eafef7de147eda854be00c846e178a8b"> <filename>imlib2.spec</filename> </sourceinfo> So this shouldn't even have caused a scheduled job. Greetings, Stephan -- Ma muaß weiterkämpfen, kämpfen bis zum Umfalln, a wenn die ganze Welt an Arsch offen hat, oder grad deswegn. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Andreas Schwab

10:19

New subject: [opensuse-buildservice] Re: build-compare forces republish for new commit and new rpm

Stephan Kulow writes:

...

Am 17.11.2014 um 10:42 schrieb Andreas Schwab:

...
Olaf Hering writes:

...
I think the ultimate check if the sources change is DISTURL. How is the release number handled anyway? I'm using cicount="copy" (for which docu is hard to find via google). Last week I noticed imlib2 was republished after a new rpm checkin in PMBS 11.4:

[ 154s] compare /.build.oldpackages/imlib2-1.4.5-49.1.src.rpm /usr/src/packages/SRPMS/imlib2-1.4.5-51.1.src.rpm [ 154s] release prefix mismatch

The source in OBS:graphics/imlib2 was unchanged.

Doesn't look like.

2014-11-03 13:22:37 f7643ef68995631f0a7025e4c066d984 31 1.4.5-49.1 2014-11-07 20:22:12 3e552432d6989973c8ceaaeb2f33ff22 32 1.4.5-51.1

The sources changed as the _link was updated - but the verifymd5 didn't change:

You get a new DISTURL, which is exactly what Olaf wants, see above. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Olaf Hering

18 Nov 18 Nov

13:43

On Wed, Nov 12, Olaf Hering wrote:

...

I wonder why its important to republish a package which just had a new commit. If the resulting binary package is identical there is no need to republish, just because the rpm %RELEASE string happens to differ.

I tried to address this. AFAIK only kernel and kmp packages encode all or parts of %release in directory entries. The new code could handle kernel-* packages, but this part is disabled for several reasons: - kernel signing does not handle build-compare well, only the first stage runs build-compare. But its only needed in the second stage, which does not run it at all. - signing encodes a timestamp into vmlinuz, which would invalidate the new package anyway - the %release may be required by other packages, I have not checked which places would care. It will likely work for kmp packages, they have their own %release independent from any kernel-* package. https://build.opensuse.org/request/show/262176 Olaf -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

3448

Age (days ago)

3454

Last active (days ago)

List overview

Download

15 comments

5 participants

participants (5)

Adrian Schröter
Andreas Schwab
Marcus Meissner
Olaf Hering
Stephan Kulow