[opensuse-buildservice] Patch: drop "Suggests:" and "Recommends:" if rpm does not know them
Hi, I'm running an OBS instance, used amongst others to build stuff for multiple distributions (SLES and RHEL, mostly). Now if I import packages from openSUSE OBS, I often have to patch the spec file. 95% of these patches are like this: +%if 0%{?suse_version} Recommends: not_really_necessary_thing_1 Suggests: totally_unnecessary_thing_2 +%endif I automated this task in build-recipe-spec: If the target's RPM does not know "Recommends:" or "Suggests:", just drop these lines. This is made configurable with the following prjconf Buildflags: dropunknownrpmtags here is the patch for discussion, if this would be interesting for upstream, I'll prepare a proper pull request. diff --git a/build-recipe-spec b/build-recipe-spec index d72a3b7..f92d857 100644 --- a/build-recipe-spec +++ b/build-recipe-spec @@ -85,6 +85,40 @@ recipe_prepare_spec() { test -z "$ABUILD_TARGET" || echo "build target is $ABUILD_TARGET" fi + DROP_UNKNOWN=$(queryconfig --dist "$BUILD_DIST" --configdir "$CONFIG_DIR" --archpath "$BUILD_ARCH" buildflags dropunknownrpmtags) + if [ -n "$DROP_UNKNOWN" ]; then + declare -i SUSE_VERSION=$(chroot $BUILD_ROOT rpm --eval '%{?suse_version}' 2>/dev/null) + if ! [ -e "$BUILD_ROOT/usr/bin/rpmbuild" ]; then + echo "NOTE: no /usr/bin/rpmbuild in target, no check for unnown specfile tags" + elif ((SUSE_VERSION == 0)); then # just assume that every SUSE has patched RPM... + # echo "SUSE_VERSION is zero" + for i in Suggests: Recommends:; do + # if no suggests/recommends, then we don't need to test + grep "^$i" "$BUILD_ROOT/.spec.new" >/dev/null 2>&1 || continue + rm -f "$BUILD_ROOT/.spec.test" + cat > "$BUILD_ROOT/.spec.test" <<-EOF + $i foo + Name: foo + Version: 0 + Release: 0 + Summary: foo + License: WTFPL + + %description + foo + EOF + UNKNOWN="`chroot $BUILD_ROOT /bin/rpmbuild --nobuild /.spec.test 2>&1`" + # echo "found unknown tag '$UNKNOWN'" + RES=$? + rm -f "$BUILD_ROOT/.spec.test" + test $RES = 0 && continue + # echo "nobuild result for $i $RES" + echo "WARNING: removing unknown tag on non-SUSE distro: '$i'" + sed -i -e "/^$i/d" "$BUILD_ROOT/.spec.new" + done + fi + fi + # report specfile changes if test -f $BUILD_ROOT/.spec.new ; then if ! cmp -s $BUILD_ROOT$TOPDIR/SOURCES/$RECIPEFILE $BUILD_ROOT/.spec.new ; then Have fun, seife -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Sat, Nov 14, 2015 at 7:12 AM, Stefan Seyfried <stefan.seyfried@gmail.com> wrote:
Hi,
I'm running an OBS instance, used amongst others to build stuff for multiple distributions (SLES and RHEL, mostly).
Now if I import packages from openSUSE OBS, I often have to patch the spec file. 95% of these patches are like this:
+%if 0%{?suse_version} Recommends: not_really_necessary_thing_1 Suggests: totally_unnecessary_thing_2 +%endif
I automated this task in build-recipe-spec: If the target's RPM does not know "Recommends:" or "Suggests:", just drop these lines.
This is made configurable with the following prjconf Buildflags: dropunknownrpmtags
here is the patch for discussion, if this would be interesting for upstream, I'll prepare a proper pull request.
Your patch assumes that only SUSE would support weak dependencies. However this is not the case. While it is true SUSE's patched RPM has had it for quite a while, it was upstreamed into mainline RPM for RPM 4.12. In the current supported distribution selection list, Fedora 21 and newer (as well as Mageia 5 and newer) have support for it. And if they are building with debbuild (which can use RPM spec to produce native Debian packages), the tags are supported as well for all Debian targets. My suggestion: add a check for the BuildEngine so that it doesn't do anything when debbuild is being used and also ask RPM what version it is if it isn't SUSE or debbuild and evaluate based on that. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am 14.11.2015 um 14:29 schrieb Neal Gompa:
On Sat, Nov 14, 2015 at 7:12 AM, Stefan Seyfried <stefan.seyfried@gmail.com> wrote:
Hi,
I'm running an OBS instance, used amongst others to build stuff for multiple distributions (SLES and RHEL, mostly).
Now if I import packages from openSUSE OBS, I often have to patch the spec file. 95% of these patches are like this:
+%if 0%{?suse_version} Recommends: not_really_necessary_thing_1 Suggests: totally_unnecessary_thing_2 +%endif
I automated this task in build-recipe-spec: If the target's RPM does not know "Recommends:" or "Suggests:", just drop these lines.
This is made configurable with the following prjconf Buildflags: dropunknownrpmtags
here is the patch for discussion, if this would be interesting for upstream, I'll prepare a proper pull request.
Your patch assumes that only SUSE would support weak dependencies.
No, it does not. The only thing that it does assume (maybe wrong, this comes from the initial version which was not configurable via prjconf) is, that every SUSE version's rpm supports Suggests: and Recommends.
However this is not the case. While it is true SUSE's patched RPM has had it for quite a while, it was upstreamed into mainline RPM for RPM 4.12. In the current supported distribution selection list, Fedora 21 and newer (as well as Mageia 5 and newer) have support for it.
The patch does the following (pseudo code): if ! building_for_suse() and buildflags[dropunknownrpmtags]; then for i in Suggests: Recommends; do if ! fancy_code_to_check_if_targets_rpm_supports($i); then remove_from_specfile($i); fi done fi The "building_for_suse()" could actually go away. It was originally there to make sure to not break things that were working before.
And if they are building with debbuild (which can use RPM spec to produce native Debian packages), the tags are supported as well for all Debian targets.
And? Does my patch change this? I can't imagine how, but then I never tried to build debian packages from rpm specs.
My suggestion: add a check for the BuildEngine so that it doesn't do anything when debbuild is being used and also ask RPM what version it is if it isn't SUSE or debbuild and evaluate based on that.
Version checks are unreliable. I check if the target rpm knows the tag, which is much better. I really don't like magic version numbers hard coded into this. What if someone uses a prerelease of rpm with an older number, but the feature supported? What if a distribution decides to backport one of those tags to their released product? My code does catch all those cases. And even if debbuild would use the code in "build-recipe-spec", then I still believe it could just work anyway. Best regards, seife -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 11/14/2015 7:12 AM, Stefan Seyfried wrote:
Hi,
I'm running an OBS instance, used amongst others to build stuff for multiple distributions (SLES and RHEL, mostly).
Now if I import packages from openSUSE OBS, I often have to patch the spec file. 95% of these patches are like this:
+%if 0%{?suse_version} Recommends: not_really_necessary_thing_1 Suggests: totally_unnecessary_thing_2 +%endif
I automated this task in build-recipe-spec: If the target's RPM does not know "Recommends:" or "Suggests:", just drop these lines.
This is made configurable with the following prjconf Buildflags: dropunknownrpmtags
here is the patch for discussion, if this would be interesting for upstream, I'll prepare a proper pull request.
diff --git a/build-recipe-spec b/build-recipe-spec index d72a3b7..f92d857 100644 --- a/build-recipe-spec +++ b/build-recipe-spec @@ -85,6 +85,40 @@ recipe_prepare_spec() { test -z "$ABUILD_TARGET" || echo "build target is $ABUILD_TARGET" fi
+ DROP_UNKNOWN=$(queryconfig --dist "$BUILD_DIST" --configdir "$CONFIG_DIR" --archpath "$BUILD_ARCH" buildflags dropunknownrpmtags) + if [ -n "$DROP_UNKNOWN" ]; then + declare -i SUSE_VERSION=$(chroot $BUILD_ROOT rpm --eval '%{?suse_version}' 2>/dev/null) + if ! [ -e "$BUILD_ROOT/usr/bin/rpmbuild" ]; then + echo "NOTE: no /usr/bin/rpmbuild in target, no check for unnown specfile tags" + elif ((SUSE_VERSION == 0)); then # just assume that every SUSE has patched RPM... + # echo "SUSE_VERSION is zero" + for i in Suggests: Recommends:; do + # if no suggests/recommends, then we don't need to test + grep "^$i" "$BUILD_ROOT/.spec.new" >/dev/null 2>&1 || continue + rm -f "$BUILD_ROOT/.spec.test" + cat > "$BUILD_ROOT/.spec.test" <<-EOF + $i foo + Name: foo + Version: 0 + Release: 0 + Summary: foo + License: WTFPL + + %description + foo + EOF + UNKNOWN="`chroot $BUILD_ROOT /bin/rpmbuild --nobuild /.spec.test 2>&1`" + # echo "found unknown tag '$UNKNOWN'" + RES=$? + rm -f "$BUILD_ROOT/.spec.test" + test $RES = 0 && continue + # echo "nobuild result for $i $RES" + echo "WARNING: removing unknown tag on non-SUSE distro: '$i'" + sed -i -e "/^$i/d" "$BUILD_ROOT/.spec.new" + done + fi + fi + # report specfile changes if test -f $BUILD_ROOT/.spec.new ; then if ! cmp -s $BUILD_ROOT$TOPDIR/SOURCES/$RECIPEFILE $BUILD_ROOT/.spec.new ; then
Have fun,
seife
Is the original data lost or just not used to build? If you had these tags in a spec on obs, what is in the spec of the resulting .src.rpm? I think ideally, you want to output a .src.rpm that will build on the target outside of obs, so *something* has to happen to them rather than copy the original spec verbatim. But you also don't want to lose data, and so the removed tags should be converted into comments for those output src.rpm. Maybe even echo commands to make the comment visible at install time. But it's a tiny point. I wish obs had this since 5 years ago or really since the first time obs used an rpm that was newer than any build target. Same for new convenience macros, though the action isn't usually as easy as dropping the new/unknown macro. But it is often possible to replace the new convenience macro with a lower denominator equivalent, or by including a copy of the macro itself. Thank you for making this. -- bkw -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Monday 16 November 2015, 13:10:21 wrote Brian K. White:
On 11/14/2015 7:12 AM, Stefan Seyfried wrote:
Hi,
I'm running an OBS instance, used amongst others to build stuff for multiple distributions (SLES and RHEL, mostly).
Now if I import packages from openSUSE OBS, I often have to patch the spec file. 95% of these patches are like this:
+%if 0%{?suse_version} Recommends: not_really_necessary_thing_1 Suggests: totally_unnecessary_thing_2 +%endif
I automated this task in build-recipe-spec: If the target's RPM does not know "Recommends:" or "Suggests:", just drop these lines.
This is made configurable with the following prjconf Buildflags: dropunknownrpmtags
here is the patch for discussion, if this would be interesting for upstream, I'll prepare a proper pull request.
diff --git a/build-recipe-spec b/build-recipe-spec index d72a3b7..f92d857 100644 --- a/build-recipe-spec +++ b/build-recipe-spec @@ -85,6 +85,40 @@ recipe_prepare_spec() { test -z "$ABUILD_TARGET" || echo "build target is $ABUILD_TARGET" fi
+ DROP_UNKNOWN=$(queryconfig --dist "$BUILD_DIST" --configdir "$CONFIG_DIR" --archpath "$BUILD_ARCH" buildflags dropunknownrpmtags) + if [ -n "$DROP_UNKNOWN" ]; then + declare -i SUSE_VERSION=$(chroot $BUILD_ROOT rpm --eval '%{?suse_version}' 2>/dev/null) + if ! [ -e "$BUILD_ROOT/usr/bin/rpmbuild" ]; then + echo "NOTE: no /usr/bin/rpmbuild in target, no check for unnown specfile tags" + elif ((SUSE_VERSION == 0)); then # just assume that every SUSE has patched RPM... + # echo "SUSE_VERSION is zero" + for i in Suggests: Recommends:; do + # if no suggests/recommends, then we don't need to test + grep "^$i" "$BUILD_ROOT/.spec.new" >/dev/null 2>&1 || continue + rm -f "$BUILD_ROOT/.spec.test" + cat > "$BUILD_ROOT/.spec.test" <<-EOF + $i foo + Name: foo + Version: 0 + Release: 0 + Summary: foo + License: WTFPL + + %description + foo + EOF + UNKNOWN="`chroot $BUILD_ROOT /bin/rpmbuild --nobuild /.spec.test 2>&1`" + # echo "found unknown tag '$UNKNOWN'" + RES=$? + rm -f "$BUILD_ROOT/.spec.test" + test $RES = 0 && continue + # echo "nobuild result for $i $RES" + echo "WARNING: removing unknown tag on non-SUSE distro: '$i'" + sed -i -e "/^$i/d" "$BUILD_ROOT/.spec.new" + done + fi + fi + # report specfile changes if test -f $BUILD_ROOT/.spec.new ; then if ! cmp -s $BUILD_ROOT$TOPDIR/SOURCES/$RECIPEFILE $BUILD_ROOT/.spec.new ; then
Have fun,
seife
Is the original data lost or just not used to build?
it stays untouched in source repository.
If you had these tags in a spec on obs, what is in the spec of the resulting .src.rpm?
the edited spec file will get packaged.
I think ideally, you want to output a .src.rpm that will build on the target outside of obs, so *something* has to happen to them rather than copy the original spec verbatim. But you also don't want to lose data, and so the removed tags should be converted into comments for those output src.rpm. Maybe even echo commands to make the comment visible at install time.
If you use the src.rpm for your target platform you will get the right spec file.
But it's a tiny point.
I wish obs had this since 5 years ago or really since the first time obs used an rpm that was newer than any build target.
Same for new convenience macros, though the action isn't usually as easy as dropping the new/unknown macro. But it is often possible to replace the new convenience macro with a lower denominator equivalent, or by including a copy of the macro itself.
OBS is not providing our updating rpm, the distribution content is doing this. So, if you want to have macro definitions in old distros, you need to provide them either via package or prjconf.
Thank you for making this.
-- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 11/17/2015 2:04 AM, Adrian Schröter wrote:
On Monday 16 November 2015, 13:10:21 wrote Brian K. White:
I wish obs had this since 5 years ago or really since the first time obs used an rpm that was newer than any build target.
Same for new convenience macros, though the action isn't usually as easy as dropping the new/unknown macro. But it is often possible to replace the new convenience macro with a lower denominator equivalent, or by including a copy of the macro itself.
OBS is not providing our updating rpm, the distribution content is doing this. So, if you want to have macro definitions in old distros, you need to provide them either via package or prjconf.
I currently do it with prjconf, but then the resulting src.rpm does not build on a real target machine. To me that is the definition of a broken src.rpm Or I edit the spec, but then it's harder to track a devel or otherwise upstream project. A package which could just be a buildrequires sounds like a good idea if there is a way to supply rpm macros without conflicting with a distro maintained file. Thanks. -- bkw -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hi Brian, On 16.11.2015 19:10, Brian K. White wrote:
On 11/14/2015 7:12 AM, Stefan Seyfried wrote:
Hi,
I'm running an OBS instance, used amongst others to build stuff for multiple distributions (SLES and RHEL, mostly).
Now if I import packages from openSUSE OBS, I often have to patch the spec file. 95% of these patches are like this:
+%if 0%{?suse_version} Recommends: not_really_necessary_thing_1 Suggests: totally_unnecessary_thing_2 +%endif
I automated this task in build-recipe-spec: If the target's RPM does not know "Recommends:" or "Suggests:", just drop these lines.
This is made configurable with the following prjconf Buildflags: dropunknownrpmtags
here is the patch for discussion, if this would be interesting for upstream, I'll prepare a proper pull request.
diff --git a/build-recipe-spec b/build-recipe-spec index d72a3b7..f92d857 100644 --- a/build-recipe-spec +++ b/build-recipe-spec @@ -85,6 +85,40 @@ recipe_prepare_spec() { test -z "$ABUILD_TARGET" || echo "build target is $ABUILD_TARGET" fi
+ DROP_UNKNOWN=$(queryconfig --dist "$BUILD_DIST" --configdir "$CONFIG_DIR" --archpath "$BUILD_ARCH" buildflags dropunknownrpmtags) + if [ -n "$DROP_UNKNOWN" ]; then + declare -i SUSE_VERSION=$(chroot $BUILD_ROOT rpm --eval '%{?suse_version}' 2>/dev/null) + if ! [ -e "$BUILD_ROOT/usr/bin/rpmbuild" ]; then + echo "NOTE: no /usr/bin/rpmbuild in target, no check for unnown specfile tags" + elif ((SUSE_VERSION == 0)); then # just assume that every SUSE has patched RPM... + # echo "SUSE_VERSION is zero" + for i in Suggests: Recommends:; do + # if no suggests/recommends, then we don't need to test + grep "^$i" "$BUILD_ROOT/.spec.new" >/dev/null 2>&1 || continue + rm -f "$BUILD_ROOT/.spec.test" + cat > "$BUILD_ROOT/.spec.test" <<-EOF + $i foo + Name: foo + Version: 0 + Release: 0 + Summary: foo + License: WTFPL + + %description + foo + EOF + UNKNOWN="`chroot $BUILD_ROOT /bin/rpmbuild --nobuild /.spec.test 2>&1`" + # echo "found unknown tag '$UNKNOWN'" + RES=$? + rm -f "$BUILD_ROOT/.spec.test" + test $RES = 0 && continue + # echo "nobuild result for $i $RES" + echo "WARNING: removing unknown tag on non-SUSE distro: '$i'" + sed -i -e "/^$i/d" "$BUILD_ROOT/.spec.new"
in the line above, the spec file is modified...
+ done + fi + fi + # report specfile changes if test -f $BUILD_ROOT/.spec.new ; then if ! cmp -s $BUILD_ROOT$TOPDIR/SOURCES/$RECIPEFILE $BUILD_ROOT/.spec.new ; then
Is the original data lost or just not used to build?
If you had these tags in a spec on obs, what is in the spec of the resulting .src.rpm?
AFAICT, the spec used to build is packaged in the .src.rpm, so if you download the .src.rpm which is built for, say, CentOS-7, it will not have the unknown tag. But that's actually a thing I did not think about: the .src.rpm will then have different contents, depending on the distribution it was built for. That's a "price" I personally am willing to pay :-)
I think ideally, you want to output a .src.rpm that will build on the target outside of obs, so *something* has to happen to them rather than copy the original spec verbatim. But you also don't want to lose data, and so the removed tags should be converted into comments for those output src.rpm. Maybe even echo commands to make the comment visible at install time.
Then we would need to automatically transform these lines into a %post script. I'd rather not try to do that, because it might break more things that it fixes.
But it's a tiny point.
I wish obs had this since 5 years ago or really since the first time obs used an rpm that was newer than any build target.
As I know you are still building for rather old SUSE versions: do you remember if the "Suggests:" / "Recommends:" is available for, say, SLES10? If not, simply removing the SUSE_VERSION check might be better (and make the code simpler).
Same for new convenience macros, though the action isn't usually as easy as dropping the new/unknown macro. But it is often possible to replace the new convenience macro with a lower denominator equivalent, or by including a copy of the macro itself.
Well, these can be solved with prjconf, but unknown spec file syntax unfortunately cannot.
Thank you for making this.
As there seems to be at least *some* interest, I'll put this into a proper pull request on github. For those who consider this useless: unless enabled with Buildflags: dropunknownrpmtags in prjconf, this will do nothing, so you do not need to use this. And I'm not sure if it is a feature which should be enabled on the openSUSE build service at all (or only for a few selected projects, e.g. where projects are building their official packages for different distros on OBS), but it is a feature which is very useful for many private OBS installations, where people need to build for many different distributions, and thist might make it easier for them to share code with the "official" repositories (that was the use case which triggered me to develop this thing). Best regards, seife -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 11/17/2015 8:20 AM, Stefan Seyfried wrote:
where people need to build for many different distributions, and thist might make it easier for them to share code with the "official" repositories (that was the use case which triggered me to develop this thing).
To my mind, that is the use case for the very existence of OBS itself. If you only care about building in one environment, that what do you need OBS for? We already had that 20 years ago. -- bkw -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (4)
-
Adrian Schröter -
Brian K. White -
Neal Gompa -
Stefan Seyfried