Unannounced change of shlib-policy-name-error rpmlint strictness on TW
Hi, a few days ago any violation of shlib-policy-name-error has become a fatal error. This is especially problematic for many packages in the science repository, when packages are built as flavors for e.g. openmpi2/3/4, and the library packages carry a corresponding suffix. E.g.: libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) SONAME: libhdf5.so.103 (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package suffix: 103 This is a case not covered by https://en.opensuse.org/ openSUSE:Shared_library_packaging_policy Can this please be reverted until the policy has been extended to cover these cases? Preferably, rpmlint learns to understand this pattern. Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen home: +49 241 53809034 mobile: +49 151 50412019
On Thu, 14 Jul 2022, Stefan Brüns wrote:
Hi,
a few days ago any violation of shlib-policy-name-error has become a fatal error.
This is especially problematic for many packages in the science repository, when packages are built as flavors for e.g. openmpi2/3/4, and the library packages carry a corresponding suffix. E.g.:
libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) SONAME: libhdf5.so.103 (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package suffix: 103
This is a case not covered by https://en.opensuse.org/ openSUSE:Shared_library_packaging_policy
Can this please be reverted until the policy has been extended to cover these cases? Preferably, rpmlint learns to understand this pattern.
The policy wasn't originally supposed to apply to shared libraries that do not reside in the default runpath which means /usr/lib*/$package/... should have been excempted. Possibly the implementation doesn't reflect that or the implementation is too narrow? IIRC there's also a whitelist(?) - the GCC package excempts itself via rpmlintrc filtering the check ... Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman; HRB 36809 (AG Nuernberg)
On Donnerstag, 14. Juli 2022 14:34:27 CEST Richard Biener wrote:
On Thu, 14 Jul 2022, Stefan Brüns wrote:
Hi,
a few days ago any violation of shlib-policy-name-error has become a fatal error.
This is especially problematic for many packages in the science repository, when packages are built as flavors for e.g. openmpi2/3/4, and the library packages carry a corresponding suffix. E.g.:
libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) SONAME: libhdf5.so.103 (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package suffix: 103
This is a case not covered by https://en.opensuse.org/ openSUSE:Shared_library_packaging_policy
Can this please be reverted until the policy has been extended to cover these cases? Preferably, rpmlint learns to understand this pattern.
The policy wasn't originally supposed to apply to shared libraries that do not reside in the default runpath which means /usr/lib*/$package/... should have been excempted. Possibly the implementation doesn't reflect that or the implementation is too narrow?
IIRC there's also a whitelist(?) - the GCC package excempts itself via rpmlintrc filtering the check ...
Richard.
In the current code I can not find such a generic exemption. Of course one can add an exemption via rpmlintrc. I would prefer not do so: 1. There are probably about 50 source packages in science alone affected by this. 2. Each rpmlintrc entry bears the risk of hiding actual packaging errors. Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen phone: +49 241 53809034 mobile: +49 151 50412019
On 7/14/22 22:04, Richard Biener wrote:
On Thu, 14 Jul 2022, Stefan Brüns wrote:
Hi,
a few days ago any violation of shlib-policy-name-error has become a fatal error.
This is especially problematic for many packages in the science repository, when packages are built as flavors for e.g. openmpi2/3/4, and the library packages carry a corresponding suffix. E.g.:
libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) SONAME: libhdf5.so.103 (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package suffix: 103
This is a case not covered by https://en.opensuse.org/ openSUSE:Shared_library_packaging_policy
Can this please be reverted until the policy has been extended to cover these cases? Preferably, rpmlint learns to understand this pattern.
The policy wasn't originally supposed to apply to shared libraries that do not reside in the default runpath which means /usr/lib*/$package/... should have been excempted. Possibly the implementation doesn't reflect that or the implementation is too narrow?
This is probably worth a bugreport at https://github.com/rpm-software-management/rpmlint because there is a chance that not all developers are reading this list given the development now happens upstream. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On Friday 2022-07-15 07:48, Simon Lees wrote:
libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) SONAME: libhdf5.so.103 (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package suffix: 103
This is a case not covered by https://en.opensuse.org/ openSUSE:Shared_library_packaging_policy
JFYI https://github.com/rpm-software-management/rpmlint/pull/849#issuecomment-113...
On Freitag, 15. Juli 2022 07:48:20 CEST Simon Lees wrote:
This is probably worth a bugreport at https://github.com/rpm-software-management/rpmlint
https://github.com/rpm-software-management/rpmlint/issues/901 -- Stefan Brüns / Bergstraße 21 / 52062 Aachen home: +49 241 53809034 mobile: +49 151 50412019
Thu, 14 Jul 2022 14:16:59 +0200 Stefan Brüns <stefan.bruens@rwth-aachen.de>:
expected package suffix: 103
This is an unhelpful message. Helpful would be: "got rpm package name x1, expected rpm package name y103" I guess in this case a package named "libhdf5-openmpi4-103" would be rejected as well. Olaf
On Donnerstag, 14. Juli 2022 15:09:47 CEST Olaf Hering wrote:
Thu, 14 Jul 2022 14:16:59 +0200 Stefan Brüns <stefan.bruens@rwth-aachen.de>:
expected package suffix: 103
This is an unhelpful message.
Helpful would be: "got rpm package name x1, expected rpm package name y103"
I guess in this case a package named "libhdf5-openmpi4-103" would be rejected as well.
Olaf
Unfortunately, it is not possible to give a definitive name, as there are multiple valid names for a given library. And things get more complicated when you have just one or multiple libraries in a package. As the actual checks comes down to "pkgname.startswith('lib') and pkgname.endswith(suffix)", libhdf5-openmpi4-103 might actually work. I.e. the pkgname here must match "lib.*103". Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen phone: +49 241 53809034 mobile: +49 151 50412019
On Thu, 14 Jul 2022, Olaf Hering wrote:
Thu, 14 Jul 2022 15:40:52 +0200 Stefan Brüns <stefan.bruens@rwth-aachen.de>:
the pkgname here must match "lib.*103".
Correct, this is the helpful message which rpmlint must show, instead of the current crap..
In fact the check computes the exact name that is required and IIRC that was shown at some point ... Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman; HRB 36809 (AG Nuernberg)
Am Montag, 18. Juli 2022, 09:10:09 CEST schrieb Richard Biener:
On Thu, 14 Jul 2022, Olaf Hering wrote:
Thu, 14 Jul 2022 15:40:52 +0200 Stefan Brüns <stefan.bruens@rwth-aachen.de>:
the pkgname here must match "lib.*103".
Correct, this is the helpful message which rpmlint must show, instead of the current crap..
In fact the check computes the exact name that is required and IIRC that was shown at some point ...
Yes, it shows the exact name, that is expected. Here's a related case, that I tripped over this weekend: graphics:OpenColorIO had the wrong so_ver defined, therefore this check is helpful indeed. After fixing, the build of the secondary package (icio_tools) failed, due to Have choice: libOpenColorIO2_0 and libOpenColorIO2_1 Easiest would have been: Prefer: libOpenColorIO2_1 on prjconf, but with obvious issues (propagation into Factory, et.al.). Hence it took the second possibility: OpenColorIO.spec: # BuildIgnore can be removed, when libOCIO2_0 disappeared from repos #!BuildIgnore: libOpenColorIO2_0 Next, the OpenShadingLanguage build broke, due to the dependency to OpenImageIO, which in turn depend in OpenColorIO, which resulted in the "Have Choice" again. My solution to this problem was to add the dependency of OCIO to OSL explicitly, in order to: BuildRequires: cmake(OpenColorIO) # BuildIgnore can be removed, when libOCIO2_0 disappeared from repos #!BuildIgnore: libOpenColorIO2_0 which probably isn't the sanest thing to do. Alternatively, I could add a hard OCIO library dependency to the OIIO library. What do you think? So, while it's nice to catch such issues, handling them can be nasty.
Richard.
Cheers, Pete -- Life without chameleons is possible, but pointless.
On Montag, 18. Juli 2022 12:59:29 CEST Hans-Peter Jansen wrote:
Am Montag, 18. Juli 2022, 09:10:09 CEST schrieb Richard Biener:
On Thu, 14 Jul 2022, Olaf Hering wrote:
Thu, 14 Jul 2022 15:40:52 +0200 Stefan Brüns
<stefan.bruens@rwth-aachen.de>:
the pkgname here must match "lib.*103".
Correct, this is the helpful message which rpmlint must show, instead of the current crap..
In fact the check computes the exact name that is required and IIRC that was shown at some point ...
Yes, it shows the exact name, that is expected.
Here's a related case, that I tripped over this weekend:
graphics:OpenColorIO had the wrong so_ver defined, therefore this check is helpful indeed.
After fixing, the build of the secondary package (icio_tools) failed, due to Have choice: libOpenColorIO2_0 and libOpenColorIO2_1
Easiest would have been:
Prefer: libOpenColorIO2_1
on prjconf, but with obvious issues (propagation into Factory, et.al.).
I think in this case the Prefer would have worked fine. After all, the issue only appears as Factory still has the old package providing the wanted library/SONAME, and the devel/branch project also has the new one, i.e. both are visible at the same time. As soon as the new package is moved to Factory/Staging, the old package is discarded, and only the new package exists. Kind regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen phone: +49 241 53809034 mobile: +49 151 50412019
On Mon, 18 Jul 2022, Hans-Peter Jansen wrote:
Am Montag, 18. Juli 2022, 09:10:09 CEST schrieb Richard Biener:
On Thu, 14 Jul 2022, Olaf Hering wrote:
Thu, 14 Jul 2022 15:40:52 +0200 Stefan Brüns <stefan.bruens@rwth-aachen.de>:
the pkgname here must match "lib.*103".
Correct, this is the helpful message which rpmlint must show, instead of the current crap..
In fact the check computes the exact name that is required and IIRC that was shown at some point ...
Yes, it shows the exact name, that is expected.
Here's a related case, that I tripped over this weekend:
graphics:OpenColorIO had the wrong so_ver defined, therefore this check is helpful indeed.
After fixing, the build of the secondary package (icio_tools) failed, due to Have choice: libOpenColorIO2_0 and libOpenColorIO2_1
Interesting - what's the provides/requires situation here? You shouldn't have explicit Requires: libOpenColorIO2_{0,1} in a .spec file, but obviously rpms auto-ReqProv machinery could be the offender here (but all those provides/requries create the something including the SONAME ...) So is it just failure to clean the wrongly named package from the build _repository? A wipebinaries can do the trick here for devel projects.
Easiest would have been:
Prefer: libOpenColorIO2_1
on prjconf, but with obvious issues (propagation into Factory, et.al.).
For Factory the staging process should weed those out ...
Hence it took the second possibility:
OpenColorIO.spec: # BuildIgnore can be removed, when libOCIO2_0 disappeared from repos #!BuildIgnore: libOpenColorIO2_0
Next, the OpenShadingLanguage build broke, due to the dependency to OpenImageIO, which in turn depend in OpenColorIO, which resulted in the "Have Choice" again.
My solution to this problem was to add the dependency of OCIO to OSL explicitly, in order to:
BuildRequires: cmake(OpenColorIO) # BuildIgnore can be removed, when libOCIO2_0 disappeared from repos #!BuildIgnore: libOpenColorIO2_0
which probably isn't the sanest thing to do. Alternatively, I could add a hard OCIO library dependency to the OIIO library.
What do you think?
So, while it's nice to catch such issues, handling them can be nasty.
Yeah. Usually it requires some manual fiddling during the staging project build but that's better than trying to workaround the issue with #!BuildIgnore or other hacks. At least unless I misunderstood the issue. Richard.
Richard.
Cheers, Pete -- Life without chameleons is possible, but pointless.
-- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman; HRB 36809 (AG Nuernberg)
On Monday 2022-07-18 13:51, Richard Biener wrote:
After fixing, the build of the secondary package (icio_tools) failed, due to Have choice: libOpenColorIO2_0 and libOpenColorIO2_1
Interesting - what's the provides/requires situation here? You shouldn't have explicit Requires: libOpenColorIO2_{0,1} in a .spec file
rpms have an implied Requires: libopencolor.so.1()(64bit). That symbol however is (erroneously) provided by both libOpenColorIO2_0 and libOpenColorIO2_1, hence the break for choice. It's why the error score was raised back from 1 to 10000, so these problems are more readily visible.
Am Montag, 18. Juli 2022, 14:16:08 CEST schrieb Jan Engelhardt:
On Monday 2022-07-18 13:51, Richard Biener wrote:
After fixing, the build of the secondary package (icio_tools) failed, due to Have choice: libOpenColorIO2_0 and libOpenColorIO2_1
Interesting - what's the provides/requires situation here? You shouldn't have explicit Requires: libOpenColorIO2_{0,1} in a .spec file
rpms have an implied Requires: libopencolor.so.1()(64bit). That symbol however is (erroneously) provided by both libOpenColorIO2_0 and libOpenColorIO2_1, hence the break for choice.
which begs the question, how to avoid *these* kind of failures in the first place?
It's why the error score was raised back from 1 to 10000, so these problems are more readily visible.
Yeah, this fallout now will pay off later on. Cheers, Pete -- Life without chameleons is possible, but pointless.
On Monday 2022-07-18 15:00, Hans-Peter Jansen wrote:
rpms have an implied Requires: libopencolor.so.1()(64bit). That symbol however is (erroneously) provided by both libOpenColorIO2_0 and libOpenColorIO2_1, hence the break for choice.
which begs the question, how to avoid *these* kind of failures in the first place?
Everyone needs to enforce more discipline on themselves I guess. https://github.com/rpm-software-management/rpmlint/commit/6dc5311137e4d4215d... Commit without rationale: pretty bad. Lack of review with regard to commit 6dc53: bad. Package trickling into openSUSE: oof. Packagers ignoring rpmlint messages with a score of 1: meh.
On Mon, 18 Jul 2022, Jan Engelhardt wrote:
On Monday 2022-07-18 15:00, Hans-Peter Jansen wrote:
rpms have an implied Requires: libopencolor.so.1()(64bit). That symbol however is (erroneously) provided by both libOpenColorIO2_0 and libOpenColorIO2_1, hence the break for choice.
which begs the question, how to avoid *these* kind of failures in the first place?
Everyone needs to enforce more discipline on themselves I guess.
https://github.com/rpm-software-management/rpmlint/commit/6dc5311137e4d4215d... Commit without rationale: pretty bad. Lack of review with regard to commit 6dc53: bad. Package trickling into openSUSE: oof. Packagers ignoring rpmlint messages with a score of 1: meh.
It also seems to me the enforcing focus is no longer the original intent of the policy - namely that shared library packages for the same library but with different SONAME (aka different ABI) should be able to be installed at the same time. The original policy explicitely excluded cases where the package setup takes a different approach to ensure this (as in the HPC packages case), and thus exempted libraries in non-standard paths (which also conveniently exempts shared objects used as plugins, usually also in paths like /usr/lib/$package/$plugin.so). To enforce this a mapping from SONAME to packge name is provided. It ignores the case of multiple packages providing a library with the same SONAME but could probably by checking for a pattern the GCC packges rely on - providing the canonical name and conflicting with it. Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman; HRB 36809 (AG Nuernberg)
Am Montag, 18. Juli 2022, 14:16:08 CEST schrieb Jan Engelhardt:
On Monday 2022-07-18 13:51, Richard Biener wrote:
After fixing, the build of the secondary package (icio_tools) failed, due to Have choice: libOpenColorIO2_0 and libOpenColorIO2_1
Interesting - what's the provides/requires situation here? You shouldn't have explicit Requires: libOpenColorIO2_{0,1} in a .spec file
rpms have an implied Requires: libopencolor.so.1()(64bit). That symbol however is (erroneously) provided by both libOpenColorIO2_0 and libOpenColorIO2_1, hence the break for choice.
It's why the error score was raised back from 1 to 10000, so these problems are more readily visible.
BTW, this also leads to the unexpected effect of not being considered by a zypper dup --allow-vendor-change in the transition phase. $ zyp dup --allow-vendor-change [...] OpenColorIO-debuginfo 2.1.2-66.3 -> 2.1.2-79.1 OpenColorIO-debugsource 2.1.2-66.3 -> 2.1.2-79.1 OpenImageIO-debuginfo 2.3.15.0-109.7 -> 2.3.15.0-112.1 OpenImageIO-debugsource 2.3.15.0-109.7 -> 2.3.15.0-112.1 OpenShadingLanguage-debuginfo 1.11.17.0-80.14 -> 1.11.17.0-83.1 OpenShadingLanguage-debugsource 1.11.17.0-80.14 -> 1.11.17.0-83.1 [...] The following 2 packages are going to be downgraded: libOpenColorIO2_0 2.1.2-66.3 -> 2.1.2-1.2 libOpenImageIO2_3 2.3.15.0-109.7 -> 2.3.15.0-1.3 The following 3 packages are going to change vendor: libkdecorations2private9 5.25.3-1.1 -> 5.25.3git. 20220627T100049~5df2f5b-9.1 openSUSE -> obs://build.opensuse.org/ home:frispete libOpenColorIO2_0 2.1.2-66.3 -> 2.1.2-1.2 obs://build.opensuse.org/home:frispete -> openSUSE libOpenImageIO2_3 2.3.15.0-109.7 -> 2.3.15.0-1.3 obs://build.opensuse.org/home:frispete -> openSUSE [Ignore the libkde.. part] Vlad fixed the long standing kwin5 window shutter bug on X11 yesterday! Yay! One has to explicitly install the new version: $ zyp in libOpenColorIO2_1 [...] Problem: the to be installed libOpenColorIO2_1-2.1.2-79.1.x86_64 conflicts with 'libOpenColorIO2_0 = 2.1.2' provided by the installed libOpenColorIO2_0-2.1.2-66.3.x86_64 Solution 1: deinstallation of libOpenColorIO2_0-2.1.2-66.3.x86_64 Solution 2: do not install libOpenColorIO2_1-2.1.2-79.1.x86_64 Choose from above solutions by number or cancel [1/2/c/d/?] (c): 1 Applying solution 1 Resolving dependencies... Resolving package dependencies... Force resolution: No The following NEW package is going to be installed: libOpenColorIO2_1 2.1.2-79.1 The following package is going to be REMOVED: libOpenColorIO2_0 2.1.2-66.3 1 new package to install, 1 to remove. Overall download size: 1.1 MiB. Already cached: 0 B. After the operation, 168.0 KiB will be freed. Sure, it doesn't make much difference for the usage of the library, since that effectively didn't change. So the implied provides seems to be stickier than the version update: [before the forced update] $ rpm -q --provides libOpenColorIO2_0 libOpenColorIO.so.2.1()(64bit) libOpenColorIO2_0 = 2.1.2-66.3 libOpenColorIO2_0(x86-64) = 2.1.2-66.3 [after the forced update] $ rpm -q --provides libOpenColorIO2_1 libOpenColorIO.so.2.1()(64bit) libOpenColorIO2_1 = 2.1.2-79.1 libOpenColorIO2_1(x86-64) = 2.1.2-79.1 Best, Pete -- Life without chameleons is possible, but pointless.
Am Montag, 18. Juli 2022, 13:51:07 CEST schrieb Richard Biener:
On Mon, 18 Jul 2022, Hans-Peter Jansen wrote:
So, while it's nice to catch such issues, handling them can be nasty.
Yeah. Usually it requires some manual fiddling during the staging project build but that's better than trying to workaround the issue with #!BuildIgnore or other hacks.
At least unless I misunderstood the issue.
FYI: Asterios didn't manage to commit my BuildIgnore hack yet, hence: https://build.opensuse.org/project/show/openSUSE:Factory:Staging:adi:33 Best, Pete -- Life without chameleons is possible, but pointless.
On Thu, 2022-07-14 at 14:16 +0200, Stefan Brüns wrote:
Hi,
a few days ago any violation of shlib-policy-name-error has become a fatal error.
This is especially problematic for many packages in the science repository, when packages are built as flavors for e.g. openmpi2/3/4, and the library packages carry a corresponding suffix. E.g.:
libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) SONAME: libhdf5.so.103 (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package suffix: 103
This is a case not covered by https://en.opensuse.org/ openSUSE:Shared_library_packaging_policy
Can this please be reverted until the policy has been extended to cover these cases? Preferably, rpmlint learns to understand this pattern.
I second this. This rpmlint error means we have to now carry an rpmlintrc for _every_ openmpi flavoured package that installs a shared lib to `%_libdir/mpi/gcc/openmpi*/lib*/`. Taken all together, for about 50 odd packages, this is additional work one could do without. Best wishes, -- Atri Bhattacharya Fri 15 Jul 02:50:19 CEST 2022 Sent from openSUSE Tumbleweed 20220712 on my laptop.
participants (7)
-
Atri Bhattacharya -
Hans-Peter Jansen -
Jan Engelhardt -
Olaf Hering -
Richard Biener -
Simon Lees -
Stefan Brüns