Different source package versions building library with identical soname
Hi, I want to submit a package into Factory that provides a library with an soname identical to that provided by another package, but this seems to conflict with the convention to name such packages lib%{name}%{sonum}, since obviously the packages can't share the same name. [1] The background: Clang has both a stable API (the "C API" in libclang.so) and an instable API (the "C++ API" in libclang-cpp.so). Now upstream noticed that this should perhaps be reflected by not increasing the so number of libclang.so [2], so that with LLVM 14 we now have libclang-cpp.so -> libclang-cpp.so.14 libclang-cpp.so.14 libclang.so -> libclang.so.13 libclang.so.13 -> libclang.so.14.0.0 libclang.so.14.0.0 Of course libclang-cpp.so.* libraries need to be able to coexist, whereas libclang.so cannot coexist with the previous version that provides libclang.so.13 -> libclang.so.13.0.1 So I have to split up the libraries. But then both package versions provide libclang.so.13()(64bit) libclang.so.13(LLVM_13)(64bit) Indeed only the newer version should ever be needed, so should I stop building the older version or, more realistically, don't package it? (There are binaries in clang13 that use it, so I can't stop building or shipping it that easily. These binaries should of course also work with libclang.so.13 provided by LLVM 14.) Easier would be to keep shipping the old package, but have the new version obsolete it. Is that enough? Additionally, how should I name that package? Of course I can't also call it libclang13 unless I modify the LLVM 13 packaging to remove the library there, so should it be libclang14 or libclang13-14? The former would be the easiest, but rpmlint doesn't like it: libclang14.x86_64: E: shlib-policy-name-error SONAME: libclang.so.13, expected package suffix: 13 Best regards, Aaron [1] https://en.opensuse.org/openSUSE:Shared_library_packaging_policy [2] https://reviews.llvm.org/D105527
On Sun, 27 Mar 2022, Aaron Puchert wrote:
Hi,
I want to submit a package into Factory that provides a library with an soname identical to that provided by another package, but this seems to conflict with the convention to name such packages lib%{name}%{sonum}, since obviously the packages can't share the same name. [1]
The background: Clang has both a stable API (the "C API" in libclang.so) and an instable API (the "C++ API" in libclang-cpp.so). Now upstream noticed that this should perhaps be reflected by not increasing the so number of libclang.so [2], so that with LLVM 14 we now have
libclang-cpp.so -> libclang-cpp.so.14 libclang-cpp.so.14 libclang.so -> libclang.so.13 libclang.so.13 -> libclang.so.14.0.0 libclang.so.14.0.0
Of course libclang-cpp.so.* libraries need to be able to coexist, whereas libclang.so cannot coexist with the previous version that provides
libclang.so.13 -> libclang.so.13.0.1
So I have to split up the libraries. But then both package versions provide
libclang.so.13()(64bit) libclang.so.13(LLVM_13)(64bit)
Indeed only the newer version should ever be needed, so should I stop building the older version or, more realistically, don't package it? (There are binaries in clang13 that use it, so I can't stop building or shipping it that easily. These binaries should of course also work with libclang.so.13 provided by LLVM 14.) Easier would be to keep shipping the old package, but have the new version obsolete it. Is that enough?
Additionally, how should I name that package? Of course I can't also call it libclang13 unless I modify the LLVM 13 packaging to remove the library there, so should it be libclang14 or libclang13-14? The former would be the easiest, but rpmlint doesn't like it:
libclang14.x86_64: E: shlib-policy-name-error SONAME: libclang.so.13, expected package suffix: 13
Yes, OBS plays not nice with this situation. But what should direct you
to the solution is rpm/zypp which do not handle auto-updating of
packages with different names. For example if you had libclang13-llvm13
and libclang13-llvm14 which should be possible with an entry into the
exception table then you need Provide/Obsoletes entries for those
libraries that will be an ever growing list.
For GCC we shift the pain to the OBS side by packaging
%package -n libgcc_s%{libgcc_s}%{libgcc_s_suffix}
where %{libgcc_s_suffix} is either empty (GCC version selected by
the OBS projconf) or of the form '-gcc<version>' where the suffixed
package versions will be never auto-installed/updated (but a user
can choose to manually install newer versions). OBS in the
project config then adds magic to select the gcc version plus has
Prefer: libgcc_s1
(because otherwise it wouldn't know whether to install libgcc_s1
or libgcc_s1-gcc10 for example)
Since in GCC we have multiple stable ABI libraries but some do
change SONAMEs at some point and thus which GCC version provides
exactly which stable ABI library depends on the library (not sure
if you are there yet) the project config has entries like
# The following shlibs have the latest version built from GCC 10 sources
%product_libs_gcc_ver_libgphobos1 10
%product_libs_gcc_ver_libgdruntime1 10
%product_libs_gcc_ver_libgo16 10
in addition to
# define which gcc package builds the system libraries
%product_libs_gcc_ver 11
To use those we have (just quoting the simple case of libgcc_s1):
%define libgcc_s_suffix %{plv libgcc_s %{libgcc_s}}
with
%define itsme11 1
%define plv_ %{!?product_libs_gcc_ver:11}%{?product_libs_gcc_ver}
%define plv()
%{expand:%%{!?itsme%{expand:%%{!?product_libs_gcc_ver_%{1}%{2}:%%{plv_}}%%{?product_libs_gcc_ver_%{1}%{2}}}:-gcc11}}
(it was some trial-and-error to find a way that works across SLE10 to
Factory).
So my advice would be to go a similar way for the stable ABI libraries.
Our rpmlintrc has
# We have names lib libgcc_s1-gcc7 for non-default GCCs
addFilter ("shlib-policy-name-error")
and the -gccN suffixed packages never end up on any product media in SLE.
Richard.
--
Richard Biener
Hi Richard, Thanks for the detailed answer, which I'm still processing. Am 28.03.22 um 08:51 schrieb Richard Biener:
Yes, OBS plays not nice with this situation. But what should direct you to the solution is rpm/zypp which do not handle auto-updating of packages with different names. For example if you had libclang13-llvm13 and libclang13-llvm14 which should be possible with an entry into the exception table then you need Provide/Obsoletes entries for those libraries that will be an ever growing list. Indeed a versioned Provides/Obsoletes on something like libclang13 doesn't cause zypper to update from one to the other, but it seems to have removed a previous libclang13. So Obsoletes works properly only for actual package names, not for anything they provide? (In this way it would be different than Conflicts?) For GCC we shift the pain to the OBS side by packaging
%package -n libgcc_s%{libgcc_s}%{libgcc_s_suffix}
where %{libgcc_s_suffix} is either empty (GCC version selected by the OBS projconf) or of the form '-gcc<version>' where the suffixed package versions will be never auto-installed/updated (but a user can choose to manually install newer versions).
Let's say I have libgcc_s1-gcc12 installed, which is not the default yet. Would this migrate me to libgcc_s1 when you update the default GCC, and have you any special mechanism for that? Might not be an issue for me since we usually switch the default LLVM right away, but would be good to know.
OBS in the project config then adds magic to select the gcc version plus has
Prefer: libgcc_s1
(because otherwise it wouldn't know whether to install libgcc_s1 or libgcc_s1-gcc10 for example) Seems I'll need that as well, otherwise OBS doesn't know what to choose for resolving e.g. 'libclang.so.13()(64bit)'. Since in GCC we have multiple stable ABI libraries but some do change SONAMEs at some point and thus which GCC version provides exactly which stable ABI library depends on the library (not sure if you are there yet) the project config has entries like
# The following shlibs have the latest version built from GCC 10 sources %product_libs_gcc_ver_libgphobos1 10 %product_libs_gcc_ver_libgdruntime1 10 %product_libs_gcc_ver_libgo16 10
in addition to
# define which gcc package builds the system libraries %product_libs_gcc_ver 11
So "%product_libs_gcc_ver 11" selects gcc11 for producing the newest SONAME of all libraries, while the other variables only select which gcc version produces an older SONAME, right? Since for now I'll only touch llvm13 and llvm14, I wouldn't need this and could go with a single %product_libs_llvm_ver 14 but of course we might need a %product_libs_llvm_ver_libclang13 XXX should we get a libclang14 some day.
To use those we have (just quoting the simple case of libgcc_s1):
%define libgcc_s_suffix %{plv libgcc_s %{libgcc_s}}
with
%define itsme11 1 %define plv_ %{!?product_libs_gcc_ver:11}%{?product_libs_gcc_ver} %define plv() %{expand:%%{!?itsme%{expand:%%{!?product_libs_gcc_ver_%{1}%{2}:%%{plv_}}%%{?product_libs_gcc_ver_%{1}%{2}}}:-gcc11}} Got it, that decides whether we need a suffix or not, or rather what the suffix is going to be. Our rpmlintrc has
# We have names lib libgcc_s1-gcc7 for non-default GCCs addFilter ("shlib-policy-name-error") It seems that going with a prefix instead of a suffix makes that check happy. Currently I have llvm14-libclang13. Maybe this is a bad idea though?
Aaron
On Sat, 2 Apr 2022, Aaron Puchert wrote:
Hi Richard,
Thanks for the detailed answer, which I'm still processing.
Yes, OBS plays not nice with this situation. But what should direct you to the solution is rpm/zypp which do not handle auto-updating of packages with different names. For example if you had libclang13-llvm13 and libclang13-llvm14 which should be possible with an entry into the exception table then you need Provide/Obsoletes entries for those libraries that will be an ever growing list. Indeed a versioned Provides/Obsoletes on something like libclang13 doesn't cause zypper to update from one to the other, but it seems to have removed a
Am 28.03.22 um 08:51 schrieb Richard Biener: previous libclang13. So Obsoletes works properly only for actual package names, not for anything they provide? (In this way it would be different than Conflicts?)
Yes, Obsoletes does not work for "virtual" packages.
For GCC we shift the pain to the OBS side by packaging
%package -n libgcc_s%{libgcc_s}%{libgcc_s_suffix}
where %{libgcc_s_suffix} is either empty (GCC version selected by the OBS projconf) or of the form '-gcc<version>' where the suffixed package versions will be never auto-installed/updated (but a user can choose to manually install newer versions).
Let's say I have libgcc_s1-gcc12 installed, which is not the default yet. Would this migrate me to libgcc_s1 when you update the default GCC, and have you any special mechanism for that?
No, you'll need to manually resolve the update in that case. Likewise when you first install libgcc_s1-gcc12 via installing gcc12 you will be asked if you want to remove libgcc_s1.
Might not be an issue for me since we usually switch the default LLVM right away, but would be good to know.
OBS in the project config then adds magic to select the gcc version plus has
Prefer: libgcc_s1
(because otherwise it wouldn't know whether to install libgcc_s1 or libgcc_s1-gcc10 for example) Seems I'll need that as well, otherwise OBS doesn't know what to choose for resolving e.g. 'libclang.so.13()(64bit)'. Since in GCC we have multiple stable ABI libraries but some do change SONAMEs at some point and thus which GCC version provides exactly which stable ABI library depends on the library (not sure if you are there yet) the project config has entries like
# The following shlibs have the latest version built from GCC 10 sources %product_libs_gcc_ver_libgphobos1 10 %product_libs_gcc_ver_libgdruntime1 10 %product_libs_gcc_ver_libgo16 10
in addition to
# define which gcc package builds the system libraries %product_libs_gcc_ver 11
So "%product_libs_gcc_ver 11" selects gcc11 for producing the newest SONAME of all libraries, while the other variables only select which gcc version produces an older SONAME, right? Since for now I'll only touch llvm13 and llvm14, I wouldn't need this and could go with a single
Right.
%product_libs_llvm_ver 14
but of course we might need a
%product_libs_llvm_ver_libclang13 XXX
should we get a libclang14 some day.
Yes.
To use those we have (just quoting the simple case of libgcc_s1):
%define libgcc_s_suffix %{plv libgcc_s %{libgcc_s}}
with
%define itsme11 1 %define plv_ %{!?product_libs_gcc_ver:11}%{?product_libs_gcc_ver} %define plv() %{expand:%%{!?itsme%{expand:%%{!?product_libs_gcc_ver_%{1}%{2}:%%{plv_}}%%{?product_libs_gcc_ver_%{1}%{2}}}:-gcc11}} Got it, that decides whether we need a suffix or not, or rather what the suffix is going to be. Our rpmlintrc has
# We have names lib libgcc_s1-gcc7 for non-default GCCs addFilter ("shlib-policy-name-error") It seems that going with a prefix instead of a suffix makes that check happy. Currently I have llvm14-libclang13. Maybe this is a bad idea though?
Interesting - I suppose the regexp is a bit too forgiving there ;) I think we should try to do the same for packages hitting this issue, maybe others figured out the above workaround so it would be interesting to see how many examples we have in the distro right now ... Richard.
Aaron
--
Richard Biener
Am 04.04.22 um 07:58 schrieb Richard Biener:
Our rpmlintrc has
# We have names lib libgcc_s1-gcc7 for non-default GCCs addFilter ("shlib-policy-name-error") It seems that going with a prefix instead of a suffix makes that check happy. Currently I have llvm14-libclang13. Maybe this is a bad idea though? Interesting - I suppose the regexp is a bit too forgiving there ;)
That thought occurred to me. The Wiki doesn't leave this loophole, in fact there is no exception at all for our case. [1] Jan, since you're the most common contributor to that Wiki page, do we want to define a naming scheme for libraries where the same SONAME is generated by multiple source packages? The library packages can't have the same name then.
I think we should try to do the same for packages hitting this issue,
I don't feel strongly about it either way, it just seemed more natural to simply drop "-n" when naming the package, which makes RPM add the same prefix that it would usually add to a subpackage. I don't have a submit to Factory open, because I still need to fix an issue, so I can still easily change this. Aaron [1] https://en.opensuse.org/openSUSE:Shared_library_packaging_policy
On Monday 2022-04-04 23:44, Aaron Puchert wrote:
# We have names lib libgcc_s1-gcc7 for non-default GCCs addFilter ("shlib-policy-name-error")
It seems that going with a prefix instead of a suffix makes that check happy. Currently I have llvm14-libclang13. Maybe this is a bad idea though?
The Wiki doesn't leave this loophole, in fact there is no exception at all for our case. [1] [1] https://en.opensuse.org/openSUSE:Shared_library_packaging_policy
The Rationale section is non-normative, but still informative (guidance-giving):: "[...] make it possible to use multiple shared libraries that share the same name stem but having different SO version". A package set like {libgcc_s1-gcc7, libgcc_s1-gcc8, etc.} all have the *same* SO version. One could argue the policy is not applicable.
Do we want to define a naming scheme for libraries where the same SONAME is generated by multiple source packages? The library packages can't have the same name then.
It is a bit difficult to detect name (mis-)compliance: Mesa-libGL1.rpm is normally flagged as "there is no Mesa-libGL.so.1 in this package" and libgcc_s1-gcc8 as "there is no libgcc_s1-gcc.so.8 in this package". Existing practice is few and far between (off the top of my head, just gccX/gccY, Mesa/nvidia and now llvmX/llvmY). I'd say let's just treat it as exceptional.
On Tue, 5 Apr 2022, Jan Engelhardt wrote:
On Monday 2022-04-04 23:44, Aaron Puchert wrote:
# We have names lib libgcc_s1-gcc7 for non-default GCCs addFilter ("shlib-policy-name-error")
It seems that going with a prefix instead of a suffix makes that check happy. Currently I have llvm14-libclang13. Maybe this is a bad idea though?
The Wiki doesn't leave this loophole, in fact there is no exception at all for our case. [1] [1] https://en.opensuse.org/openSUSE:Shared_library_packaging_policy
The Rationale section is non-normative, but still informative (guidance-giving)::
"[...] make it possible to use multiple shared libraries that share the same name stem but having different SO version".
A package set like {libgcc_s1-gcc7, libgcc_s1-gcc8, etc.} all have the *same* SO version. One could argue the policy is not applicable.
Note that the fact that we use suffixes at all instead of just building libgcc_s1 from more than one source rpm is due to (old?) restrictions of tooling which had no way to distinguish the different versions and ended up picking a random one to put on the media or to use for building packages with useforbuild. I believe that to some extent those limitations still exist even if the behavior is now maybe less random, but eventually still not as desired (like if it would always pick the last built one). Michael can probably confirm for sure if there's still tools limitations.
Do we want to define a naming scheme for libraries where the same SONAME is generated by multiple source packages? The library packages can't have the same name then.
It is a bit difficult to detect name (mis-)compliance: Mesa-libGL1.rpm is normally flagged as "there is no Mesa-libGL.so.1 in this package" and libgcc_s1-gcc8 as "there is no libgcc_s1-gcc.so.8 in this package". Existing practice is few and far between (off the top of my head, just gccX/gccY, Mesa/nvidia and now llvmX/llvmY). I'd say let's just treat it as exceptional.
Agreed.
Note that since we know at build time whether the package is suffixed
(aka irrelevant) if there was a way to signal the rpmbuild result
consumer it would be possible to simply throw those packages away
(IIRC for old products we indeed rm'ed the rpms from the built
packages directory at some carefully chosen %<step> time which did
this trick). Like maybe
%package -n libgcc_s1-gcc7
#!BuildThrowaway
...
or so. That would work to throw away older versions of the same
shared library package but of course would not work to introduce
a newer (but not yet supposed to be default) version - that would
still need to have an alternate name, also to avoid exposing an
update path.
Richard.
--
Richard Biener
On Tuesday 2022-04-05 08:23, Richard Biener wrote:
A package set like {libgcc_s1-gcc7, libgcc_s1-gcc8, etc.} all have the *same* SO version. One could argue the policy is not applicable.
Note that the fact that we use suffixes at all instead of just building libgcc_s1 from more than one source rpm is due to (old?) restrictions of tooling which had no way to distinguish the different versions and ended up picking a random one to put on the media or to use for building packages with useforbuild.
The big issue is that two source packages in an OBS, pick Mesa and nvidiagfx if you need a thought example, could both be producing a libGL-1.0-1.1.rpm (first commit, first build attempt, and both having Version: 1.0). The same issue befalls gcc in principle, it just never manifests because gcc7 pretty much always has a different Version: field compared to gcc8.
Am 04.04.22 um 07:58 schrieb Richard Biener:
On Sat, 2 Apr 2022, Aaron Puchert wrote:
Hi Richard,
Thanks for the detailed answer, which I'm still processing.
Am 28.03.22 um 08:51 schrieb Richard Biener:
Our rpmlintrc has
# We have names lib libgcc_s1-gcc7 for non-default GCCs addFilter ("shlib-policy-name-error") It seems that going with a prefix instead of a suffix makes that check happy. Currently I have llvm14-libclang13. Maybe this is a bad idea though? Interesting - I suppose the regexp is a bit too forgiving there ;) I think we should try to do the same for packages hitting this issue, maybe others figured out the above workaround so it would be interesting to see how many examples we have in the distro right now ...
This seems to be more of an accident: a change in rpmlint 2.2 makes the check only fire for packages starting with "lib", apparently to exclude applications that have some libraries packaged with them. [1] But since that change isn't in Leap I might have to add the filter as well. Aaron [1] https://github.com/rpm-software-management/rpmlint/pull/735
participants (3)
-
Aaron Puchert -
Jan Engelhardt -
Richard Biener