- openSUSE Science

Re: Project wide OpenMPI ABI breakage for SLE/Leap 15
by Egbert Eich 26 Oct '23

26 Oct '23

Hi Nicolas, Nicolas Morey writes: > On 10/26/23 19:01, Egbert Eich wrote: > > Stefan Brüns writes: > > > Hi, > > > > > > the recent removal of OpenMPI 1/2/3 from the science project has broken the > > > ABI. > > > > > > On SLE/LEAP 15, "openmpi" always meant OpenMPI 1, while suddenly it has become > > > OpenMPI 4. Anyone using packages from the science project will end up with > > > packages which are linked to either openmpi1 or openmpi4, and an application > > > may pull in both at the same time - obviously not a good idea. > > > > > > So, can this please be fixed ASAP, before too many users are hit by this? > > > > Ok, I see the problem now. Leap has inherited from earlier versions all > > outdated openmpi1 packages which were denoted as 'openmpi'. Other than > > SLE, Leap does not drop any packages, so recycling the term 'openmpi' > > for the default OpenMPI version causes havoc. > > So even if we denote the default OpenMPI version for *build* purposes, > > it should never be be used for package naming and dependencies - here the > > explicit version should always be specified (or 'openmpi' for OpenMPI v1). > > @Nicolas, could you look into this? > > > > Cheers, > > Egbert. > > > There is no "openmpi" package. > There is a "Provides: openmpi- %{version}" for the package that is the current default (right now openmpi4). On a Leap 15.4 I get: zypper search openmpi | grep " openmpi " [...] | mpitests-openmpi | MPI Benchmarks and tests for openmpi | package | openmpi | A powerful implementation of MPI | package | openmpi-debuginfo | Debug information for package openmpi | package | openmpi-debugsource | Debug sources for package openmpi > As stated in my other email, this has been the case for over 4 years ! > The right way to build against the default openMPI version is what is done in hpx: > - BuildRequires: openmpi-macros-devel > - Use %{openmpi_requires} and %{openmpi_devel_requires} in right packages to force a dependency to the right version (not just a libmpi.so.XX one) > - %{setup_openmpi} in the %build/%install section to setup the openMPI path/ld_library_path and so on Yes, this is true - for the future, but not the past - you cannot change the past. The past is what'son Leap 15.n - anything that was shipped since Leap 15.0. Example (on Leap 15.4): zypper info --requires parallel-netcdf-openmpi-devel [...] Requires : [3] parallel-netcdf-devel-data openmpi-devel libpnetcdf1-openmpi = 1.7.0 Here, you have an unversioned dependency to openmpi-devel. > If there are (not _multibuild) packages that do not use these macros, they > should be changed to use them. For this to happen, you will have to rebuild a considerable portion of the packages you no longer want to deal with. If Leap 15.6 will acquire your packages, you will see a lot of conflicts from these old packages. > I thought I caught all of them when openmpi1 was renamed in Factory but there may be some non Factory one that I missed and luckily kept working since them. Factory may be fine if everything has been rebuilt with your macros. Leap still contains packages that pre-date your fixes. SLE may be Ok because we hopefully do not ship these old packages any more. Cheers, Egbert.

2 1

Re: Project wide OpenMPI ABI breakage for SLE/Leap 15
by Egbert Eich 26 Oct '23

26 Oct '23

Stefan Brüns writes: > Hi, > > the recent removal of OpenMPI 1/2/3 from the science project has broken the > ABI. > > On SLE/LEAP 15, "openmpi" always meant OpenMPI 1, while suddenly it has become > OpenMPI 4. Anyone using packages from the science project will end up with > packages which are linked to either openmpi1 or openmpi4, and an application > may pull in both at the same time - obviously not a good idea. > > So, can this please be fixed ASAP, before too many users are hit by this? Ok, I see the problem now. Leap has inherited from earlier versions all outdated openmpi1 packages which were denoted as 'openmpi'. Other than SLE, Leap does not drop any packages, so recycling the term 'openmpi' for the default OpenMPI version causes havoc. So even if we denote the default OpenMPI version for *build* purposes, it should never be be used for package naming and dependencies - here the explicit version should always be specified (or 'openmpi' for OpenMPI v1). @Nicolas, could you look into this? Cheers, Egbert.

2 1

Project wide OpenMPI ABI breakage for SLE/Leap 15
by Stefan Brüns 26 Oct '23

26 Oct '23

Hi, the recent removal of OpenMPI 1/2/3 from the science project has broken the ABI. On SLE/LEAP 15, "openmpi" always meant OpenMPI 1, while suddenly it has become OpenMPI 4. Anyone using packages from the science project will end up with packages which are linked to either openmpi1 or openmpi4, and an application may pull in both at the same time - obviously not a good idea. So, can this please be fixed ASAP, before too many users are hit by this? Regards, Stefan

2 1

Mass addition of python packages
by Stefan Brüns 08 Aug '23

08 Aug '23

During the last two days, there was a mass addition of linked package from devel:languages:python to the science project. I think it is quite bad this was neither discussed nor announced here. I also think this is plain wrong: 1. d:l:p is not meant as a base project for other projects, but as a devel project 2. in case it was meant to fill in some missing packages for Leap 15, d:l:p:backports would have been a better choice 3. The packages from d:l:p often require a python > 3.6, so many won't build on Leap anyway 4. For Tumbleweed, linking and rebuilding the packages from d:l:p is rather pointless and wasteful - we get more rebuilds, and longer rebuilds Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen phone: +49 241 53809034 mobile: +49 151 50412019

1 0

Leave
by Slawomir Siejka 31 Jul '22

31 Jul '22

Empty Message

1 0

Re: Unannounced change of shlib-policy-name-error rpmlint strictness on TW
by Stefan Brüns 14 Jul '22

14 Jul '22

On Donnerstag, 14. Juli 2022 14:34:27 CEST Richard Biener wrote: > On Thu, 14 Jul 2022, Stefan Brüns wrote: > > Hi, > > > > a few days ago any violation of shlib-policy-name-error has become a fatal > > error. > > > > This is especially problematic for many packages in the science > > repository, > > when packages are built as flavors for e.g. openmpi2/3/4, and the library > > packages carry a corresponding suffix. E.g.: > > > > libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) > > SONAME: libhdf5.so.103 > > (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package > > suffix: 103 > > > > This is a case not covered by https://en.opensuse.org/ > > openSUSE:Shared_library_packaging_policy > > > > Can this please be reverted until the policy has been extended to cover > > these cases? Preferably, rpmlint learns to understand this pattern. > > The policy wasn't originally supposed to apply to shared libraries > that do not reside in the default runpath which means > /usr/lib*/$package/... should have been excempted. Possibly the > implementation doesn't reflect that or the implementation > is too narrow? > > IIRC there's also a whitelist(?) - the GCC package excempts itself > via rpmlintrc filtering the check ... > > Richard. In the current code I can not find such a generic exemption. Of course one can add an exemption via rpmlintrc. I would prefer not do so: 1. There are probably about 50 source packages in science alone affected by this. 2. Each rpmlintrc entry bears the risk of hiding actual packaging errors. Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen phone: +49 241 53809034 mobile: +49 151 50412019

1 0

Unannounced change of shlib-policy-name-error rpmlint strictness on TW
by Stefan Brüns 14 Jul '22

14 Jul '22

Hi, a few days ago any violation of shlib-policy-name-error has become a fatal error. This is especially problematic for many packages in the science repository, when packages are built as flavors for e.g. openmpi2/3/4, and the library packages carry a corresponding suffix. E.g.: libhdf5-103-openmpi4.x86_64: E: shlib-policy-name-error (Badness: 10000) SONAME: libhdf5.so.103 (/usr/lib64/mpi/gcc/openmpi4/lib64/libhdf5.so.103.3.1), expected package suffix: 103 This is a case not covered by https://en.opensuse.org/ openSUSE:Shared_library_packaging_policy Can this please be reverted until the policy has been extended to cover these cases? Preferably, rpmlint learns to understand this pattern. Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen home: +49 241 53809034 mobile: +49 151 50412019

1 0

Re: RFC: Proper automatic library dependencies for MPI/serial flavors
by Egbert Eich 05 Jul '22

05 Jul '22

Hi Stefan, Stefan Brüns writes: > Hi everyone, > > we have a reocurring problem with multiple providers of libraries built as > serial and openMPI(1-4)/MVAPICH/... flavors. > > Currently, both the serial and e.g. openMPI4 flavors provide the exact same > resolvables, e.g. "libsundials_sunlinsolklu.so.4.2.0()(64bit)" [1]. This is > then manually fixed up on a case-by-case basis by manually specifying the > default package. > > Fedora solves this problem by adding the MPI flavor to the resolvable, e.g. > "libfoo.so()(64bit)(openmpi-x86_64)". For details, see [2][3]. > > I would propose for (open)SUSE to follow this notation, and finally get rid > of the manual workarounds. Indeed, this problem is not new and I believe we've talked about this some years ago already. We had the same with HPC packages as well, we 'solved' it by replacing the dependency generator by one that would ignore any dependencies pointing into the HPC directories (/usr/lib/hpc) and adding dependencies manually. I had the plan to resolve it the way you've suggested adding more 'attribute tags' do the dependency to specify it with greater detail. This of course wasn't the real solution, just a workaround. I never found time to do the real solution. So indeed, if Fedora has a solution for this already (at least for the MPI problem, for HPC packages there are further challanges), we should take this into consideration. Thanks for picking this up again! Cheers, Egbert. > > Kind regards, Stefan > > > [1] https://build.opensuse.org/package/binary/science/sundials:openmpi4/ > openSUSE_Tumbleweed/x86_64/libsundials4-openmpi4-6.2.0-19.1.x86_64.rpm > > [2] https://fedoraproject.org/wiki/Changes/RpmMPIReqProv > [3] https://packages.fedoraproject.org/pkgs/rpm-mpi-hooks/rpm-mpi-hooks/ > index.html > -- Egbert Eich (Res. & Dev.) SUSE Labs - Architect HPC Tel: +49 911-740 53 0 https://www.suse.com ----------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg Geschaeftsfuehrer: Ivo Totev (HRB 36809, AG Nürnberg)

1 0

RFC: Proper automatic library dependencies for MPI/serial flavors
by Stefan Brüns 05 Jul '22

05 Jul '22

Hi everyone, we have a reocurring problem with multiple providers of libraries built as serial and openMPI(1-4)/MVAPICH/... flavors. Currently, both the serial and e.g. openMPI4 flavors provide the exact same resolvables, e.g. "libsundials_sunlinsolklu.so.4.2.0()(64bit)" [1]. This is then manually fixed up on a case-by-case basis by manually specifying the default package. Fedora solves this problem by adding the MPI flavor to the resolvable, e.g. "libfoo.so()(64bit)(openmpi-x86_64)". For details, see [2][3]. I would propose for (open)SUSE to follow this notation, and finally get rid of the manual workarounds. Kind regards, Stefan [1] https://build.opensuse.org/package/binary/science/sundials:openmpi4/ openSUSE_Tumbleweed/x86_64/libsundials4-openmpi4-6.2.0-19.1.x86_64.rpm [2] https://fedoraproject.org/wiki/Changes/RpmMPIReqProv [3] https://packages.fedoraproject.org/pkgs/rpm-mpi-hooks/rpm-mpi-hooks/ index.html

1 0

Re: openblas setup screwed in science project
by Egbert Eich 05 May '22

05 May '22

Atri Bhattacharya writes: > On Wed, 2022-05-04 at 14:19 +0200, Stefan Brüns wrote: > > > > Hi Egbert, > > > > Another problem with openblas is the missing .baselibs configuration > > for 32bit > > packages. Some time ago it was still there, but then went missing > > without a > > mention in the changelog. This breaks at least arpack-ng. > > > > At least for TW, i586 (actully, somthing like P4) is still supported, > > and for > > any library package still working on i586 we should also have the > > baselibs > > configuration (IMHO). > > > > > > Related to this, I am trying to restore the baselibs.conf to openblas, > see WIP at > <https://build.opensuse.org/package/show/home:badshah400:branches:science/op… > >. That's fine, especially since you are creating it dynamically ;) You may have to specify the library files by hand as lib(64)?/ contains only links. The real libraries live in lib(64)?/openblas-<flavor>. Cheers, Egbert. > > Optionally, I made an sr to obs://science/arpack-ng to disable the - > 32bit bi-arch package generation in > <https://build.opensuse.org/request/show/974867> > but feel free to decide whether to decline this and accept the changes > to openblas (in a future sr) instead. > > Best wishes, > > > -- > Atri Bhattacharya -- Egbert Eich (Res. & Dev.) SUSE Labs - Architect HPC Tel: +49 911-740 53 0 https://www.suse.com ----------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg Geschaeftsfuehrer: Ivo Totev (HRB 36809, AG Nürnberg)

2 1