Hi Stefan,
Stefan Brüns writes:
> Hi everyone,
>
> we have a reocurring problem with multiple providers of libraries built as
> serial and openMPI(1-4)/MVAPICH/... flavors.
>
> Currently, both the serial and e.g. openMPI4 flavors provide the exact same
> resolvables, e.g. "libsundials_sunlinsolklu.so.4.2.0()(64bit)" [1]. This is
> then manually fixed up on a case-by-case basis by manually specifying the
> default package.
>
> Fedora solves this problem by adding the MPI flavor to the resolvable, e.g.
> "libfoo.so()(64bit)(openmpi-x86_64)". For details, see [2][3].
>
> I would propose for (open)SUSE to follow this notation, and finally get rid
> of the manual workarounds.
Indeed, this problem is not new and I believe we've talked about this
some years ago already.
We had the same with HPC packages as well, we 'solved' it by replacing
the dependency generator by one that would ignore any dependencies pointing
into the HPC directories (/usr/lib/hpc) and adding dependencies manually.
I had the plan to resolve it the way you've suggested adding more
'attribute tags' do the dependency to specify it with greater detail.
This of course wasn't the real solution, just a workaround. I never found
time to do the real solution.
So indeed, if Fedora has a solution for this already (at least for the
MPI problem, for HPC packages there are further challanges), we should
take this into consideration.
Thanks for picking this up again!
Cheers,
Egbert.
>
> Kind regards, Stefan
>
>
> [1] https://build.opensuse.org/package/binary/science/sundials:openmpi4/
> openSUSE_Tumbleweed/x86_64/libsundials4-openmpi4-6.2.0-19.1.x86_64.rpm
>
> [2] https://fedoraproject.org/wiki/Changes/RpmMPIReqProv
> [3] https://packages.fedoraproject.org/pkgs/rpm-mpi-hooks/rpm-mpi-hooks/
> index.html
>
--
Egbert Eich (Res. & Dev.) SUSE Labs - Architect HPC
Tel: +49 911-740 53 0 https://www.suse.com
-----------------------------------------------------------------------
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg
Geschaeftsfuehrer: Ivo Totev
(HRB 36809, AG Nürnberg)
Hi everyone,
we have a reocurring problem with multiple providers of libraries built as
serial and openMPI(1-4)/MVAPICH/... flavors.
Currently, both the serial and e.g. openMPI4 flavors provide the exact same
resolvables, e.g. "libsundials_sunlinsolklu.so.4.2.0()(64bit)" [1]. This is
then manually fixed up on a case-by-case basis by manually specifying the
default package.
Fedora solves this problem by adding the MPI flavor to the resolvable, e.g.
"libfoo.so()(64bit)(openmpi-x86_64)". For details, see [2][3].
I would propose for (open)SUSE to follow this notation, and finally get rid
of the manual workarounds.
Kind regards, Stefan
[1] https://build.opensuse.org/package/binary/science/sundials:openmpi4/
openSUSE_Tumbleweed/x86_64/libsundials4-openmpi4-6.2.0-19.1.x86_64.rpm
[2] https://fedoraproject.org/wiki/Changes/RpmMPIReqProv
[3] https://packages.fedoraproject.org/pkgs/rpm-mpi-hooks/rpm-mpi-hooks/
index.html
Atri Bhattacharya writes:
> On Wed, 2022-05-04 at 14:19 +0200, Stefan Brüns wrote:
> >
> > Hi Egbert,
> >
> > Another problem with openblas is the missing .baselibs configuration
> > for 32bit
> > packages. Some time ago it was still there, but then went missing
> > without a
> > mention in the changelog. This breaks at least arpack-ng.
> >
> > At least for TW, i586 (actully, somthing like P4) is still supported,
> > and for
> > any library package still working on i586 we should also have the
> > baselibs
> > configuration (IMHO).
> >
> >
>
> Related to this, I am trying to restore the baselibs.conf to openblas,
> see WIP at
> <https://build.opensuse.org/package/show/home:badshah400:branches:science/op…
> >.
That's fine, especially since you are creating it dynamically ;)
You may have to specify the library files by hand as
lib(64)?/ contains only links. The real libraries live in
lib(64)?/openblas-<flavor>.
Cheers,
Egbert.
>
> Optionally, I made an sr to obs://science/arpack-ng to disable the -
> 32bit bi-arch package generation in
> <https://build.opensuse.org/request/show/974867>
> but feel free to decide whether to decline this and accept the changes
> to openblas (in a future sr) instead.
>
> Best wishes,
>
>
> --
> Atri Bhattacharya
--
Egbert Eich (Res. & Dev.) SUSE Labs - Architect HPC
Tel: +49 911-740 53 0 https://www.suse.com
-----------------------------------------------------------------------
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg
Geschaeftsfuehrer: Ivo Totev
(HRB 36809, AG Nürnberg)
Hi Stefan,
Stefan Brüns writes:
> On Freitag, 8. April 2022 16:08:01 CEST Stefan Brüns wrote:
> > Hi,
> >
> > someone has apparently disabled *all* builds for openblas.
> >
> > Whats the reason for this?
> >
> > As openblas 0.3.20 was never built inside science, we now have 0.3.20 in
> > Factory and 0.3.17 (binary packages) in science. This causes a lot of
> > unresolvables, as OBS tries to use openblas 0.3.17 from science.
> >
> > Kind regards,
> >
> > Stefan
>
> Hi Egbert,
>
> as you are the de-facto maintainer of openblas, can you shed some light here?
>
> Another problem with openblas is the missing .baselibs configuration for 32bit
> packages. Some time ago it was still there, but then went missing without a
> mention in the changelog. This breaks at least arpack-ng.
>
> At least for TW, i586 (actully, somthing like P4) is still supported, and for
> any library package still working on i586 we should also have the baselibs
> configuration (IMHO).
I'm not aware that I've thrown these out. On the packages I've worked on,
I've converted the static baselibs.conf files to static ones as there
was no good way to use static baselibs.conf file for all the different
flavors - moreover, I don't want these 32bit binaries for HPC anyway.
I've looked back, the version where I've claimed I added these, did not
have it. Going further back I wasn't able to find any static baselibs.conf
either, which makes me believe there was never any.
I personally do not care for 32-bit libraries, neither professionally nor
personally - at least for openblas. I had my fair share of openblas
- the upgrade to 0.3.20 gave me some grief - and I do not want to invest
any more time on topics that are of no interest for me.
If you feel like a baselibs.conf setup is useful and want to create one
*and* maintain it, please feel free.
There were some breakages introduced thru my changes - which I've hopefully
rectified, they had nothing to do with a missing baselibs.conf.
These issues should disappear from Factory as the fix has been accepted
last night.
I have no idea who disabled build of openblas in science. It wasn't me
- at least not intentionally.
On the bright sight, switching between netlib-lapack and openblas should
be possible, now.
Cheers,
Egbert.
>
> Kind regards,
>
> Stefan
>
> --
> Stefan Brüns / Bergstraße 21 / 52062 Aachen
> home: +49 241 53809034 mobile: +49 151 50412019
>
--
Egbert Eich (Res. & Dev.) SUSE Labs - Architect HPC
Tel: +49 911-740 53 0 https://www.suse.com
-----------------------------------------------------------------------
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg
Geschaeftsfuehrer: Ivo Totev
(HRB 36809, AG Nürnberg)
Hi,
someone has apparently disabled *all* builds for openblas.
Whats the reason for this?
As openblas 0.3.20 was never built inside science, we now have 0.3.20 in
Factory and 0.3.17 (binary packages) in science. This causes a lot of
unresolvables, as OBS tries to use openblas 0.3.17 from science.
Kind regards,
Stefan
--
Stefan Brüns / Bergstraße 21 / 52062 Aachen
home: +49 241 53809034 mobile: +49 151 50412019
Hi,
I decided not to be that much lazy and when I need to compile some software
(mostly for molecular biology, genetics, ...), I package it. So far it is in
https://build.opensuse.org/project/show/home:vojtaeus but I think it could be
useful to submit them to https://build.opensuse.org/project/show/science but
I'm still bit beginner with packaging (and biologist, not programmer), so I'm
not sure if I follow all best-practice guidelines and so...
Anyway, can I just do such a "mass submission" of my packages into science? Is
there any special procedure? Any review? I suppose I'm then maintainer of the
packages, but not of whole science repository...?
There are still issues with some packages I don't know how to solve now:
https://build.opensuse.org/package/show/home:vojtaeus/bamtools - I'm not sure
if I'm correctly handling shared library and don't really get why it works for
TW, but fails for 15.2 and 15.3... Similarly I fail with BEAGLE library (not
in OBS), which should add extra support for MrBayes.
https://build.opensuse.org/package/show/home:vojtaeus/bandage - I wonder if it
fails for 15.2 due to "ERROR: No sufficient Category definition" and "Errors
in installed desktop file detected.". Is this so different comparing to 15.3
and TW? Hint in build log doesn't really help me... :-/ I was also bit
confused when looking into various Qt applications by diversity of usage of
BuildRequires tags, but I hope I package it correctly... :-)
https://build.opensuse.org/package/show/home:vojtaeus/bwa - I don't really get
why it fails for i586 TW, but it's probably unimportant. ;-)
https://build.opensuse.org/package/show/home:vojtaeus/seaview - I wonder if
failures for 15.2 and 15.3 are due to errors in desktop file (which works for
TW). Also, it is supposed to deliver HTML help file, which I think I package
correctly (well, lines 48 and 49 of spec file can be probably dropped, the
don't do what I was hoping for), but it fails to find it. Interestingly, when
I packaged the RPM locally with this spec file, it can find the help, but not
when I do it in OBS...
I have also list of things in Java, Maven, bazel, and more which I'd like to
add. Especially with thinks like IGV, GATK or Picard I don't really have any
idea, how to compile it... We'll see :-)
So can I just submit what I have into science, or am I supposed to undergo
some sophisticated procedure? :-)
Yours,
V.
--
Vojtěch Zeisek
https://trapa.cz/
Komunita openSUSE GNU/Linuxu
Community of the openSUSE GNU/Linux
https://www.opensuse.org/
Hi Nicolas!
Nicolas Morey-Chaisemartin writes:
>
>
> Le 10/22/20 à 10:57 AM, Egbert Eich a écrit :
> >
> > Hi Nicolas,
> >
> > Nicolas Morey-Chaisemartin writes:
> > >
> > >
>
> > Is this available or all MPI flavors? If so, we could just do a:
> > BuildRequires: %{mpi}-macros-devel
> >
>
> No it's only for openmpi* packages at the moment.
Ok, if you can fix this for mpich and mvapich we can integrate this
into more packages.
I'd be easier if we were able to do things the same way for all
MPI flavors.
> > The define of %mpi should be avaiable to the build dependency
> > script as it doesn't depend on other macro packages.
> > If this is the case I will make adjustments when visiting packages
> > again.
> >
> > > It'll provide the macros:
> > > %setup_openmpi # (Load the orpper mpivars.sh)
> > > %openmpi_requires # ( Add the right Requires:..... for the openmpi package)
> > > %openmpi_devel_requires # same but for devel packages)
> > >
> > > Also for using the "default" version of openmpi (currently openmpi2), you can juste BuildRequires: openmpi-macros-devel.
> > > It'll pull the macros from the default version so there is no need to specify which one and it can be changed more easily when we decide to go to openmpi 3 or 4 by default.
> >
> > Right. Ok, this should work as well - if 'openmpi' stands for
> > 'openmpi-default' instead of 'openmpi1' it'd be fine.
Right.
> It does. That's the reason openmpi was renamed to openmpi1
>
> We will
> > build for openmpi2 twice then: once for 'default' and once for
> > it explicitly and we don't build for openmpi1 at all. But maybe
> > people don't care any more ...
> >
>
> You probably don't need that. The default flavor is mostly for package which don't really care which one they use (stuff like boost and other core libs) but you need all of them to use the same one.
> It was introduced as there was a huge mess of some libs building against openmpi1, some against openmpi2 so the stack wasn't working.
> Package like lapack where the pick of flavor might matter should build against all, not necesseraly against the default one.
Yes. I would think for many of the packages in science this is true.
With multibuild this will be quite easy - one does not necessarily
need all the HPC stuff with full environment module support.
Thanks!
Cheers,
Egbert.--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
Hi Nicolas,
Nicolas Morey-Chaisemartin writes:
>
>
> Le 10/21/20 à 9:58 AM, Nicolas Morey-Chaisemartin a écrit :
> >
> >
> > Le 10/21/20 à 9:35 AM, Egbert Eich a écrit :
> >>
> >> Hi Alin,
> >>
> >> please specify what system you are using (Leap<version>, Tumbleweed ...).
> >> It seems you are mixing HPC (openmpi4-gnu-hpc-devel) and non-HPC packages.
> >> Moreover, there is no scalapack-openmpi4 (not even among the non-HPC
> >> packages).
> >>
> >> There is a bit of a problem with MPI depenendencies with non-HPC packages.
> >> It seems like both openmpi3 and 4 provide libmpi.so.40()(64bit).
> >> @Nicolas - is this correct?
> >
> > Yes they do.
> > But in theory, packages depending on a specific MPI version like this should
> > require the right openmpi, not just the libmpi.so
Yes, this is what we do for the HPC versions. We even strip these
'generic' library dependencies from the list using our own dependency
generator.
> > WHich lapack does not seem to be doing/
> > BuildRequires: %{mpi_flavor}%{?mpi_ext}-devel
> >
> > It's only a BuildRequires but a
> > Requires: %{mpi_flavor}%{?mpi_ext}-libs is also needed.
>
> Scratch some of that (it works but there is a newer/better way)
> openmpi* packages were cleaned up some time back.
> What should be done for all openmpi (at least non hpc) dependant packages is:
> BuildRequires: openmpi<N>-macros-devel
Ok.
Is this available or all MPI flavors? If so, we could just do a:
BuildRequires: %{mpi}-macros-devel
The define of %mpi should be avaiable to the build dependency
script as it doesn't depend on other macro packages.
If this is the case I will make adjustments when visiting packages
again.
> It'll provide the macros:
> %setup_openmpi # (Load the orpper mpivars.sh)
> %openmpi_requires # ( Add the right Requires:..... for the openmpi package)
> %openmpi_devel_requires # same but for devel packages)
>
> Also for using the "default" version of openmpi (currently openmpi2), you can juste BuildRequires: openmpi-macros-devel.
> It'll pull the macros from the default version so there is no need to specify which one and it can be changed more easily when we decide to go to openmpi 3 or 4 by default.
Right. Ok, this should work as well - if 'openmpi' stands for
'openmpi-default' instead of 'openmpi1' it'd be fine. We will
build for openmpi2 twice then: once for 'default' and once for
it explicitly and we don't build for openmpi1 at all. But maybe
people don't care any more ...
Thanks!
Cheers,
Egbert.
>
> Nicolas
>
> >
> >
> > Nicolas
> >
> >>
> >> For the HPC variants this has been fixed.
> >> If both openmpi3 and openmpi4 provide the libmpi.so.4 ABI, then you
> >> will have to install openmpi4-libs explicitly, I'm afraid - or use
> >> the HPC variant of scalapack.
> >>
> >> Cheers,
> >> Egbert.
> >>
> >> Alin Marin Elena writes:
> >> > Hi.
> >> >
> >> > I try to install openmpi4 but somehow openmpi3 is pulled see below.
> >> >
> >> > drFaustroll@circassia:~> sudo zypper in scalapack-openmpi4
> >> > openmpi4-gnu-hpc-devel
> >> > Problem retrieving files from 'oneAPI'.
> >> > Permission to access
> >> > 'https://yum.repos.intel.com/oneapi/media.1/media' denied.
> >> > Please see the above error message for a hint.
> >> > Warning: Skipping repository 'oneAPI' because of the above error.
> >> > Some of the repositories have not been refreshed because of an error.
> >> > Loading repository data...
> >> > Reading installed packages...
> >> > 'scalapack-openmpi4' not found in package names. Trying capabilities.
> >> > Resolving package dependencies...
> >> >
> >> > The following 31 NEW packages are going to be installed:
> >> > gcc-c++ gnu-compilers-hpc gnu-compilers-hpc-devel infiniband-diags
> >> > libfabric1 libibmad5 libibnetdisc5 libibumad3 libinfinipath4
> >> > libopenmpi_4_0_5-gnu-hpc
> >> > libpsm2-2 libpsm_infinipath1 libscalapack2-openmpi4 libucm0 libucp0
> >> > libucs0 libuct0 lua53 lua53-luafilesystem lua53-luaposix lua53-luaterm
> >> > lua-lmod
> >> > mpi-selector openmpi3 openmpi3-libs openmpi_4_0_5-gnu-hpc
> >> > openmpi_4_0_5-gnu-hpc-devel openmpi4-config openmpi4-gnu-hpc-devel
> >> > rdma-core-devel rsocket
> >> >
> >> > The following recommended package was automatically selected:
> >> > openmpi4-config
> >> >
> >> > 31 new packages to install.
> >> >
> >> > Without Questions there are no Answers!
> >> > ______________________________________________________________________
> >> > Dr. Alin Marin ELENA
> >> > http://alin.elena.space/
> >> > ______________________________________________________________________
> >> > --
> >> > To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
> >> > To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
> >>
--
Egbert Eich (Res. & Dev.) SUSE Labs - Project Manager HPC
Tel: +49 911-740 53 0 https://www.suse.com
-----------------------------------------------------------------------
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg
Geschaeftsfuehrer: Felix Imendoerffer
(HRB 36809, AG Nürnberg)
--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
Hi Alin,
please specify what system you are using (Leap<version>, Tumbleweed ...).
It seems you are mixing HPC (openmpi4-gnu-hpc-devel) and non-HPC packages.
Moreover, there is no scalapack-openmpi4 (not even among the non-HPC
packages).
There is a bit of a problem with MPI depenendencies with non-HPC packages.
It seems like both openmpi3 and 4 provide libmpi.so.40()(64bit).
@Nicolas - is this correct?
For the HPC variants this has been fixed.
If both openmpi3 and openmpi4 provide the libmpi.so.4 ABI, then you
will have to install openmpi4-libs explicitly, I'm afraid - or use
the HPC variant of scalapack.
Cheers,
Egbert.
Alin Marin Elena writes:
> Hi.
>
> I try to install openmpi4 but somehow openmpi3 is pulled see below.
>
> drFaustroll@circassia:~> sudo zypper in scalapack-openmpi4
> openmpi4-gnu-hpc-devel
> Problem retrieving files from 'oneAPI'.
> Permission to access
> 'https://yum.repos.intel.com/oneapi/media.1/media' denied.
> Please see the above error message for a hint.
> Warning: Skipping repository 'oneAPI' because of the above error.
> Some of the repositories have not been refreshed because of an error.
> Loading repository data...
> Reading installed packages...
> 'scalapack-openmpi4' not found in package names. Trying capabilities.
> Resolving package dependencies...
>
> The following 31 NEW packages are going to be installed:
> gcc-c++ gnu-compilers-hpc gnu-compilers-hpc-devel infiniband-diags
> libfabric1 libibmad5 libibnetdisc5 libibumad3 libinfinipath4
> libopenmpi_4_0_5-gnu-hpc
> libpsm2-2 libpsm_infinipath1 libscalapack2-openmpi4 libucm0 libucp0
> libucs0 libuct0 lua53 lua53-luafilesystem lua53-luaposix lua53-luaterm
> lua-lmod
> mpi-selector openmpi3 openmpi3-libs openmpi_4_0_5-gnu-hpc
> openmpi_4_0_5-gnu-hpc-devel openmpi4-config openmpi4-gnu-hpc-devel
> rdma-core-devel rsocket
>
> The following recommended package was automatically selected:
> openmpi4-config
>
> 31 new packages to install.
>
> Without Questions there are no Answers!
> ______________________________________________________________________
> Dr. Alin Marin ELENA
> http://alin.elena.space/
> ______________________________________________________________________
> --
> To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
> To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
--
Egbert Eich (Res. & Dev.) SUSE Labs - Project Manager HPC
Tel: +49 911-740 53 0 https://www.suse.com
-----------------------------------------------------------------------
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg
Geschaeftsfuehrer: Felix Imendoerffer
(HRB 36809, AG Nürnberg)
--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
Hi.
I try to install openmpi4 but somehow openmpi3 is pulled see below.
drFaustroll@circassia:~> sudo zypper in scalapack-openmpi4
openmpi4-gnu-hpc-devel
Problem retrieving files from 'oneAPI'.
Permission to access
'https://yum.repos.intel.com/oneapi/media.1/media' denied.
Please see the above error message for a hint.
Warning: Skipping repository 'oneAPI' because of the above error.
Some of the repositories have not been refreshed because of an error.
Loading repository data...
Reading installed packages...
'scalapack-openmpi4' not found in package names. Trying capabilities.
Resolving package dependencies...
The following 31 NEW packages are going to be installed:
gcc-c++ gnu-compilers-hpc gnu-compilers-hpc-devel infiniband-diags
libfabric1 libibmad5 libibnetdisc5 libibumad3 libinfinipath4
libopenmpi_4_0_5-gnu-hpc
libpsm2-2 libpsm_infinipath1 libscalapack2-openmpi4 libucm0 libucp0
libucs0 libuct0 lua53 lua53-luafilesystem lua53-luaposix lua53-luaterm
lua-lmod
mpi-selector openmpi3 openmpi3-libs openmpi_4_0_5-gnu-hpc
openmpi_4_0_5-gnu-hpc-devel openmpi4-config openmpi4-gnu-hpc-devel
rdma-core-devel rsocket
The following recommended package was automatically selected:
openmpi4-config
31 new packages to install.
Without Questions there are no Answers!
______________________________________________________________________
Dr. Alin Marin ELENA
http://alin.elena.space/
______________________________________________________________________
--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
