Hi Peter,
I did overlook your message as at the time you wrote it, I was
extremely busy to get a release out.
I need to apologize for the delay and that I'm only answering now.
Peter Hill writes:
> Hello,
>
> I just noticed the gnu-hpc packages in Tumbleweed recently, and I was
> wondering if there is an explanation of how they work somewhere? I had
> a look for some information or discussion, but I've not been able to
> find anything so far.
Right, I should probably have blogged about this - and I may still do.
>
> I gather they are linked somehow to environment modules, but it's not
> clear to me how exactly. For instance, "petsc-devel" installs a module
> for PETSc, but "hdf5-devel" does not install one for HDF5. Are there
> certain packages we're supposed to install to get this functionality?
This could have been a bug that has been fixed meanwhile.
Let me get a very brief overview:
The theory is that
1. the runtime library package installs a module.
2. the devel package pulls in the runtime library package and any
other package that may be needed for building.
3. There are packages which have the version numbers in the package
name. Other packages don't - these are the 'master packages' which
will pull in the latest version of a package thru a dependency.
This allows to have mutliple version of the same base code.
4. Environment modules are used to select toolchain flavor/version,
MPI flavor/version and packages and versions. By default, the latest
version of a package is loaded, however, you can specify a version
explicitly if you need an older version.
The modules enforce some dependencies between libraries, ie you will
have to select a toolchain for most of the libraries and you need
to select the MPI version for MPI dependent libraries. Without this,
these libraries will not become available.
5. For environment modules we use Lmod:
https://lmod.readthedocs.io/en/latest/
There is the 'module' command available to load/unload, list, switch
the modules. To get a help, call 'module help', modules are loaded
with 'module load <module_name>[/<version>]', to get a list of
accessible modules, do 'module avail', to get a list of all modules,
call'module spider'.
> Also, are they supposed to play nice with pkg-config? petsc-devel does
> install PETSc.pc, but loading the module doesn't update
> $PKG_CONFIG_PATH to be able to find it.
This is a bug as well, lemme check if it is still there ...
The module file of the package I'm looking at right now has:
prepend-path PKG_CONFIG_PATH /usr/lib/hpc/gnu7.2/openmpi3/petsc/3.7.6/lib64/pkgconfig
Thus it should work.
If you find any more issues, please let me know - or create a ticket
on bugzilla.opensuse.org.
Please add hpc-maintainer-bugs(a)forge.provo.novell.com as the assignee.
Plan is to provide these packages with openSUSE Leap 42.3. This should
give you a more stable (and enterprise grade!) environment than openSUSE
Tumbleweed.
Cheers,
Egbert.
--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
Adrian Schröter writes:
> On Freitag, 19. Januar 2018, 15:16:04 CET wrote Egbert Eich:
> >
> > Yeah, I didn't want to point to you directly ;)
>
> ah, the shy Egbert ...
Hehe ;)
> > You mean the use-for-build flags? I was under the *assumption* that
> > hdf5 has them set. It would explain why things break for the HPC
> > flavors of the netcdf build: the hdf5.1.8 package doesn't generate
> > these flavors as the package has not been converted.
>
> no, they were set to use only the 1.8 version.
>
> This was (also?) be done since 1.10 has a different on disk format.
> While it should still be able to read 1.8 it will break the workflow
> for people who share the their files (as FreeCAD users).
FreeCAD doesn't look like the classic HPC application while HDF5 is
specifically targetting the problems of HPC users and environments
- storing very large files on cluster file systems.
These problems arise because this file format library is used for
use cases it was never intended for :/
> > > I really should have written down the reason for the 5-1.8 package
> > > in the package description, but I didn't :/
> > >
> > > So, I will reconfigure it to use the 5-1.10 version now to see
> > > what was the reason behind (most likely opencascade & FreeCAD).
> >
> > We do have hdf5 1.10.1 in Factory only, so anyone needing to build
> > against hdf5 and is available in Factory would do so already.
>
> k, will keep an eye on the project. 5-1.8 is currently disabled and
> not used, just not dropped yet ...
As I said, we can keep it, but the package should be renamed (and packages
that depend on it should explicitely use this package name to make sure
they get the 'stable' version).
Cheers,
Egbert.
--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
Adrian Schröter writes:
> On Freitag, 19. Januar 2018, 15:26:02 CET wrote Adrian Schröter:
> > On Freitag, 19. Januar 2018, 15:16:04 CET wrote Egbert Eich:
> ...
>
> med-tools are failing now, because they explicit warn about 1.10 due
> to on disk incompabilities:
>
> [ 206s] This HDF5 version 1.10.1 must not be used with med-fichier3.2.1.
> [ 206s] The HDF5 library version used by med-fichier3.y.z MUST NOT be > 1.8 and have to be at least 5-1.8.11.
> [ 206s] DO NOT TRY TO COMPILE med-fichier3.2.1 version with an HDF5 library which would generate an hdf5 file not compliant with HDF5-1.8.z library.
> [ 206s] This would BREAK med-fichier compatibility between files with the same revision number !
>
>
> I can of course patch this but it feels a bit bad if they warn that way...
>
> Will try to find out more about it and leave the build broken atm,
> so the published repository should not build anything what breaks the
> files a user is storing to disk...
>
Let's include the version number into the package name of hdf5.1.8 then.
This is what we do for HPC packages all the time. If hdf5 1.8 is considered to
be some sort of reference base, we could flag it as such and build packages
that require a stable reference base on it. This still doesn't give you files
which you can exchange with your friends who are using tools from other distros
with different 'reference bases' ...
In theory, we could use environment modules to make other packages available
only for the hdf5 version they have been built for. Our packaging macros for
HPC are currently not set up for this. It could be done but would make things
even more ugly.
I at least hope these packages do not use the MPI version of hdf5.
Generally, in an HPC environment, these issues don't arise as people stick
to the version they have started their project with. This is why for a distro
(like Leap) we keep the old versions around.
Cheers,
Egbert.
--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
Hi Adrian!
Adrian Schröter writes:
> On Freitag, 19. Januar 2018, 14:41:29 CET wrote Egbert Eich:
> >
> > Hi!
> >
> > About 7 month ago, hdv5 v 1.8 was copypac-ed from science:unstable
> > to science:
> > https://build.opensuse.org/package/revisions/science/hdf5.1.8
> >
> >
> > The package (directory-)name was set to hdf5.1.8 to not
> > conflict with the already existing package 'hdf5, which
> > was updated to 1.10 roughly 1.5 years ago already.
>
> right, was me, I believe ;)
Yeah, I didn't want to point to you directly ;)
> > Unfortunately, the spec file and package names do not
> > carry the ".1.8".
> > Unfortunately, due to not carrying thru the naming scheme
> > to the package name inside the spec-file, both packages
> > generate the same files - with different version numbers.
> > This breaks the build of packages depending on the hdf5-
> > package which is in Factory.
>
> This is true, but bother versions are available for download.
> You can only build against one of them though.
For netcdf
$ osc -A https://api.opensuse.org results -v science netcdf:gnu-hpc \
-a x86_64 -r openSUSE_Tumbleweed
returns:
openSUSE_Tumbleweed x86_64 unresolvable: nothing provides hdf5-devel-data = 1.10.1 needed by hdf5_1_10_1-gnu-hpc-devel
You are right, on download.opensuse.org, both versions are available:
$ wget -q -O - http://download.opensuse.org/repositories/science/openSUSE_Tumbleweed/x86_6… \
| grep hdf5-devel-data
[..]
<a href="hdf5-devel-data-1.8.17-2.13.x86_64.rpm">hdf5-devel-data-1.8.17-2.13.x86_64.rpm</a>
[..]
<a href="hdf5-devel-data-1.10.1-72.1.x86_64.rpm">hdf5-devel-data-1.10.1-72.1.x86_64.rpm</a>
[..]
> > The package has no maintainer, so there is noone to ask to
> > fix this.
> >
> > If anyone here still needs version 1.8 of hdf5, please speak
> > up and fix this package *ASAP*!
> >
> > If not, I will assume there is noone who cares any more and
> > I will request deletion.
> >
> > For now, I've disabled the build.
>
> Does not change much, only the 5.1.8 is used for building against
> atm via the flags.
You mean the use-for-build flags? I was under the *assumption* that
hdf5 has them set. It would explain why things break for the HPC
flavors of the netcdf build: the hdf5.1.8 package doesn't generate
these flavors as the package has not been converted.
> I really should have written down the reason for the 5-1.8 package
> in the package description, but I didn't :/
>
> So, I will reconfigure it to use the 5-1.10 version now to see
> what was the reason behind (most likely opencascade & FreeCAD).
We do have hdf5 1.10.1 in Factory only, so anyone needing to build
against hdf5 and is available in Factory would do so already.
Thanks for your quick reply!
Cheers,
Egbert.
--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
Hi!
About 7 month ago, hdv5 v 1.8 was copypac-ed from science:unstable
to science:
https://build.opensuse.org/package/revisions/science/hdf5.1.8
The package (directory-)name was set to hdf5.1.8 to not
conflict with the already existing package 'hdf5, which
was updated to 1.10 roughly 1.5 years ago already.
Unfortunately, the spec file and package names do not
carry the ".1.8".
Unfortunately, due to not carrying thru the naming scheme
to the package name inside the spec-file, both packages
generate the same files - with different version numbers.
This breaks the build of packages depending on the hdf5-
package which is in Factory.
The package has no maintainer, so there is noone to ask to
fix this.
If anyone here still needs version 1.8 of hdf5, please speak
up and fix this package *ASAP*!
If not, I will assume there is noone who cares any more and
I will request deletion.
For now, I've disabled the build.
Cheers,
Egbert.
--
Egbert Eich (Res. & Dev.) SUSE LINUX GmbH
SUSE Labs - Project Manager HPC
Tel: +49 911-740 53 0 http://www.suse.com
-----------------------------------------------------------------
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
Maxfeldstraße 5, D90409 Nürnberg, Germany--
To unsubscribe, e-mail: opensuse-science+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-science+owner(a)opensuse.org
