Re: [opensuse-science] gnu-hpc packages
Hi Stefan, Stefan Brüns writes:
On Montag, 22. Januar 2018 22:11:12 CEST Egbert Eich wrote:
Hi Peter,
Any chance you can write up some documentation about this?
There is some documentation available at: https://github.com/openSUSE/hpc however, no user documentation, yet. More you can find in the release notes of the SLE-12 HPC modules: https://www.suse.com/releasenotes/x86_64/SLE-Module-HPC/12/ An update to this is still pending. All this information should be provided for Leap as well. Volunteers to help me there are welcome :)
Currently, I have a few questions
A: Buildtime Every package with an MPI dependency should be build in several flavours. Each flavour has buildtime requirements on other packages, if these dependencies are flavoured as well, the flavours of all buildrequires match. Is this correct?
Yes. If this is not the case for a package it would be a bug.
Currently, there a some packages which only build the traditional openmpi flavour (e.g. science/scotch), some only have gnu-*-hpc flavours (e.g. petsc), and some have both (e.g. hdf5).
We've tried to keep the 'traditional' build if there was one. Especially, if a package has been used by other packages which are not related to HPC. Generally, the goal was to not break exisitng setups. petsc is different: it existed in the develproject 'science' but not in Factory, thus I've tried to maintain the 'traditional' build. Since this was failing due to missing dependencies, I didn't bother to fix it myself. The build is still available in case anyone wants to fix it. Recently, I've 'recycled' the 'traditional' build to generate the doc package only, this can easily be reverted should anybody be interested to fix this build. It seems like the package as it was before the conversion had been abandoned.
B: Runtime The environment is setup from the module. Installing a package pulls in the requirements in the same flavour.
B1: Is there a default flavour? There is non-MPI and openmpi, and now all the hpc named variants as well.
There is no default flavor per se. You will have to load the compiler and (where applicable) the MPI flavor. Although we presently support only the gnu compiler toolchain we haven't made this the dafault as we will be supporting multiple versions of this toolchain on Leap/SLE-HPC.
B2: How is the system kept consistent? What happens if the user installs a package depending on e.g. libhdf5? Currently, it is possible to install libhdf5-101-openmpi instead of libhd5-101, which both provide 'libhdf5.so.101()(64bit)', but doing so will break any program linking to libhdf5.
The packages you have mentioned are 'traditional' ones. Nothing should have changed there: programs linked against libhdf5 (without MPI) should still work like before. As the 'traditional' MPI variant already required module support, these libs will not be found by an application that doesn't ask for them explicitly. There is an issue with traditional libs - and this existed before we started to add the HPC variants: the RPM dependency generator will generate the same provides for the non-MPI and MPI variants. zypper will pick the variant which happens to be first in its list and install it - thus this might lead to inconsistencies. I do consider this a bug in the dependency generator - which I've addressed for the HPC variants - as this is what I'm interested the most and where I have the most flexibiliy. For the 'traditional' builds I've tried to take it up with the rpm maintainer, however, I've never received a reply. If you obeserve issues with getting the wrong library versions installed, I'd suggest to open a ticket on bugzilla where you describe the problem. This can then be amended with an analysis of the issue and be assigned to the rpm maintainer.
(Possible packaging bug - libhdf5-101-*openmpi* pulls in mpi-selector and *mpich*, while this should probably be *openmpi*).
This is a slightly different issue, but related. It has existed before we did the HPC conversion and is unrelated to it: libhdf5-101-*openmpi* is a 'traditional' package. If you check its requirements - it will pull in libmpi.so.12. This requirement will be satisifed by any MPI flavor available. Thus, zypper will pick any of the existing one. Again, this has been fixed for the HPC variants, not for the 'traditional' build: Our goal was to not cause more breakages in the 'traditional' builds to allow things that worked to keep as before. If you need to use the MPI variant of hdf5 I'd recommend to use the HPC flavor (downside, you will have to use it for everything) as we have tried to design this much more consistently. If you discover inconsistencies there, I'd be interested to hear about them! Cheers, Egbert. -- Egbert Eich (Res. & Dev.) SUSE LINUX GmbH SUSE Labs - Project Manager HPC Tel: +49 911-740 53 0 http://www.suse.com ----------------------------------------------------------------- SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5, D90409 Nürnberg, Germany-- To unsubscribe, e-mail: opensuse-science+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-science+owner@opensuse.org
participants (1)
-
Egbert Eich