CUDA as a build dependency
Hi all. I was hoping to get some advice on how to handle CUDA as a dependency for a package build on the openSUSE Build Service. A bit of background first. My packages are at https://build.opensuse.org/project/show/home:lawmurray:birch. There is a package there called numbirch that contains numerical routines that can run on either CPU or GPU, using CUDA in the latter case (much like e.g. PyTorch and TensorFlow). Currently I package the CPU version only. Users of the GPU version must currently build from source themselves. For ease of use, I am hoping to find a way to package the GPU version too. To do this I would need to have CUDA as a build dependency. Not a runtime dependency mind you, I think it's fine to expect users to install CUDA separately via their preferred means (Nvidia's repository or runfile installer, see https://developer.nvidia.com/cuda-downloads) to make required shared libraries available at runtime. But I would need to have it available at build time to access the nvcc compiler that CUDA provides. I made an attempt to create a wrapper package for CUDA here: https://build.opensuse.org/package/show/home:lawmurray:birch/cuda-numbirch. The idea was to download the CUDA installer at build time, install it together with a wrapper script that is necessary later, and then I could include this wrapper package as a build dependency for the numbirch package. I quickly discovered that this won't work, as network connections are restricted (understandable). An alternative would be to include the CUDA runfile installer in the package sources. I think that should work technically, although it's large, and unclear to me if having proprietary code included in a package like that is permitted on the openSUSE Build Service. There are some repository-specific approaches that could be taken also. Arch seems to include CUDA in its community repository, and I believe Ubuntu has it available in its standard repositories too. I will experiment with some of this. I'd appreciate any guidance that the community can give on how to proceed.
Following up on the repository-specific approaches, Arch works with the cuda package in its Community repository, whereas Ubuntu does not (nvidia-cuda-toolkit appears in its Multiverse repository, which contains restricted software and seems unavailable on openSUSE Build Service). I don't believe any of the other distributions have CUDA packages available outside of Nvidia's repositories.
On Fri 25 Nov 2022 08:53:57 PM CST, Lawrence Murray wrote:
Following up on the repository-specific approaches, Arch works with the cuda package in its Community repository, whereas Ubuntu does not (nvidia-cuda-toolkit appears in its Multiverse repository, which contains restricted software and seems unavailable on openSUSE Build Service). I don't believe any of the other distributions have CUDA packages available outside of Nvidia's repositories.
Hi Perhaps leave it to the end user to rebuild locally? You could add a build condition wrapper, eg %bcond_with nvcc Then users can pass this through when running rpmbuild locally? See what is done with libheif, then when built by packman the project config defines x256. https://build.opensuse.org/package/view_file/multimedia:libs/libheif/libheif... -- Cheers Malcolm °¿° (Linux Counter #276890) Tumbleweed 20221123 | GNOME Shell 43.1 | 6.0.8-1-default HP Z440 | Xeon E5-2690 V3 X24 @ 2.60GHz | Nvidia Quadro T400/Nvidia Tesla P4 up 3 days 19:12, 2 users, load average: 0.04, 0.16, 0.25
Short update, I've had a look at _service files as well, which allow fetching the CUDA installer before builds. That provides a technical solution, but publishing the CUDA installer on OBS once fetched would violate its EULA on redistribution I believe. For what it's worth, I added a feature request to try to explain the issue and some suggestions to address it, but these may be naive: https://github.com/openSUSE/open-build-service/issues/13432 Still looking at Malcolm's suggestion and rebuild options for Debian packages too. Appreciate the help.
participants (2)
-
Lawrence Murray
-
Malcolm