Plans for packaging numpy 2.x
Hi, With numpy 2.0 ostensibly releasing soon [1] and breaking backward compatibility with numpy 1.2X [2], I wonder what the plan for numpy packages will be for openSUSE. Will we maintain two different series of numpy as, for instance, `python-numpy` and `python-numpy2`? This will perhaps be the least breaking path to take for packages dependent on `numpy` — which will probably take a while to catch up — but I understand if the additional maintenance efforts this will require compel otherwise. Also, as someone who maintains a few packages in d:l:p:numeric, I would be happy to help out if needed (maintaining the 1.x branch or any other way). Thanks for the great work you folks do maintaining the seemingly infinite number of python modules as RPM packages. Cheers, [1] https://github.com/numpy/numpy/releases/tag/v2.0.0rc1 [2] https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-abi-handling -- Atri
On vie, 2024-04-12 at 14:44 +0000, Atri Bhattacharya wrote:
Hi, With numpy 2.0 ostensibly releasing soon [1] and breaking backward compatibility with numpy 1.2X [2], I wonder what the plan for numpy packages will be for openSUSE. Will we maintain two different series of numpy as, for instance, `python-numpy` and `python-numpy2`? This will perhaps be the least breaking path to take for packages dependent on `numpy` — which will probably take a while to catch up — but I understand if the additional maintenance efforts this will require compel otherwise.
Also, as someone who maintains a few packages in d:l:p:numeric, I would be happy to help out if needed (maintaining the 1.x branch or any other way).
Thanks for the great work you folks do maintaining the seemingly infinite number of python modules as RPM packages.
Cheers,
[1] https://github.com/numpy/numpy/releases/tag/v2.0.0rc1 [2] https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-abi-handling
-- Atri
I would go for keeping python-numpy as the latest version (in this case, 2.0) and create a new package called python-numpy_1. Any package that depends on old numpy can just add the version limit to the spec file like: Requires: python-numpy < 2 And we can keep the old package around for some time, for packages that depends on it or even if there are people using version 1 directly, until upstream drop maintenance support completely. We're doing this for python-urllib3 with python-urllib3_1 [1], so it's a similar pattern. [1] https://build.opensuse.org/package/show/openSUSE:Factory/python-urllib3_1 -- Daniel García Moreno Python Packager
daniel.garcia＠suse.com wrote:
On vie, 2024-04-12 at 14:44 +0000, Atri Bhattacharya wrote:
Hi, With numpy 2.0 ostensibly releasing soon [1] and breaking backward compatibility with numpy 1.2X [2], I wonder what the plan for numpy packages will be for openSUSE. Will we maintain two different series of numpy as, for instance, `python-numpy` and `python-numpy2`? This will perhaps be the least breaking path to take for packages dependent on `numpy` — which will probably take a while to catch up — but I understand if the additional maintenance efforts this will require compel otherwise. Also, as someone who maintains a few packages in d:l:p:numeric, I would be happy to help out if needed (maintaining the 1.x branch or any other way).
I would go for keeping python-numpy as the latest version (in this case, 2.0) and create a new package called python-numpy_1. Any package that depends on old numpy can just add the version limit to the spec file like:
Requires: python-numpy < 2
..which would entail manual fixes for a rather large number of specfiles. What would be the benefit, if I may ask, as opposed to using `python-numpy2` as I suggested above? Thanks for your response. -- Atri
On lun, 2024-04-15 at 20:37 +0000, Atri Bhattacharya wrote:
daniel.garcia＠suse.com wrote:
On vie, 2024-04-12 at 14:44 +0000, Atri Bhattacharya wrote:
Hi, With numpy 2.0 ostensibly releasing soon [1] and breaking backward compatibility with numpy 1.2X [2], I wonder what the plan for numpy packages will be for openSUSE. Will we maintain two different series of numpy as, for instance, `python-numpy` and `python-numpy2`? This will perhaps be the least breaking path to take for packages dependent on `numpy` — which will probably take a while to catch up — but I understand if the additional maintenance efforts this will require compel otherwise. Also, as someone who maintains a few packages in d:l:p:numeric, I would be happy to help out if needed (maintaining the 1.x branch or any other way).
I would go for keeping python-numpy as the latest version (in this case, 2.0) and create a new package called python-numpy_1. Any package that depends on old numpy can just add the version limit to the spec file like:
Requires: python-numpy < 2
..which would entail manual fixes for a rather large number of specfiles. What would be the benefit, if I may ask, as opposed to using `python- numpy2` as I suggested above?
The benefit is that we will keep the upstream name for the package and in the future we just need to drop python-numpy_1. If we do it the other way, at some point we will need to rename the package or we will keep the package name mismatch forever. I mean, if upstream doesn't change the name to numpy2, all the projects that depends on it should also update their dependencies to pin the version when it's released. I would like to keep the package name coherent with the upstream name provided in pypi.org -- Daniel García Moreno Python Packager
Am 16.04.24 um 07:30 schrieb daniel.garcia@suse.com:
daniel.garcia＠suse.com wrote:
I would go for keeping python-numpy as the latest version (in this case, 2.0) and create a new package called python-numpy_1. Any package that depends on old numpy can just add the version limit to the spec file like:
Requires: python-numpy < 2
..which would entail manual fixes for a rather large number of specfiles. What would be the benefit, if I may ask, as opposed to using `python- numpy2` as I suggested above? The benefit is that we will keep the upstream name for the package and in
On lun, 2024-04-15 at 20:37 +0000, Atri Bhattacharya wrote: the future we just need to drop python-numpy_1. If we do it the other way, at some point we will need to rename the package or we will keep the package name mismatch forever.
I mean, if upstream doesn't change the name to numpy2, all the projects that depends on it should also update their dependencies to pin the version when it's released.
= %{min_numpy_ver} with python-numpy < %{max_numpy_ver})
+1. Many packages do already pin to numpy < 2. The proposed scheme mirrors the upstream logic and avoids confusion. Name: python-numpy Version: 2.0 Provides: python-numpy = %{version}-%{release} Conflicts: python-numpy1 %python_subpackages Name: python-numpy1 Version: 1.26.4 Provides: python-numpy = %{version}-%{release} Conflicts: python-numpy %python_subpackages https://build.opensuse.org/projects/openSUSE:Factory/prjconf Prefer: -python310-numpy1 -python311-numpy1 -python312-numpy1 [ben@skylab:…osc/bmwiedemann/openSUSE]% git grep 'numpy <' packages/*/*/*.spec packages/a/apache-arrow/python-pyarrow.spec:Requires: (python-numpy >= 1.16.6 with python-numpy < 2) packages/p/python-astropy/python-astropy.spec:Requires: (python-numpy >= 1.22 with python-numpy < 2) packages/p/python-matplotlib/python-matplotlib.spec:Requires: (python-numpy >= 1.21 with python-numpy < 2) packages/p/python-nptyping/python-nptyping.spec:Requires: (python-numpy >= 1.21.5 with python-numpy < 2) packages/p/python-nptyping/python-nptyping.spec:BuildRequires: %{python_module numpy >= 1.20.0 with %python-numpy < 2} packages/p/python-numba/python-numba.spec:# upper bound is exclusive: min-numpy_ver <= numpy < max_numpy_ver packages/p/python-numba/python-numba.spec:Requires: (python-numpy packages/p/python-numpy/python-numpy.spec:Conflicts: %{plainpython}-numpy <= 1.12.0 packages/p/python-pandas/python-pandas.spec:Requires: (python-numpy >= 1.23.2 with python-numpy < 2) packages/p/python-pandas/python-pandas.spec:Requires: (python-numpy >= 1.26 with python-numpy < 2) packages/p/python-scikit-learn/python-scikit-learn.spec:Requires: (python-numpy >= 1.19.5 with python-numpy < 2) packages/p/python-yt/python-yt.spec:Requires: (python-numpy >= 1.19.3 with python-numpy < 2) I just hope that we don't have to jump through too many hoops to make the hpc packages work. - Ben
Hi again, Am 16.04.24 um 21:05 schrieb Ben Greiner:
On lun, 2024-04-15 at 20:37 +0000, Atri Bhattacharya wrote:
daniel.garcia＠suse.com wrote:
I would go for keeping python-numpy as the latest version (in this case, 2.0) and create a new package called python-numpy_1. Any package that depends on old numpy can just add the version limit to the spec file like:
Requires: python-numpy < 2
I just hope that we don't have to jump through too many hoops to make
Am 16.04.24 um 07:30 schrieb daniel.garcia@suse.com: the hpc packages work.
- Ben
FYI I am currently experimenting with numpy 2.0.0rc1 in https://build.opensuse.org/project/show/home:bnavigator:numpy. I have prepared a python-numpy1 package in there, too. There is quite a large number of packages from d:l:p:numeric which fail to build or test when allowing numpy 2. Also, we have the shortcoming of obs to not properly resolve a pin to numpy < 2 in the dependency chain. (See the pandas test flavors for instance, they don't like the numba pinning). - Ben
Ben Greiner wrote:
Hi again,
FYI I am currently experimenting with numpy 2.0.0rc1 in https://build.opensuse.org/project/show/home:bnavigator:numpy. I have prepared a python-numpy1 package in there, too.
That's super helpful, big thanks.
There is quite a large number of packages from d:l:p:numeric which fail to build or test when allowing numpy 2.
Also, we have the shortcoming of obs to not properly resolve a pin to numpy < 2 in the dependency chain. (See the pandas test flavors for instance, they don't like the numba pinning).
On a related note, would it be useful to add Provides: python-numpy1 to the current python-numpy package in Factory, so we could already starting switching the Reqs and BReqs to python-numpy1 (instead of `python-numpy < 2`) for pkgs that will not build with numpy >= 2 right away? Cheers, -- Atri
Am 22.04.24 um 20:54 schrieb Atri Bhattacharya:
Ben Greiner wrote:
Hi again,
FYI I am currently experimenting with numpy 2.0.0rc1 in https://build.opensuse.org/project/show/home:bnavigator:numpy. I have prepared a python-numpy1 package in there, too.
That's super helpful, big thanks.
There is quite a large number of packages from d:l:p:numeric which fail to build or test when allowing numpy 2.
The list got a bit smaller, with scipy 1.13 becoming compatible.
Also, we have the shortcoming of obs to not properly resolve a pin to numpy < 2 in the dependency chain. (See the pandas test flavors for instance, they don't like the numba pinning).
On a related note, would it be useful to add Provides: python-numpy1 to the current python-numpy package in Factory, so we could already starting switching the Reqs and BReqs to python-numpy1 (instead of `python-numpy < 2`) for pkgs that will not build with numpy >= 2 right away?
I'd rather not have the custom name in any Requires or BuildRequires tag. Let's encourage packagers to pin to versions instead of non-standard rpm package names.
Cheers, -- Atri
- Ben
Hi, Please stop pinning to numpy < 2 for packages which clearly support it. Especially as long as numpy2 is not even in Factory (Pandas must be fixed, which is Ring1). Packages compiled with numpy2 also work with consumers requiring and installung numpy1, but not the other way around. - Ben Am 25.04.24 um 10:42 schrieb Ben Greiner:
Am 22.04.24 um 20:54 schrieb Atri Bhattacharya:
Ben Greiner wrote:
Hi again,
FYI I am currently experimenting with numpy 2.0.0rc1 in https://build.opensuse.org/project/show/home:bnavigator:numpy. I have prepared a python-numpy1 package in there, too.
That's super helpful, big thanks.
There is quite a large number of packages from d:l:p:numeric which fail to build or test when allowing numpy 2.
The list got a bit smaller, with scipy 1.13 becoming compatible.
Also, we have the shortcoming of obs to not properly resolve a pin to numpy < 2 in the dependency chain. (See the pandas test flavors for instance, they don't like the numba pinning).
On a related note, would it be useful to add Provides: python-numpy1 to the current python-numpy package in Factory, so we could already starting switching the Reqs and BReqs to python-numpy1 (instead of `python-numpy < 2`) for pkgs that will not build with numpy >= 2 right away?
I'd rather not have the custom name in any Requires or BuildRequires tag. Let's encourage packagers to pin to versions instead of non-standard rpm package names.
Cheers, -- Atri
- Ben
participants (3)
-
Atri Bhattacharya -
Ben Greiner -
daniel.garcia＠suse.com