[opensuse-packaging] d:l:python/python3 split logic
Greetings, I was under the naive impression that after the py2/3 split, the packages from the devel:languages:python OBS project are released into the devel:languages:python/python3 repos based on their name (everything /^python3-/ goes into d:l:python3); in other words, I wanted to configure my python3-only servers to only use the python3 repo. But apparently this simple logic is not what is happening, there are many python3-* packages in d:l:python. Can someone explain what is really happening? Is this a temporary state or will I have to keep enabling both python{,3} repos on my python3 servers? -- Kind regards Christopher 'm4z' Holm / 686f6c6d "We must respect the other fellow's religion, but only in the sense and to the extent that we respect his theory that his wife is beautiful and his children smart." --H. L. Mencken -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Mittwoch, 4. Juli 2018 19:11:49 686f6c6d wrote:
Greetings,
I was under the naive impression that after the py2/3 split, the packages from the devel:languages:python OBS project are released into the devel:languages:python/python3 repos based on their name (everything /^python3-/ goes into d:l:python3); in other words, I wanted to configure my python3-only servers to only use the python3 repo. But apparently this simple logic is not what is happening, there are many python3-* packages in d:l:python.
In fact, it's the other way around... There's a single repo d:l:python, that serves our needs (and a couple of subrepos that tries to reduce the sheer number of packages in a single repo). Most packages in d:l:py generate python2- and python3- packages from a single spec nowadays. This paves the way to generate pypy2 and/or pypy3 versions as well without touching the spec, all controlled with rpm macros.
Can someone explain what is really happening? Is this a temporary state or will I have to keep enabling both python{,3} repos on my python3 servers?
Just don't use the d:l:python3 repos anymore. If you depend on a package, that isn't in d:l:py (nor in d:l:py:{aws,azure,flask,...}, but in d:l:py3, you might want to help by converting it to single spec specification with the help of https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec or just ask. If you want to limit your systems to python3, don't install python2{,-base} nor any dependencies and add zypper locks. 15.0 and TW tend to prefer python3, but I never tried that, though. Cheers, Pete -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On 05/07/18 07:53, Hans-Peter Jansen wrote:
On Mittwoch, 4. Juli 2018 19:11:49 686f6c6d wrote:
Greetings,
I was under the naive impression that after the py2/3 split, the packages from the devel:languages:python OBS project are released into the devel:languages:python/python3 repos based on their name (everything /^python3-/ goes into d:l:python3); in other words, I wanted to configure my python3-only servers to only use the python3 repo. But apparently this simple logic is not what is happening, there are many python3-* packages in d:l:python.
In fact, it's the other way around... There's a single repo d:l:python, that serves our needs (and a couple of subrepos that tries to reduce the sheer number of packages in a single repo).
Most packages in d:l:py generate python2- and python3- packages from a single spec nowadays. This paves the way to generate pypy2 and/or pypy3 versions as well without touching the spec, all controlled with rpm macros.
Can someone explain what is really happening? Is this a temporary state or will I have to keep enabling both python{,3} repos on my python3 servers?
Just don't use the d:l:python3 repos anymore. If you depend on a package, that isn't in d:l:py (nor in d:l:py:{aws,azure,flask,...}, but in d:l:py3, you might want to help by converting it to single spec specification with the help of https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec or just ask.
If you want to limit your systems to python3, don't install python2{,-base} nor any dependencies and add zypper locks. 15.0 and TW tend to prefer python3, but I never tried that, though.
Python2 was almost completely removed for Leap but there were a few things that didn't get ported in time that were important enough to keep shipping it atleast. Python2 likely won't be available in Leap 15.X after it reaches its end of life. Further to what was mentioned, the other major change in d:l:py is that every package that is not included (or on its way to being included) in tumbleweed has been moved out of d:l:py, so on tumbleweed systems and likely leap 15 systems there is basically no longer any benefit to having d:l:py installed. In fact the openSUSE project strongly recommends not having it installed as we don't support it especially on production systems. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On Thu, Jul 5, 2018 at 12:33 AM Simon Lees
On 05/07/18 07:53, Hans-Peter Jansen wrote:
If you want to limit your systems to python3, don't install python2{,-base} nor any dependencies and add zypper locks. 15.0 and TW tend to prefer python3, but I never tried that, though.
Python2 was almost completely removed for Leap but there were a few things that didn't get ported in time that were important enough to keep shipping it atleast. Python2 likely won't be available in Leap 15.X after it reaches its end of life.
Yeah I expected that, with https://pythonclock.org/ and all that…
Further to what was mentioned, the other major change in d:l:py is that every package that is not included (or on its way to being included) in tumbleweed has been moved out of d:l:py, so on tumbleweed systems and likely leap 15 systems there is basically no longer any benefit to having d:l:py installed. In fact the openSUSE project strongly recommends not having it installed as we don't support it especially on production systems.
Do I understand you correctly, that at some point in the future, all the singlespec packages will be included in the base distro? (I guess they might still be built in d:l:py, but _aggregate'd or similar?) That's awesome news! -- Kind regards Christopher 'm4z' Holm / 686f6c6d "We must respect the other fellow's religion, but only in the sense and to the extent that we respect his theory that his wife is beautiful and his children smart." --H. L. Mencken -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On 05/07/18 22:06, 686f6c6d wrote:
On Thu, Jul 5, 2018 at 12:33 AM Simon Lees
wrote: On 05/07/18 07:53, Hans-Peter Jansen wrote:
If you want to limit your systems to python3, don't install python2{,-base} nor any dependencies and add zypper locks. 15.0 and TW tend to prefer python3, but I never tried that, though.
Python2 was almost completely removed for Leap but there were a few things that didn't get ported in time that were important enough to keep shipping it atleast. Python2 likely won't be available in Leap 15.X after it reaches its end of life.
Yeah I expected that, with https://pythonclock.org/ and all that…
Further to what was mentioned, the other major change in d:l:py is that every package that is not included (or on its way to being included) in tumbleweed has been moved out of d:l:py, so on tumbleweed systems and likely leap 15 systems there is basically no longer any benefit to having d:l:py installed. In fact the openSUSE project strongly recommends not having it installed as we don't support it especially on production systems.
Do I understand you correctly, that at some point in the future, all the singlespec packages will be included in the base distro? (I guess they might still be built in d:l:py, but _aggregate'd or similar?) That's awesome news!
All the packages in d:l:py should end up in the base distro, but in the case of Leap for example being a "stable" distro the python version and most if not all of the d:l:py packages will probably stay at the same version, some might get updated again for point releases ie 15.1 but some may not, however if bugs are found then reported as being fixed in a newer version either the fix will be brought via a maintenance update to Leap (in the exact same way the rest of the distro works). -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On Thu, Jul 5, 2018 at 12:23 AM Hans-Peter Jansen
On Mittwoch, 4. Juli 2018 19:11:49 686f6c6d wrote:
Greetings,
I was under the naive impression that after the py2/3 split, the packages from the devel:languages:python OBS project are released into the devel:languages:python/python3 repos based on their name (everything /^python3-/ goes into d:l:python3); in other words, I wanted to configure my python3-only servers to only use the python3 repo. But apparently this simple logic is not what is happening, there are many python3-* packages in d:l:python.
In fact, it's the other way around... There's a single repo d:l:python, that serves our needs (and a couple of subrepos that tries to reduce the sheer number of packages in a single repo).
Most packages in d:l:py generate python2- and python3- packages from a single spec nowadays. This paves the way to generate pypy2 and/or pypy3 versions as well without touching the spec, all controlled with rpm macros.
Ah, thanks for the explanation! So all packages are built in d:l:py, the "default" singlespec packages go to d:l:py whether they're 2 or 3, and only the not-yet-singlespec'ed py3-only packages go to d:l:py3?
Can someone explain what is really happening? Is this a temporary state or will I have to keep enabling both python{,3} repos on my python3 servers?
Just don't use the d:l:python3 repos anymore. If you depend on a package, that isn't in d:l:py (nor in d:l:py:{aws,azure,flask,...}, but in d:l:py3, you might want to help by converting it to single spec specification with the help of https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec or just ask.
Thanks, will try (with a quick search, I only found python3-argparse and python3-django-mptt from the py3 repo)…
If you want to limit your systems to python3, don't install python2{,-base} nor any dependencies and add zypper locks. 15.0 and TW tend to prefer python3, but I never tried that, though.
Thanks, we basically do that already, but this information will certainly be helpful to others. -- Kind regards Christopher 'm4z' Holm / 686f6c6d "We must respect the other fellow's religion, but only in the sense and to the extent that we respect his theory that his wife is beautiful and his children smart." --H. L. Mencken -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Donnerstag, 5. Juli 2018 14:30:09 686f6c6d wrote:
On Thu, Jul 5, 2018 at 12:23 AM Hans-Peter Jansen
wrote: Most packages in d:l:py generate python2- and python3- packages from a single spec nowadays. This paves the way to generate pypy2 and/or pypy3 versions as well without touching the spec, all controlled with rpm macros.
Ah, thanks for the explanation! So all packages are built in d:l:py, the "default" singlespec packages go to d:l:py whether they're 2 or 3, and only the not-yet-singlespec'ed py3-only packages go to d:l:py3?
I just noticed, d:l:py3 is history already... Even py3 only packages reside in d:l:py now, python2 exclusion is done with macros. d:l:py:misc is the place for not-yet-singlespec'ed python packages.
you might want to help by converting it to single spec specification with the help of https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec or just ask. Thanks, will try (with a quick search, I only found python3-argparse and python3-django-mptt from the py3 repo)…
Well, there's no python-argparse package anymore, because it's part of Python2 and Python3 stdlib. See https://pypi.org/project/argparse/ Hence, you're done already with the former ;-). The latter is eagerly awaiting some loving touches: https://build.opensuse.org/package/show/devel:languages:python:misc/python-d... Note, it is converted already, don't know what's missing. @Jan Matejek: since some time, python3 assimilates certain packages in the standard lib. Since almost all of them exist as separate packages, wouldn't it be sensible to provide those properly in python3-base (e.g. python3-argparse). Cheers, Pete -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On 05/07/18 23:08, Hans-Peter Jansen wrote:
On Donnerstag, 5. Juli 2018 14:30:09 686f6c6d wrote:
On Thu, Jul 5, 2018 at 12:23 AM Hans-Peter Jansen
wrote: Most packages in d:l:py generate python2- and python3- packages from a single spec nowadays. This paves the way to generate pypy2 and/or pypy3 versions as well without touching the spec, all controlled with rpm macros.
Ah, thanks for the explanation! So all packages are built in d:l:py, the "default" singlespec packages go to d:l:py whether they're 2 or 3, and only the not-yet-singlespec'ed py3-only packages go to d:l:py3?
I just noticed, d:l:py3 is history already...
Even py3 only packages reside in d:l:py now, python2 exclusion is done with macros.
Yes that is correct, python2 has basically already been dropped from most places in the base system, the aim is to have openSUSE not using python2 anywhere internally.
d:l:py:misc is the place for not-yet-singlespec'ed python packages.
This is close but not quite true, d:l:py:misc is basically the grave yard for all the packages that were in d:l:py but were never submitted into tumbleweed, they are left here just in case someone else comes along and wants to do the work to get them building and acceptable to submit into tumbleweed, then they can be moved back to d:l:py, there is no point in adding d:l:py:misc to your system though because it intentionally has either building or publishing disabled. To make the packages there acceptable for tumbleweed you don't necessarily need to single spec them, python3 only versions will likely also be accepted. In limited circumstances we may also accept a python2 only version, but only if upstream never had any intention of making it work with python3 and it will disappear when python2 does.
you might want to help by converting it to single spec specification with the help of https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec or just ask. Thanks, will try (with a quick search, I only found python3-argparse and python3-django-mptt from the py3 repo)…
Well, there's no python-argparse package anymore, because it's part of Python2 and Python3 stdlib. See https://pypi.org/project/argparse/
Hence, you're done already with the former ;-).
The latter is eagerly awaiting some loving touches: https://build.opensuse.org/package/show/devel:languages:python:misc/python-d...
Note, it is converted already, don't know what's missing.
@Jan Matejek: since some time, python3 assimilates certain packages in the standard lib. Since almost all of them exist as separate packages, wouldn't it be sensible to provide those properly in python3-base (e.g. python3-argparse).
Jan left SUSE several months back and his replacement is finding his feet. It might make sense to have provides / obsoletes statements in the python spec, which we do have for some things like python3-asyncio and python3-xml, but if the package was never in tumbleweed its quite likely that no one bothered. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
participants (3)
-
686f6c6d
-
Hans-Peter Jansen
-
Simon Lees