New subject: [opensuse-packaging] Re: [opensuse-factory] plan for 12.3 - multiple Python runtimes

10 Sep 2012

      fellow openSUSErs,

with the advent of Python 3 and still almost unnoticeable decline of Python 2, i 
have come to a realization that the way we separate them into "python" and 
"python3" is less than ideal. The granularity of this can be often too coarse 
(you can't run two apps requiring two different python2 versions, unless they 
are specifically packaged and bring their own python runtime package), and too 
fine at the same time (there are already packages that don't actually care 
whether they run in python 2 or 3)

In the spirit of fixing what ain't broken, I humbly propose a grand plan: let's 
turn "python" into "python2.7", "python3" into "python3.3", and tinker with 
Provides until it all works just fine.
Then, as a last step, symlink /usr/bin/python into /etc/alternatives and let the 
users pick whichever runtime they like better.

This would probably go better if broken into several phases. My current idea is 
this:

Phase 1:
- rename "python" to "python2.7", providing "python = 2.7"
- rename "python3" to "python3.3", providing "python3 = 3.3"
- install the update-alternatives symlink

Impact:
- Unsure of what would happen on package upgrade. Would we need 
Obsoletes/Provides combo for this? Perhaps we'd have to create the metapackages 
outlined in the following phases at this point already.

Phase 2:
- unify RPM macros - python3 will have %py_(something) macros as well as 
%py3_(something) macros, the %py3_ versions being aliases to the %py_ versions
- make python2.7 provide "python2 = 2.7" instead of "python = 2.7"
- make a "python" meta-package requiring, at this time, python2
- each pythonX.Y package now only installs /usr/bin/pythonX.Y binary,
python2 and python3 are alternatives for python2.Y and python3.Y respectively, 
and python is an alternative for either python2, python3, or a specific version.
I don't know update-alternatives too well - is this even viable?

Impact:
- this should mark a clear upgrade path for each of the python variants and for 
the systemwide "default python" as well
- at this phase, i would like to make it possible to build a package for any 
python (2 or 3 or 50) from the same specfile. i'm not sure that these outlined 
steps are enough, though. Originally I thought that every python would provide 
"python = X.Y" and the packages could select which "Requires: python" to pick 
through package or project meta. But that would result in a horrible breakage, 
and we need "python" as a separate package for upgrading in any case. We could 
use "python(abi)" symbol for this, or do the same just with "python-devel"? Any 
other options?

Phase 3A:
- when python 3.4 comes out, introduce a new package "python3.4", new 
meta-package "python3" requiring "python3.4", and corresponding alternatives

Impact:
- This is the other difficult one. What about lifetime of the pythonX.Y 
packages? What happens with old systems on perpetual-upgrade? Such as tumbleweed 
- will the old python runtimes rot there forever-and-ever, or can we force 
uninstall at some point? Perhaps "Obsoletes: python3 < 3.x" in the "python3" 
metapackage? Which versions do we support? The newest and the most popular, 
probably?
I know we do something similar with gcc, how does it work there?

Phase 3B:
- steal and improve Debian's python-central (or however they call it these days)
- designate /usr/lib/python-central or something similar for packages that don't 
want to depend on a specific python runtime version
- introduce %post/%postun scriptlet macros for packages that would compile 
bytecode in /usr/lib/python-central for all known python runtimes on the system, 
and for pythons so that they compile every package for their versions. This is 
now actually fully supported by upstream, we can have as many different versions 
of bytecode for the -same- .py file as we want.

Impact:
- package can specify "Requires: python" and Just Work(tm) from now until 
eternity, or until it runs into a deprecated API. Or specify a range of pythons 
it wants to support - by the time we get here, I'm sure RPM will allow that ;e)
With a limited ABI in python3, this can even work for arch-based packages across 
python minor versions.

Phase 4:
- when we decide to switch to python3 system-wide, make "python" require 
"python3" instead of "python2", change default alternative, and be done with it

Impact:
- breakage of packages still incompatible at that time. that will help us see 
what needs fixing and what needs to explicitly specify that it wants python2.

Somewhere in this is lurking a possibility of supporting different python 
runtimes such as Jython, Cython, PyPy and whatever else crops up. That's neat, too.

Thanks for your attention. Comments, questions?

regards
m.
-- 
To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org
To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

[opensuse-factory] plan for 12.3 - multiple Python runtimes

Jan Matejek

Michal Vyskocil

Jan Matejek

Claudio Freire

Vincent Untz

Claudio Freire

Jan Matejek

Claudio Freire

Jan Matějek

Claudio Freire

Jan Matějek

Andreas Jaeger

Karl Eichwalder

Alin M Elena

Karl Eichwalder

Jan Matejek

Karl Eichwalder

Vincent Untz

Karl Eichwalder

Togan Muftuoglu

tags

participants (10)