[opensuse-factory] Switching default python to Python 3
Fellow openSUSists, this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed. First of all, big thanks to everyone who is converting (or has converted) their packages to singlespec. That alone makes the Python 3 switch much easier than it would be otherwise. Then, details. When: * in the coming weeks. I'd like to do some tests and gather feedback before actually moving forward. A heads-up e-mail will come before anything is actually headed to TW. Why: * Python 2 is very near its end of life [1], the vast majority of our stuff already works in python3. We want to have python3 as default in SLE 15, so now's a very good time to start. What will happen: * `python` package will be renamed to `python2` and will obsolete/provide `python <= 2.7.13` * `python3` already provides `python = 3.6` and will also obsolete `python < %version` This should mean that if you don't require python2 on your system, it will be uninstalled, otherwise it will be replaced by python2. * /usr/bin/python will be switched to point to /usr/bin/python3. Other unversioned python binaries will also be switched. * python packages in patterns will be replaced by the corresponding python3 ones What else will happen: * Stuff will break! The breakage will be contained to a testing project at first. * python packages that are not yet converted to singlespec will switch over to python3. Some of them will magically continue working, others will break. * For those we will also start getting file conflicts with python3 versions. A good opportunity to revisit and either completely drop the python2 version, convert to singlespec, or to explicitly-python2 package. * python packages that _are_ converted to singlespec will work just fine * as will explicitly-python3 packages from d:l:py3 * packages that _require_ python stuff will also magically get python3. They might very well also break * through this, we will learn where we still depend on Python 2, and hopefully fix it * note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version regards m. [1] https://pythonclock.org/
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
What will happen: * python packages in patterns will be replaced by the corresponding python3 one * python packages that are not yet converted to singlespec will switch over to python3. Some of them will magically continue working, others will break Is there a plan for python-* packages and how they are going to relate to the python3-* set of packages? By which I mean, if I want to install
Awesome work, super excited about finally having Python 3 as the default! python-numpy will that install the py3 or py2 version? Or is that all handled by singlespec? As an aside, when is the plan to merge the two wiki pages[1,2] for Python packaging? I ask because I remember trying to package Python a while ago and I don't remember the tiny mention of single-spec in the main page.
* note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
Quick question, is "#!/usr/bin/python3" better or worse than "#!/usr/bin/env python3"? I've always used the latter in my own projects because it avoids weird distros breaking scripts. [1]: https://en.opensuse.org/openSUSE:Packaging_Python [2]: https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec -- Aleksa Sarai Software Engineer (Containers) SUSE Linux GmbH https://www.cyphar.com/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 26.6.2017 15:44, Aleksa Sarai wrote:
Is there a plan for python-* packages and how they are going to relate to the python3-* set of packages? By which I mean, if I want to install python-numpy will that install the py3 or py2 version? Or is that all handled by singlespec?
singlespec generates python2-numpy and python3-numpy, so the version is explicit. Today you shouldn't have "python-numpy" on your system. As part of the switch, the "python-numpy" symbol will be provided by "python3-numpy" instead of python2. If you now have "python2-numpy", nothing special will happen to replace that with "python3-numpy". (at least no plan to do so at the moment) But a package requiring "python-numpy" will pull in "python3-numpy" instead.
As an aside, when is the plan to merge the two wiki pages[1,2] for Python packaging? I ask because I remember trying to package Python a while ago and I don't remember the tiny mention of single-spec in the main page.
there is a plan, yes :) but no roadmap. Basically when I have the time, or when someone else does it.
* note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
Quick question, is "#!/usr/bin/python3" better or worse than "#!/usr/bin/env python3"? I've always used the latter in my own projects because it avoids weird distros breaking scripts.
For a script that is part of Tumbleweed, the second is wrong and the first is correct. This is because "/usr/bin/env" will find a python that a user has installed into their own path, and that might be a strange unsupported version which breaks your app ;) The latter is (only) useful if you're distributing the script between different distros.
[1]: https://en.opensuse.org/openSUSE:Packaging_Python [2]: https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
What will happen: * python packages in patterns will be replaced by the corresponding python3 one * python packages that are not yet converted to singlespec will switch over to python3. Some of them will magically continue working, others will break Is there a plan for python-* packages and how they are going to relate to the python3-* set of packages? By which I mean, if I want to install
Awesome work, super excited about finally having Python 3 as the default! python-numpy will that install the py3 or py2 version? Or is that all handled by singlespec? As an aside, when is the plan to merge the two wiki pages[1,2] for Python packaging? I ask because I remember trying to package a Python library a while ago and I don't remember the tiny mention of single-spec in the main page.
* note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
Quick question, is "#!/usr/bin/python3" better or worse than "#!/usr/bin/env python3"? I've always used the latter in my own projects because it avoids weird distros breaking scripts. [1]: https://en.opensuse.org/openSUSE:Packaging_Python [2]: https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec -- Aleksa Sarai Software Engineer (Containers) SUSE Linux GmbH https://www.cyphar.com/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hello, Am Montag, 26. Juni 2017, 15:26:12 CEST schrieb jan matejek: ...
* /usr/bin/python will be switched to point to /usr/bin/python3. Other unversioned python binaries will also be switched. ... * Stuff will break! The breakage will be contained to a testing project at first.
I really like the switch to py3, but I don't like the "Stuff will break" part ;-) Especially switching the /usr/bin/python symlink to py3 sounds like it will break *lots of* things because everybody expects it to point it to py2.
* note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
Is there a way to check all packages for unversioned python shebangs? (Install and grep them all, add a rpmlint check, whatever?) After collecting this information, contact the package maintainers and ask them to fix the shebang (ideally by putting py3 there ;-) That would be much better (and less painful) than finding all the breakage after doing the switch ;-) Regards, Christian Boltz -- Microsoft is not the Answer - Microsoft is the Question, and the Answer is no ! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mon, Jun 26, 2017 at 2:38 PM, Christian Boltz <opensuse@cboltz.de> wrote:
Is there a way to check all packages for unversioned python shebangs? (Install and grep them all, add a rpmlint check, whatever?)
After collecting this information, contact the package maintainers and ask them to fix the shebang (ideally by putting py3 there ;-)
That would be much better (and less painful) than finding all the breakage after doing the switch ;-)
+1 Even if I wanted to be proactive I suspect I support 50 or more packages with python involvement. Some form of automated tool to make sure all is well is pretty important. Thanks Greg -- Greg Freemyer Advances are made by answering questions. Discoveries are made by questioning answers. — Bernard Haisch -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 06/26/2017 10:01 PM, Greg Freemyer wrote:
On Mon, Jun 26, 2017 at 2:38 PM, Christian Boltz <opensuse@cboltz.de> wrote:
Is there a way to check all packages for unversioned python shebangs? (Install and grep them all, add a rpmlint check, whatever?)
After collecting this information, contact the package maintainers and ask them to fix the shebang (ideally by putting py3 there ;-)
That would be much better (and less painful) than finding all the breakage after doing the switch ;-)
+1
Even if I wanted to be proactive I suspect I support 50 or more packages with python involvement. Some form of automated tool to make sure all is well is pretty important.
Also, please think about the users. Users who use python3 already now are using "python3" explicitly. Python2 users (I guess still majority in some fields) may just use "python" and we will break them. So from the user's view, renaming things will not improve anything but may break any user script with non-versioned shebang. FYI I have access to a very large SLE cluster. There are many users running "python" jobs and they expect python2. Let them survive the update! cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am 26.06.2017 um 22:34 schrieb Rüdiger Meier:
FYI I have access to a very large SLE cluster. There are many users running "python" jobs and they expect python2. Let them survive the update!
Oh! Of course Rüdiger! We really need to have your users survive, we shouldn't do anything that kills them or even endangers their lives. We definitely are in agreement here! This is a RED LINE! But FYI: this is just a switch in python defaults for Tumbleweed, no one will be hurt. Greetings, Stephan -- Ma muaß weiterkämpfen, kämpfen bis zum Umfalln, a wenn die ganze Welt an Arsch offen hat, oder grad deswegn. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, 2017-06-27 at 08:15 +0200, Stephan Kulow wrote:
Am 26.06.2017 um 22:34 schrieb Rüdiger Meier:
FYI I have access to a very large SLE cluster. There are many users running "python" jobs and they expect python2. Let them survive the update!
Oh! Of course Rüdiger! We really need to have your users survive, we shouldn't do anything that kills them or even endangers their lives. We definitely are in agreement here! This is a RED LINE!
But FYI: this is just a switch in python defaults for Tumbleweed, no one will be hurt.
I guess that this would be good selling point to promote openSUSE - something like "openSUSE - distribution that does not kill it's users" or if we want be more aggressive we could say "Unlike XY, we don't kill our users" M -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wednesday 28 June 2017, Martin Pluskal wrote:
On Tue, 2017-06-27 at 08:15 +0200, Stephan Kulow wrote:
Am 26.06.2017 um 22:34 schrieb Rüdiger Meier:
FYI I have access to a very large SLE cluster. There are many users running "python" jobs and they expect python2. Let them survive the update!
Oh! Of course Rüdiger! We really need to have your users survive, we shouldn't do anything that kills them or even endangers their lives. We definitely are in agreement here! This is a RED LINE!
But FYI: this is just a switch in python defaults for Tumbleweed, no one will be hurt.
I guess that this would be good selling point to promote openSUSE - something like "openSUSE - distribution that does not kill it's users" or if we want be more aggressive we could say "Unlike XY, we don't kill our users"
Yeah, this seems appropriate: "We don't care what you need, we just laugh about you, but we don't kill you." At least you have this selling point, better than nothing. cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mon, 2017-06-26 at 15:26 +0200, jan matejek wrote:
Fellow openSUSists,
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
I'd like to point to https://www.python.org/dev/peps/pep-0394/ again... """The main barrier to a distribution switching the python command from python2 to python3 isn't breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren't yet familiar with the backwards incompatible changes in Python 3.""" is openSUSE *REALLY* the distro that is willing to break user scripts?
What else will happen: * Stuff will break! The breakage will be contained to a testing project at first.
That will be an interesting project - and breakages will be many, I'm sure
* note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
As this has so far not been any rule to enforce, I can ensure you a ton of fallouts. zypper se --requires -x /usr/bin/python => this lists 635 items (of which 211 are source packages). zypper se --requires -x /usr/bin/env => 747 hits; no analysis done what those will run behind; could be anything (0 of them are source packages) Also, if those shebangs are so wrong we definitively need a way to support our packagers to change it in the shipped .py files - without having to go out and write 1000 patches for this now. The bare minimum is a documentation with recommendations on how to easily find, then change the files. Followed with rpmlint warnings that can be searched for (once rpmlint warns, the warnings will show up on rpmlint.opensuse.org) Cheers, Dominique
On 06/27/2017 09:32 AM, Dominique Leuenberger / DimStar wrote:
On Mon, 2017-06-26 at 15:26 +0200, jan matejek wrote:
Fellow openSUSists,
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
I'd like to point to https://www.python.org/dev/peps/pep-0394/ again...
"""The main barrier to a distribution switching the python command from python2 to python3 isn't breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren't yet familiar with the backwards incompatible changes in Python 3."""
is openSUSE *REALLY* the distro that is willing to break user scripts?
Can we make /usr/bin/python smart to know if it was started as part of a shebang and output a warning? Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Can we make /usr/bin/python smart to know if it was started as part of a shebang and output a warning?
What is to be preferred against shebang? For me python2 on TW does not work any more for some time. h5py and vtk are appear to me to be not supported any more. I modified our scripts to run with python3 and python2 (for my colleagues, some on quite old distros with python3 not installed). So I cannot explicitly call pyhton2 or python3 in shebang, which makes no sense as the scripts run on both. The dirty fix for me is, to shebang #!/usr/bin/env python and have my ~/bin/python link to /usr/bin/python3 So if you want to "enforce" the explicit selection of python2 and python3 I vote against it. BTW, vtk cannot exist concurrently for python2 and python3 AFAIK. Fabian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 06/27/2017 10:00 AM, Stephan Kulow wrote:
On 06/27/2017 09:32 AM, Dominique Leuenberger / DimStar wrote:
On Mon, 2017-06-26 at 15:26 +0200, jan matejek wrote:
Fellow openSUSists,
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
I'd like to point to https://www.python.org/dev/peps/pep-0394/ again...
"""The main barrier to a distribution switching the python command from python2 to python3 isn't breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren't yet familiar with the backwards incompatible changes in Python 3."""
is openSUSE *REALLY* the distro that is willing to break user scripts?
Can we make /usr/bin/python smart to know if it was started as part of a shebang and output a warning?
Such a warning could be good. But it doesn't matter whether it's a shebang or not. I know many existing cron jobs or other job scripts which are shell scripts using "python xyz.py". Also many tutorials and examples on the web are still teaching python. cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Stephan Kulow writes:
Can we make /usr/bin/python smart to know if it was started as part of a shebang and output a warning?
That warning would be quite annoying to people who actually make an effort to write scripts that work with both Python 2 and 3. Best regards, Peter -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Dominique Leuenberger / DimStar wrote:
I'd like to point to https://www.python.org/dev/peps/pep-0394/ again...
"""The main barrier to a distribution switching the python command from python2 to python3 isn't breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren't yet familiar with the backwards incompatible changes in Python 3."""
is openSUSE *REALLY* the distro that is willing to break user scripts?
I'd vote against this breakage. Especially because it does not fix a real problem. And surely nobody is willing to maintain thousands of shee-bang patches. Ciao, Michael.
On 06/27/2017 05:06 AM, Michael Ströder wrote:
Dominique Leuenberger / DimStar wrote:
I'd like to point to https://www.python.org/dev/peps/pep-0394/ again...
"""The main barrier to a distribution switching the python command from python2 to python3 isn't breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren't yet familiar with the backwards incompatible changes in Python 3."""
is openSUSE *REALLY* the distro that is willing to break user scripts?
I'd vote against this breakage. Especially because it does not fix a real problem. And surely nobody is willing to maintain thousands of shee-bang patches.
Sorry, but "voting against" is not sufficient, you'll have to offer a viable alternative solution, IMHO. Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic. Thus at some point the switch needs to be made, which means things will be broken. I hate when things break as much as the next person. But in the end that is the decision the Python community made. Therefore we have to deal with it sooner or later. Now lets argue a bit about why it may be a good idea to do in TW what SLE is planning to do. - Lets say TW stays with Python 2 as the default interpreter, i.e. for now we don't break anything. But SLE switches to Python 3 as the default interpreter. Then when openSUSE Leap Next is released it will have the default interpreter be Python 3. The result is that the distribution that is being advertised as close to the cutting edge is using an older interpreter than the distribution that is supposed to be somewhat behind in the name of stability. Certainly that will create some kind of perception problem. - It is inevitable that TW will need to switch the default interpreter and thus break things. Therefore we might as well do it now. - Last but not least there is the human factor. We all know how things go, even if we make a big announcement and say "in one year from today we will switch to Python 3" 98% of the scripts that will break because of this will still be in the same state a year from now because people will not make any changes until things are broken. Am I happy that things will be broken, NO. But given that this is inevitable we might as well get over it, there is no time like the present. Later, Robert -- Robert Schweikert MAY THE SOURCE BE WITH YOU Distinguished Architect LINUX Team Lead Public Cloud rjschwei@suse.com IRC: robjo -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, Jun 28, 2017 at 11:55:30AM -0400, Robert Schweikert wrote:
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic.
I don't see what the SLE life cycle has to do with this. You need to backport fixes to outdated python code anyway after a couple of years.
Thus at some point the switch needs to be made, which means things will be broken. I hate when things break as much as the next person. But in the end that is the decision the Python community made.
AFAIK the Python community currently wants /usr/bin/python to be python2.
Therefore we have to deal with it sooner or later.
Now lets argue a bit about why it may be a good idea to do in TW what SLE is planning to do. - Lets say TW stays with Python 2 as the default interpreter, i.e. for now we don't break anything. But SLE switches to Python 3 as the default interpreter.
What SLE? SLE-15? SLE-16? Is that already decided?
Then when openSUSE Leap Next is released it will have the default interpreter be Python 3. The result is that the distribution that is being advertised as close to the cutting edge is using an older interpreter than the distribution that is supposed to be somewhat behind in the name of stability. Certainly that will create some kind of perception problem.
Leap should match SLE, no questions.
- It is inevitable that TW will need to switch the default interpreter and thus break things. Therefore we might as well do it now.
That's a pretty weak argument.
- Last but not least there is the human factor. We all know how things go, even if we make a big announcement and say "in one year from today we will switch to Python 3" 98% of the scripts that will break because of this will still be in the same state a year from now because people will not make any changes until things are broken.
It really depends on what's planned for SLE-15. If project management wants /usr/bin/pyhton to be python3 for SLE-15, I'm all for doing the switch right away. If it is just planned for SLE-16, I', for waiting a bit so that 3rd party applications don't break. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 06/28/2017 12:05 PM, Michael Schroeder wrote:
On Wed, Jun 28, 2017 at 11:55:30AM -0400, Robert Schweikert wrote:
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic.
I don't see what the SLE life cycle has to do with this.
Well, in 2 years 9 months and 14 days [1] Python 2 will no longer be maintained upstream. This would imply that if SLE sticks with Python 2 as the default interpreter SUSE would be maintaining, on their own, Python 2 for roughly 9 years. I'd say that is unrealistic and thus it is a pretty good bet that SLE 15 will have Python 3 as the default interpreter. I already eluded to what that implies w.r.t the relation between TW and Leap.
You need to backport fixes to outdated python code anyway after a couple of years.
Thus at some point the switch needs to be made, which means things will be broken. I hate when things break as much as the next person. But in the end that is the decision the Python community made.
AFAIK the Python community currently wants /usr/bin/python to be python2.
Ah yes of course, the PEP. Well what if that pep changes in 2 years 9 months and 13 days, can we then switch everything overnight? And probably most importantly will all those scripts that would be broken if we switch now be fixed? I'd say the answer to both questions is NO. So, IMHO it is prudent to be pragmatic and give people a reasonably long runway. Switch the default while we still provide the safety blanket of /usr/bin/python2. If we switch when there is no safety blanket it is worse.
Therefore we have to deal with it sooner or later.
Now lets argue a bit about why it may be a good idea to do in TW what SLE is planning to do. - Lets say TW stays with Python 2 as the default interpreter, i.e. for now we don't break anything. But SLE switches to Python 3 as the default interpreter.
What SLE? SLE-15? SLE-16? Is that already decided?
See above, SLE-15 yes. Given the upstream timeline [1] how would one realistically expect SUSE as a business to support Python 2 on its own with no help from upstream for roughly 9 years? Later, Robert [1] https://pythonclock.org/ -- Robert Schweikert MAY THE SOURCE BE WITH YOU Distinguished Architect LINUX Team Lead Public Cloud rjschwei@suse.com IRC: robjo -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wednesday 28 June 2017, Robert Schweikert wrote:
On 06/28/2017 12:05 PM, Michael Schroeder wrote:
On Wed, Jun 28, 2017 at 11:55:30AM -0400, Robert Schweikert wrote:
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic.
I don't see what the SLE life cycle has to do with this.
Well, in 2 years 9 months and 14 days [1] Python 2 will no longer be maintained upstream.
This is much longer than most other software shipped with SLE will be maintained by upstream ... BTW I don't believe that python2 will disappear in 3 years. We will see.
This would imply that if SLE sticks with Python 2 as the default interpreter SUSE would be maintaining, on their own, Python 2 for roughly 9 years. I'd say that is unrealistic and thus it is a pretty good bet that SLE 15 will have Python 3 as the default interpreter.
It would make more sense to not have /usr/bin/python at all if python2 is not installed. Pointing it to python3 makes no sense. The OP already said that using non-versioned shebang *is* broken. But there are no python3 scripts with non-versioned shebang. So we would only break py2 users for no reason. That's sooo obvious. I'm glad that somebody in this thread found PEP 394 to avoid a wrong decision for SLE/openSUSE.
I already eluded to what that implies w.r.t the relation between TW and Leap.
You need to backport fixes to outdated python code anyway after a couple of years.
Thus at some point the switch needs to be made, which means things will be broken. I hate when things break as much as the next person. But in the end that is the decision the Python community made.
AFAIK the Python community currently wants /usr/bin/python to be python2.
Ah yes of course, the PEP. Well what if that pep changes in 2 years 9 months and 13 days, can we then switch everything overnight? And probably most importantly will all those scripts that would be broken if we switch now be fixed?
I'd say the answer to both questions is NO. So, IMHO it is prudent to be pragmatic and give people a reasonably long runway. Switch the default while we still provide the safety blanket of /usr/bin/python2. If we switch when there is no safety blanket it is worse.
Therefore we have to deal with it sooner or later.
Now lets argue a bit about why it may be a good idea to do in TW what SLE is planning to do. - Lets say TW stays with Python 2 as the default interpreter, i.e. for now we don't break anything. But SLE switches to Python 3 as the default interpreter.
What SLE? SLE-15? SLE-16? Is that already decided?
See above, SLE-15 yes. Given the upstream timeline [1] how would one realistically expect SUSE as a business to support Python 2 on its own with no help from upstream for roughly 9 years?
Later, Robert
[1] https://pythonclock.org/ -- Robert Schweikert MAY THE SOURCE BE WITH YOU Distinguished Architect LINUX Team Lead Public Cloud rjschwei@suse.com IRC: robjo -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Ruediger Meier wrote:
It would make more sense to not have /usr/bin/python at all if python2 is not installed. Pointing it to python3 makes no sense.
+1
The OP already said that using non-versioned shebang *is* broken. But there are no python3 scripts with non-versioned shebang.
Full ack. Ciao, Michael.
On 29/06/17 05:15, Ruediger Meier wrote:
On Wednesday 28 June 2017, Robert Schweikert wrote:
On 06/28/2017 12:05 PM, Michael Schroeder wrote:
On Wed, Jun 28, 2017 at 11:55:30AM -0400, Robert Schweikert wrote:
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic.
I don't see what the SLE life cycle has to do with this.
Well, in 2 years 9 months and 14 days [1] Python 2 will no longer be maintained upstream.
This is much longer than most other software shipped with SLE will be maintained by upstream ...
BTW I don't believe that python2 will disappear in 3 years. We will see.
This would imply that if SLE sticks with Python 2 as the default interpreter SUSE would be maintaining, on their own, Python 2 for roughly 9 years. I'd say that is unrealistic and thus it is a pretty good bet that SLE 15 will have Python 3 as the default interpreter.
It would make more sense to not have /usr/bin/python at all if python2 is not installed. Pointing it to python3 makes no sense. The OP already said that using non-versioned shebang *is* broken. But there are no python3 scripts with non-versioned shebang. So we would only break py2 users for no reason. That's sooo obvious. I'm glad that somebody in this thread found PEP 394 to avoid a wrong decision for SLE/openSUSE.
That is not entirely correct there are a number of scripts that support both versions i.e. don't care which python version they use that are using /usr/bin/python in the shebang, so shipping no /usr/bin/python when python3 is installed will break them. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On Thursday 29 June 2017, Simon Lees wrote:
On 29/06/17 05:15, Ruediger Meier wrote:
On Wednesday 28 June 2017, Robert Schweikert wrote:
On 06/28/2017 12:05 PM, Michael Schroeder wrote:
On Wed, Jun 28, 2017 at 11:55:30AM -0400, Robert Schweikert wrote:
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic.
I don't see what the SLE life cycle has to do with this.
Well, in 2 years 9 months and 14 days [1] Python 2 will no longer be maintained upstream.
This is much longer than most other software shipped with SLE will be maintained by upstream ...
BTW I don't believe that python2 will disappear in 3 years. We will see.
This would imply that if SLE sticks with Python 2 as the default interpreter SUSE would be maintaining, on their own, Python 2 for roughly 9 years. I'd say that is unrealistic and thus it is a pretty good bet that SLE 15 will have Python 3 as the default interpreter.
It would make more sense to not have /usr/bin/python at all if python2 is not installed. Pointing it to python3 makes no sense. The OP already said that using non-versioned shebang *is* broken. But there are no python3 scripts with non-versioned shebang. So we would only break py2 users for no reason. That's sooo obvious. I'm glad that somebody in this thread found PEP 394 to avoid a wrong decision for SLE/openSUSE.
That is not entirely correct there are a number of scripts that support both versions i.e. don't care which python version they use that are using /usr/bin/python in the shebang, so shipping no /usr/bin/python when python3 is installed will break them.
That's a normal thing, missing a command. So what? Install /usr/bin/python(2) and the script will work. Nobody would rename perl to php just because a few perl scripts are valid php and perl code. It's nice that you all want to move things to python3. Make sure that your scripts use python3 ... so that they also run without python2 ... It's also fine that we will be able to have a sytem without python2 installed. I like it. But please don't rename python to python3. There is no benefit, that's obvious. cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Robert Schweikert wrote:
Well, in 2 years 9 months and 14 days [1] Python 2 will no longer be maintained upstream.
So what? This only means that people should start porting their code now. And while they're at it they will change the shee-bang to explicitly invoke python3. Everything else you're saying is speculation. Which problem do you want to solve?
Ah yes of course, the PEP. Well what if that pep changes in 2 years 9 months and 13 days, can we then switch everything overnight?
Does it really lead anywhere to guess which change might be made to the PEP? Personally I even wouldn't have expected Python releases 2.7.10 to 2.7.13 partially introducing new features. So how do you know for sure what will happen? If I follow your path of speculations anything can happen in any direction. Ciao, Michael.
On Wed, Jun 28, 2017 at 12:28:38PM -0400, Robert Schweikert wrote:
On 06/28/2017 12:05 PM, Michael Schroeder wrote:
On Wed, Jun 28, 2017 at 11:55:30AM -0400, Robert Schweikert wrote:
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic.
I don't see what the SLE life cycle has to do with this.
Well, in 2 years 9 months and 14 days [1] Python 2 will no longer be maintained upstream.
Well, in 4 years 6 months Python 3.6 will also no longer be maintained upstream. (PEP 494)
This would imply that if SLE sticks with Python 2 as the default interpreter SUSE would be maintaining, on their own, Python 2 for roughly 9 years. I'd say that is unrealistic and thus it is a pretty good bet that SLE 15 will have Python 3 as the default interpreter.
So it is not decided what SLE 15 will do? Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Moin, On Jun 29, 17 14:30:33 +0200, Michael Schroeder wrote:
On Wed, Jun 28, 2017 at 12:28:38PM -0400, Robert Schweikert wrote: [...]
This would imply that if SLE sticks with Python 2 as the default interpreter SUSE would be maintaining, on their own, Python 2 for roughly 9 years. I'd say that is unrealistic and thus it is a pretty good bet that SLE 15 will have Python 3 as the default interpreter.
So it is not decided what SLE 15 will do?
I'd prefer to go with python3. python2 packages will most likely be available for a grace period, to make switching easier. But this requires two things: a) The 'Requires python-foo' found in packages need to be changed to 'python2-foo' resp. 'python3-foo'. b) We need to get the packages having still the 'python-' prefix instead of 'python2-' or 'python3-' adapted soonish. My current assumption regarding the minor version is that we will go with 3.6. ciao, Stefan -- Stefan Behlert, SUSE LINUX Sr. Release Manager Enterprise Server Maxfeldstr. 5, D-90409 Nuernberg, Germany Phone +49-911-74053-173 SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, Jun 28, 2017 at 11:55 AM, Robert Schweikert <rjschwei@suse.com> wrote:
Am I happy that things will be broken, NO. But given that this is inevitable we might as well get over it, there is no time like the present.
I'm not anti changing the default, but the first thing that needs to happen is to establish a openSUSE packaging guideline as to when to use /usr/bin/python, /usr/bin/python2 or /usr/bin/python3. Then, a way to inform rpmlint (I assume) should be established for use when the maintainer claims the package is compatible with both python2 and python3. Then rpmlint (I assume) should be updated to complain about /usr/bin/python uses that have not been tagged as appropriate by the maintainer. And last by not least, a method to scan factory for rpmlint complaints and send off emails to the maintainer should be implemented. For now, I have 50+ packages I maintain that likely use /usr/bin/python and I have no idea if they are python3 compatible. I have a bunch of testing to do. But if they are, I currently have no way to inform the community (or myself) that I've tested them and they are fine as is. Thanks Greg -- Greg Freemyer Advances are made by answering questions. Discoveries are made by questioning answers. — Bernard Haisch -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Robert Schweikert wrote:
On 06/27/2017 05:06 AM, Michael Ströder wrote:
I'd vote against this breakage. Especially because it does not fix a real problem. And surely nobody is willing to maintain thousands of shee-bang patches.
Sorry, but "voting against" is not sufficient, you'll have to offer a viable alternative solution, IMHO.
Well, rather *you* should provide a _strong_ reason why to make this change and break things.
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic. Thus at some point the switch needs to be made, which means things will be broken.
Still your text lacks any real reason for this change. If in some time in the future openSUSE/SLES/whatever does not ship Python 2.x anymore then you can make this change. And of course everybody will agree that a missing Python 2.x will break things. Every developer who wants to prefer Python 3 will use a shee-bang with /usr/bin/python3 anyway.
Now lets argue a bit about why it may be a good idea to do in TW what SLE is planning to do. - Lets say TW stays with Python 2 as the default interpreter, i.e. for now we don't break anything. But SLE switches to Python 3 as the default interpreter. Then when openSUSE Leap Next is released it will have the default interpreter be Python 3. The result is that the distribution that is being advertised as close to the cutting edge is using an older interpreter than the distribution that is supposed to be somewhat behind in the name of stability. Certainly that will create some kind of perception problem.
IMO your simply overrating the term "default interpreter". Ciao, Michael.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Thu, 2017-06-29 at 00:20 +0200, Michael Ströder wrote:
Python 2 is not going to be around forever and for better or worse in some way or another the interests of openSUSE TW and openSUSE Leap are tied to SLE. SLE has a 10 year life cycle and having Python 2 as the default python interpreter in 2028 is not realistic. Thus at some point the switch needs to be made, which means things will be broken.
Still your text lacks any real reason for this change.
If in some time in the future openSUSE/SLES/whatever does not ship Python 2.x anymore then you can make this change.
At that point, making this change would _absolutely_ cause breakage and chaos. By introducing the change now, as a default behavior, anyone has an opportunity to "fall back" to python 2.7 in order to fix things that need fixing. If we wait until 2.x is dead, we've eliminated a great opportunity to preemptively fix problems. And realistically, all we're talking about is a small change in how update-alternatives is managed for `python`.
And of course everybody will agree that a missing Python 2.x will break things.
- -- James Mason Technical Architect, Public Cloud openSUSE Member SUSE jmason@suse.com -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEg/RjZ+RraZBnLRN4GzlRiGxEkCMFAllULlIACgkQGzlRiGxE kCN5GAgAqjG+3C4k/DWz9zvrtBNEwbPWLyjv/eO+UvLJPvIV1Bl3MWRtJpv8CdZN +nm0CaTfN5KIJynYj0ZJ0qDPZzKrBoMILJ0ZhV0SW4Has7Q1nCUdc2PXBFrD3NVF SheNfQn6eY6ZF+99ZJb+iuA5aUbGL3olxVBzZE4z63ykmlCdwWxKtGW11YP7kWoa tjQFSdnviiyNOdo67n0QHnhf/yaPyLBV7jsN9P9WBnz2Zi54+VdL/abMQu5bq0Q0 0iyTSYhpQ8qz48hwcP5YHgWn5KjHhIB8pC1cTqjk/Rc5BEVYrZu31EPB6iLov+62 VBw+imG1TFtQT2/J+5yubK2v4B16MA== =OR7p -----END PGP SIGNATURE-----
James Mason wrote:
On Thu, 2017-06-29 at 00:20 +0200, Michael Ströder wrote: By introducing the change now, as a default behavior, anyone has an opportunity to "fall back" to python 2.7 in order to fix things that need fixing.
Is this realistic?
If we wait until 2.x is dead, we've eliminated a great opportunity to preemptively fix problems.
Who is going to fix issues? Ciao, Michael.
On 27/06/17 17:02, Dominique Leuenberger / DimStar wrote:
On Mon, 2017-06-26 at 15:26 +0200, jan matejek wrote:
Fellow openSUSists,
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
I'd like to point to https://www.python.org/dev/peps/pep-0394/ again...
"""The main barrier to a distribution switching the python command from python2 to python3 isn't breakage within the distribution, but instead breakage of private third party scripts developed by sysadmins and other users. Updating the python command to invoke python3 by default indicates that a distribution is willing to break such scripts with errors that are potentially quite confusing for users that aren't yet familiar with the backwards incompatible changes in Python 3."""
is openSUSE *REALLY* the distro that is willing to break user scripts?
This is probably the wrong question to ask, A better one probably is anyone willing to maintain python2 in openSUSE post 2020 when it goes end of life? This gives us 2.5 years to try and make the migration as smooth as possible so starting now is probably a decent idea. Users have already had 9 years to migrate I suspect if they haven't started yet there probably not going to bother until stuff starts breaking anyway, unfortunately we can't do much about users but yeah we should try and get our stuff in as good a shape as possible before doing the switch. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On 06/28/17 02:00, Simon Lees wrote:
Users have already had 9 years to migrate I suspect if they haven't started yet there probably not going to bother until stuff starts breaking anyway, unfortunately we can't do much about users but yeah we should try and get our stuff in as good a shape as possible before doing the switch.
And it goes really smooth with 2to3. The only issue I had was that there is no replacement for libxml2 and I had to switch to lxml which works for python2 and python3 I might be my fault however, but I had issues in finding 2to3 and in the end installed it manually. I did not find it installed and could not figure which package contains it. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 2017-06-28 08:15, Fabian Wein wrote:
I might be my fault however, but I had issues in finding 2to3 and in the end installed it manually. I did not find it installed and could not figure which package contains it.
https://maintainer.zq1.de/?pkg=2to3 says it is in python3-devel -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, Jun 28, 2017 at 8:34 AM, Bernhard M. Wiedemann <bernhardout@lsmod.de> wrote:
On 2017-06-28 08:15, Fabian Wein wrote:
I might be my fault however, but I had issues in finding 2to3 and in the end installed it manually. I did not find it installed and could not figure which package contains it.
https://maintainer.zq1.de/?pkg=2to3 says it is in python3-devel
Can it need to be isolated in a sub-project and added to an appropriate python package/pattern as a recommends? I have no idea how many of the python packages/sub-packages have been given python3 love by upstream. Greg -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 06/26/2017, 03:26 PM, jan matejek wrote:
* note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
Ugh, this will be tough. An average system: # grep '#!.*python$' /usr/bin/*|wc 107 182 4485 # grep '#!.*python[23]' /usr/bin/*|wc 51 52 1936 You can start with fixing llvm which: * has "#!/usr/bin/env python" and "Requires Python 2.7" * thus makes the whole Xorg world depend on python 2 I don't think it makes any sense to do the switch now or in the near future. thanks, -- js suse labs
Hi, On 27 Jun 11:03, Jiri Slaby wrote:
On 06/26/2017, 03:26 PM, jan matejek wrote:
* note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
Ugh, this will be tough.
An average system:
# grep '#!.*python$' /usr/bin/*|wc 107 182 4485 # grep '#!.*python[23]' /usr/bin/*|wc 51 52 1936
You can start with fixing llvm which: * has "#!/usr/bin/env python" and "Requires Python 2.7" * thus makes the whole Xorg world depend on python 2
python dependencies are only in subpackages which is not a Mesa dependency (hence Xorg) but I guess I might have missed a file or two, can you tell me which files you see these dependencies in? Regards, ismail -- SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
On Mon, 2017-06-26 at 15:26 +0200, jan matejek wrote:
What will happen:
What else will happen: * python packages that are not yet converted to singlespec will switch over to python3.
That can have nasty side-effects too: some packages received an explicitly Provides python2-FOO so that they can assist in the single- spec migration process; if those packages would now switch to be python3, then the provides is out of place and would need to be addressed. I'm not sure (yet) how we can easily identify those special cases though. Cheers, Dominique
On Mon, Jun 26, 2017 at 03:26:12PM +0200, jan matejek wrote:
* /usr/bin/python will be switched to point to /usr/bin/python3. Other unversioned python binaries will also be switched.
Hmm, my impression was that the python folks recommend that /usr/bin/python stays python2 for the time being, so that 3rd party applications won't break. See PEP 394. Has that changed? Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 28.6.2017 10:37, Michael Schroeder wrote:
On Mon, Jun 26, 2017 at 03:26:12PM +0200, jan matejek wrote:
* /usr/bin/python will be switched to point to /usr/bin/python3. Other unversioned python binaries will also be switched.
Hmm, my impression was that the python folks recommend that /usr/bin/python stays python2 for the time being, so that 3rd party applications won't break. See PEP 394. Has that changed?
Ah, the dreaded PEP 394. All of you who point to this are right, of course. I suppose we will keep /usr/bin/python pointed to python2 for the time being. On the other hand, the PEP was last touched in 2014 -- IIUC upstream devs themselves are reluctant to touch it because it's such an obvious can of worms. I expect this recommendation to change during 2018, possibly sooner. It seems sensible to me instead to change /usr/bin/python to an alternative, with python3 on low priority at first and switching to higher priority if the recommendation changes. There is still the potential to cause confusion on (new) installations that don't contain python2 -- if such user downloads a script from somewhere, that script might be broken. OTOH, not doing the alternative would mean that "python" command doesn't work at all, which seems worse to me. regards m.
On Mon, 2017-06-26 at 15:26 +0200, jan matejek wrote:
Fellow openSUSists,
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
First of all, big thanks to everyone who is converting (or has converted) their packages to singlespec. That alone makes the Python 3 switch much easier than it would be otherwise.
Then, details.
When: * in the coming weeks. I'd like to do some tests and gather feedback before actually moving forward. A heads-up e-mail will come before anything is actually headed to TW.
Why: * Python 2 is very near its end of life [1], the vast majority of our stuff already works in python3. We want to have python3 as default in SLE 15, so now's a very good time to start.
What will happen: * `python` package will be renamed to `python2` and will obsolete/provide `python <= 2.7.13`
Thinks like 'osc' - yes head -n1 /usr/bin/osc #!/usr/bin/python imho, we should leave 'python' as an alias to 'python2' and accept that if there is no python2 installed, there is no 'python<unversioned>' available. Keep python3 as python3 and get python4 to be python4. The only sensible alternative I'd seen in this thread seemed the variant with u-a - where we could set py3 as default at one point and make it really easy (by means of yast, using the alternatives module) to switch back to py2 for the users. cheers, Dominique
Seems porting from python2 to python3 is well documented: https://docs.python.org/3/howto/pyporting.html http://python3porting.com/ and even automatized: https://docs.python.org/2/library/2to3.html -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mon, Jun 26, 2017 at 9:26 AM, jan matejek <jmatejek@suse.com> wrote:
Fellow openSUSists,
this has been a long time coming, and I'd now like to start the actual process of switching the default Python implementation to Python 3 in Tumbleweed.
First of all, big thanks to everyone who is converting (or has converted) their packages to singlespec. That alone makes the Python 3 switch much easier than it would be otherwise.
Then, details.
When: * in the coming weeks. I'd like to do some tests and gather feedback before actually moving forward. A heads-up e-mail will come before anything is actually headed to TW.
Why: * Python 2 is very near its end of life [1], the vast majority of our stuff already works in python3. We want to have python3 as default in SLE 15, so now's a very good time to start.
What will happen: * `python` package will be renamed to `python2` and will obsolete/provide `python <= 2.7.13` * `python3` already provides `python = 3.6` and will also obsolete `python < %version` This should mean that if you don't require python2 on your system, it will be uninstalled, otherwise it will be replaced by python2. * /usr/bin/python will be switched to point to /usr/bin/python3. Other unversioned python binaries will also be switched. * python packages in patterns will be replaced by the corresponding python3 ones
What else will happen: * Stuff will break! The breakage will be contained to a testing project at first. * python packages that are not yet converted to singlespec will switch over to python3. Some of them will magically continue working, others will break. * For those we will also start getting file conflicts with python3 versions. A good opportunity to revisit and either completely drop the python2 version, convert to singlespec, or to explicitly-python2 package. * python packages that _are_ converted to singlespec will work just fine * as will explicitly-python3 packages from d:l:py3 * packages that _require_ python stuff will also magically get python3. They might very well also break * through this, we will learn where we still depend on Python 2, and hopefully fix it * note that if a script is using "/usr/bin/python" as shebang (or worse, "/usr/bin/env python"), it's already wrong. The shebang must be versioned, unless the script actually works with an arbitrary python version
regards m.
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me make an alternative proposal: Step 1 (these can all be done roughly simultaneously): 1.1. Rename python to python2. 1.2. We set up /usr/bin/python2 as an update-alternative to /usr/bin/python in the python2 package. 1.3. The python3 package provides no update-alternative for /usr/bin/python. Instead we provide a "python3-as2" package (or some similar name) that provides the update-alternative for /usr/bin/python with a higher priority than python2 does. So people can explicitly choose to install that package to change the symlink. 1.4.We have a package that is installed by default as part of the base system that includes a very low-priority update-alternative for /usr/bin/python and returns an error instructing people to install the python2 or python3-as2 package when they try to run a script with /usr/bin/python with neither of these packages installed. 1.5. We set an rpmlint warning for bare "python" shebangs, recommending these be changed to "python2" or "python3". This has to be explicitly disabled on a case-by-case basis using an rpmlintrc. 1.6. We set an rpmlint warning for packages that don't have the "python2-" prefix but depend on python2 or python2-foo packages, suggesting that the package change to python3 by default if possible. Again, this can be explicitly disabled using an rpmlintrc in cases where only python2 is supported (e.g. calibre) or where there are python2 and python3 versions but not prefix (e.g. spyder). 1.7. All python-foo packages will be renamed to python2-foo Step 2 (these can all be done roughly simultaneously): 2.1. Patterns will switch from using python2-foo packages to python3-foo. 2.2. Update-alternatives will prefer python3. If possible, and rpmlint check will be implemented to guarantee this. Failure to do this will be grounds for rejection from openSUSE:Factory. 2.3. The rpmlint warning for bare "python" shebangs becomes an error and grounds for exclusion from openSUSE:Factory. Step 3 (these can all be done roughly simultaneously): 3.1. Python2 will not be installed by default. 3.2. Python3 will be installed by default. Step Infinity (after upstream says we should change what /usr/bin/python points to): Inf.1. We merge "python3-as2" package into "python3", having python3 provide the same update-alternative with the same priority and provide/obsolete python3-as2 I think this approach will minimize unexpected breakage while still putting the emphasis on python3 and allowing us to change the default implemention with less breakage down the road. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am Freitag, 30. Juni 2017, 17:48:11 CEST schrieb Todd Rme:
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me make an alternative proposal:
...and a link /usr/lib/python pointing to the respective version would be great as well (no such created in Leap 42.3) Cheers Axle -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Axel Braun wrote:
Am Freitag, 30. Juni 2017, 17:48:11 CEST schrieb Todd Rme:
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me make an alternative proposal:
...and a link /usr/lib/python pointing to the respective version would be great as well (no such created in Leap 42.3)
No! This would surely cause trouble. Ciao, Michael.
Am Samstag, 1. Juli 2017, 11:42:11 CEST schrieb Michael Ströder:
Axel Braun wrote:
Am Freitag, 30. Juni 2017, 17:48:11 CEST schrieb Todd Rme:
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me
make an alternative proposal: ...and a link /usr/lib/python pointing to the respective version would be great as well (no such created in Leap 42.3)
No! This would surely cause trouble.
Why? In earlier openSUSE versions the link existed! Cheers Axel -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Todd Rme wrote:
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me make an alternative proposal:
Step 1 (these can all be done roughly simultaneously):
While I appreciate that you're trying to accomodate all issues raised in this thread at the end your proposal still leads to /usr/bin/python pointing to python3. I'm really concerned that scripts which are not tested for Python 3 are getting started under Python 3. Based on subtle differences the script might partially work but not fully function. This could even lead to subtle security issues. I'd strongly vote for /usr/bin/python not being present on systems without Python 2.x. Ciao, Michael.
Todd Rme wrote: … While I appreciate that you're trying to accomodate all issues raised in this thread at the end your proposal still leads to /usr/bin/python pointing to python3.
But for me as a user, /usr/bin/python -> python3 is what I want. I also don’t understand the quoted recommendation that this shall be undesired. What is the point of writing python code compatible with python2 and python3 to run on new machines with insufficient python2 support (als TW) and old machines w/o python3 concurrently then? Without „/usr/bin/python“ I see no change to continue this?! I like it, that python2 can run almost all python3 code. And I like the tool 2to3 which automatically converts almost all (of my) python2 code to run concurrently on 2 and 3. If there is code, which cannot or does not wanted to be upgraded one simply has to replace /usr/bin/python by /usr/bin/python2 and wait up to the time no python2 is available any more and one finally is forced to upgrade the code. By sticking /usr/bin/python to python2, the „brand“ /usr/bin/python is dead „forever“ once python2 disappears from the system due to EOL. Indeed I currently let point /usr/bin/python -> python3 and the only issue I have is, that the textext plugin and eps importer from inkscape does not work. Fabian-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Fabian Wein wrote:
But for me as a user, /usr/bin/python -> python3 is what I want.
Why? Which issue do you want to solve? Just typing "python" instead of "python3"?
What is the point of writing python code compatible with python2 and python3 to run on new machines with insufficient python2 support (als TW) and old machines w/o python3 concurrently then?
How do you know that Python code with shee-bang /usr/bin/python is compatible to Python 2.x and 3.x? If it's not fully compatible how do you avoid damage to your system, security issues etc.?
If there is code, which cannot or does not wanted to be upgraded one simply has to replace /usr/bin/python by /usr/bin/python2 and wait up to the time no python2 is available any more and one finally is forced to upgrade the code.
*Who* is going to change the shee-bang to /usr/bin/python2 if the upstream developers uses /usr/bin/python assuming that Python 2 will implicitly be used? *Who* is going to maintain all the necessary patches?
By sticking /usr/bin/python to python2, the „brand“ /usr/bin/python is dead „forever“
Yes. So what?
Indeed I currently let point /usr/bin/python -> python3 and the only issue I have is, that the textext plugin and eps importer from inkscape does not work.
But why do you assume your personal experience to be relevant at all for all the other deployments using Python? Ciao, Michael.
Am 01.07.2017 um 15:59 schrieb Michael Ströder:
How do you know that Python code with shee-bang /usr/bin/python is compatible to Python 2.x and 3.x?
If it's not fully compatible how do you avoid damage to your system, security issues etc.?
As you seem to know python 2 vs 3 glitches: do you have any examples of python2 code that worries you to create security issues or damage if run under python3? My own experiences are rather quite fatal when even trying to run old python code under python3. Because this 'you never know what code can do if it's not adapted to the new version' argument is basically a killer argument against *any* update - and these updates are the spirit of openSUSE Tumbleweed. In this discussion I heared so far that a python switch would kill users, damage systems and cause security issues. This is tough language for an update about a programming language who's biggest harm is the abuse of whitespace ;) Greetings, Stephan -- Ma muaß weiterkämpfen, kämpfen bis zum Umfalln, a wenn die ganze Welt an Arsch offen hat, oder grad deswegn. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Stephan Kulow wrote:
Am 01.07.2017 um 15:59 schrieb Michael Ströder:
How do you know that Python code with shee-bang /usr/bin/python is compatible to Python 2.x and 3.x?
If it's not fully compatible how do you avoid damage to your system, security issues etc.?
As you seem to know python 2 vs 3 glitches: do you have any examples of python2 code that worries you to create security issues or damage if run under python3?
Personally I ran into problems where behaviour of custom dict classes is different but does not necessarily lead to exceptions being raised. Just the (caching key lookup) behaviour was different.
My own experiences are rather quite fatal when even trying to run old python code under python3.
You mean you always saw the code clearly failing with exceptions?
In this discussion I heared so far that a python switch would kill users, damage systems and cause security issues. This is tough language for an update about a programming language who's biggest harm is the abuse of whitespace ;)
The main problem with this discussion is that pretty simple assumptions are made even ignoring PEP advice without thorough analysis of all the affected Python code shipped with openSUSE (which is impossible). Ciao, Michael.
Stephan Kulow wrote:
Am 01.07.2017 um 15:59 schrieb Michael Ströder:
How do you know that Python code with shee-bang /usr/bin/python is compatible to Python 2.x and 3.x?
If it's not fully compatible how do you avoid damage to your system, security issues etc.?
As you seem to know python 2 vs 3 glitches: do you have any examples of python2 code that worries you to create security issues or damage if run under python3?
Personally I ran into problems where behaviour of custom dict classes is different but does not necessarily lead to exceptions being raised. Just the (caching key lookup) behaviour was different. This happened between perl 5.16 and perl 5.18 just the same - and no one asked for sticking to perl 5.16 forever. Problems as these are facts of
Am 01.07.2017 um 16:28 schrieb Michael Ströder: life in a moving environment.
My own experiences are rather quite fatal when even trying to run old python code under python3.
You mean you always saw the code clearly failing with exceptions?
Right. Greetings, Stephan -- Ma muaß weiterkämpfen, kämpfen bis zum Umfalln, a wenn die ganze Welt an Arsch offen hat, oder grad deswegn. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Sat, Jul 1, 2017 at 4:08 PM, Stephan Kulow <coolo@suse.de> wrote:
Am 01.07.2017 um 15:59 schrieb Michael Ströder:
How do you know that Python code with shee-bang /usr/bin/python is compatible to Python 2.x and 3.x?
If it's not fully compatible how do you avoid damage to your system, security issues etc.?
As you seem to know python 2 vs 3 glitches: do you have any examples of python2 code that worries you to create security issues or damage if run under python3?
My own experiences are rather quite fatal when even trying to run old python code under python3.
Hi, I think the changes in integer division between 2 and 3 are likely to break things in subtle ways which might first go unnoticed...2to3 does not automagically change something like print(4/3) into something like print(4//3). Cheers, Mischa -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Sat, Jul 1, 2017 at 8:53 AM, Fabian Wein <fabian.wein@fau.de> wrote:
Todd Rme wrote: … While I appreciate that you're trying to accomodate all issues raised in this thread at the end your proposal still leads to /usr/bin/python pointing to python3.
But for me as a user, /usr/bin/python -> python3 is what I want. I also don’t understand the quoted recommendation that this shall be undesired.
In my opinion if we are going to break from upstream I think we need a very, very strong reason. Personal preference isn't sufficient in my opinion. I think defaulting to upstream's recommendation while allowing people like you to easily change it in a strictly opt-in manner is the best course of action.
By sticking /usr/bin/python to python2, the „brand“ /usr/bin/python is dead „forever“ once python2 disappears from the system due to EOL.
The end result of my proposal is that nothing would be using "/usr/bin/python" anymore, so that shouldn't be an issue. And honestly I doubt python2 will be removed from openSUSE once it is EOL, Red Hat and other distros will continue to provide security updates we can pull from. But I think that upstream python2 EOL should be the last possible date where python2 is installed by default (although ideally it will not be installed by default long before then). -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 07/03/17 17:03, Todd Rme wrote:
In my opinion if we are going to break from upstream I think we need a very, very strong reason. Personal preference isn't sufficient in my opinion. I think defaulting to upstream's recommendation while allowing people like you to easily change it in a strictly opt-in manner is the best course of action.
I see your point and having the opportunity to opt in for the switch is fine enough for me. Currently every dup breaks my symlink what is indeed quite annoying. Thanks, Fabian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Sat, Jul 1, 2017 at 5:49 AM, Michael Ströder <michael@stroeder.com> wrote:
Todd Rme wrote:
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me make an alternative proposal:
Step 1 (these can all be done roughly simultaneously):
While I appreciate that you're trying to accomodate all issues raised in this thread at the end your proposal still leads to /usr/bin/python pointing to python3.
Not unless the user intentionally installs an optional package. Only people who want "/usr/bin/python" pointing to python3 would install this package. So the normal and expected end-result would be "/usr/bin/python" pointing to python2. Only people who really, really want "/usr/bin/python" pointing to python3 would install this package. People can override the symlink even now, this just provides a standardized way to do it that doesn't break after every update. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Todd Rme wrote:
On Sat, Jul 1, 2017 at 5:49 AM, Michael Ströder <michael@stroeder.com> wrote:
Todd Rme wrote:
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me make an alternative proposal:
Step 1 (these can all be done roughly simultaneously):
While I appreciate that you're trying to accomodate all issues raised in this thread at the end your proposal still leads to /usr/bin/python pointing to python3.
Not unless the user intentionally installs an optional package. Only people who want "/usr/bin/python" pointing to python3 would install this package.
Sorry, but if such a package exists surely at least one package mantainer will add it as (recommended) dependency just to enforce his/her personal preference. Ciao, Michael.
On Mon, Jul 3, 2017 at 11:43 AM, Michael Ströder <michael@stroeder.com> wrote:
Todd Rme wrote:
On Sat, Jul 1, 2017 at 5:49 AM, Michael Ströder <michael@stroeder.com> wrote:
Todd Rme wrote:
Considering the upstream recommendation that we not make /usr/bin/python point to python3 that others have brought up, let me make an alternative proposal:
Step 1 (these can all be done roughly simultaneously):
While I appreciate that you're trying to accomodate all issues raised in this thread at the end your proposal still leads to /usr/bin/python pointing to python3.
Not unless the user intentionally installs an optional package. Only people who want "/usr/bin/python" pointing to python3 would install this package.
Sorry, but if such a package exists surely at least one package mantainer will add it as (recommended) dependency just to enforce his/her personal preference.
An rpmlint error could be set that would prevent that. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (25)
-
Aleksa Sarai -
Axel Braun -
Bernhard M. Wiedemann -
Christian Boltz -
Dominique Leuenberger / DimStar -
Fabian Wein -
Greg Freemyer -
İsmail Dönmez -
James Mason -
jan matejek -
jan matejek -
Jiri Slaby -
Martin Pluskal -
Michael Schroeder -
Michael Ströder -
Mischa Salle -
opensuse.lietuviu.kalba -
Peter Simons -
Robert Schweikert -
Ruediger Meier -
Rüdiger Meier
-
Simon Lees -
Stefan Behlert -
Stephan Kulow -
Todd Rme