New version for a python package doesn't support python3.6 -- How to build for LEAP 15.4
All,
I'm trying to update security:forensics/python-dfVFS
I've branched it to: https://build.opensuse.org/package/show/home:gregfreemyer:Tools-for-forensic...
The new release builds fine for factory, but for LEAP it fails with:
[ 75s] + /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s' [ 76s] Unsupported Python version: 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC], version 3.7 or higher required. [ 76s] error: Bad exit status from /var/tmp/rpm-tmp.LM9LB1 (%build)
Is there a relatively easy way to get this build. I would really like to have this available for 15.4 at a minimum (in addition to factory).
Thanks Greg -- Greg Freemyer Advances are made by answering questions. Discoveries are made by questioning answers. — Bernard Haisch
Hi,
Am 02.12.22 um 01:19 schrieb Greg Freemyer:
All,
I'm trying to update security:forensics/python-dfVFS
I've branched it to: https://build.opensuse.org/package/show/home:gregfreemyer:Tools-for-forensic...
The new release builds fine for factory, but for LEAP it fails with:
[ 75s] + /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s' [ 76s] Unsupported Python version: 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC], version 3.7 or higher required. [ 76s] error: Bad exit status from /var/tmp/rpm-tmp.LM9LB1 (%build)
Is there a relatively easy way to get this build. I would really like to have this available for 15.4 at a minimum (in addition to factory).
You have to patch out the following line that upstream does not support Python <3.7:
https://github.com/log2timeline/dfvfs/blob/668613e22c39759de62dbde466e030054...
Afterwards you have to patch any code that uses features introduced after Python 3.6, sometimes there are backports packages which you can use.
OTOH is has been mentioned over and over again that Python 3.6 is dead and that any effort to circumvent this in Leap is hard and not supported by the community. Good luck with your journey!
Thanks Greg -- Greg Freemyer Advances are made by answering questions. Discoveries are made by questioning answers. — Bernard Haisch
- Ben
On Thu, Dec 1, 2022 at 8:46 PM Ben Greiner code@bnavigator.de wrote:
Hi,
Am 02.12.22 um 01:19 schrieb Greg Freemyer:
All,
I'm trying to update security:forensics/python-dfVFS
I've branched it to: https://build.opensuse.org/package/show/home:gregfreemyer:Tools-for-forensic...
The new release builds fine for factory, but for LEAP it fails with:
[ 75s] + /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s' [ 76s] Unsupported Python version: 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC], version 3.7 or higher required. [ 76s] error: Bad exit status from /var/tmp/rpm-tmp.LM9LB1 (%build)
Is there a relatively easy way to get this build. I would really like to have this available for 15.4 at a minimum (in addition to factory).
You have to patch out the following line that upstream does not support Python <3.7:
https://github.com/log2timeline/dfvfs/blob/668613e22c39759de62dbde466e030054...
Afterwards you have to patch any code that uses features introduced after Python 3.6, sometimes there are backports packages which you can use.
OTOH is has been mentioned over and over again that Python 3.6 is dead and that any effort to circumvent this in Leap is hard and not supported by the community. Good luck with your journey!
I was more hoping to force python-dfVFS to be a python39 or python310 package. Both python39 and python310 are in LEAP 15.4.
But, I'm not a python programmer. Am I thinking this will be simpler than it really is?
FYI: python-dfVFS uses a large number of libraries that would not be in the basic python39 RPMs. In Tumbleweed, I have the full suite of libraries for python38, python39, python310, etc. For my own personal use, can I just gather up those RPMs and manually install them?
Maybe I just need to give up on LEAP 15.4 as a target for the latest release of python-dfVFS?
Thanks, Greg
Hi Greg,
On 02.12.22 at 03:49 Greg Freemyer wrote:
I was more hoping to force python-dfVFS to be a python39 or python310 package. Both python39 and python310 are in LEAP 15.4.
I had a similar situation with ansible requiring python >= 3.8. So I built against the SLES15 Python3 module that brings python310.
I had to modify prjconf to set the %pythons macro to python310, not python36.
https://build.opensuse.org/project/show/home:ojkastl_buildservice:ansible_fo...
In your case you would have to skip building for python 3.6 in your spec file (if you want to use the same file for Tumbleweed and Leap) or just make this a single-python-version-package.
Not sure how many python310 packages Leap brings on its own, so you might have to build a lot of packages that are required.
Kind Regards, Johannes
Am Freitag, 2. Dezember 2022, 01:19:20 CET schrieb Greg Freemyer:
All,
I'm trying to update security:forensics/python-dfVFS
I've branched it to: https://build.opensuse.org/package/show/home:gregfreemyer:Tools-for-forensic -boot-cd/python-dfVFS
The new release builds fine for factory, but for LEAP it fails with:
[ 75s] + /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s' [ 76s] Unsupported Python version: 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC], version 3.7 or higher required. [ 76s] error: Bad exit status from /var/tmp/rpm-tmp.LM9LB1 (%build)
Is there a relatively easy way to get this build. I would really like to have this available for 15.4 at a minimum (in addition to factory).
No. And I wouldn't start patching this version to be able to use python 3.6. Not even 3.8 or 3.9. Because then you have to build all modules which the package needs also with python 3.8 or 3.9. It can also work without that. But only maybe. Why suse doesn't build python in leap like it does under tumbleweed is beyond me. Because then you could still leave 3.6 as default (If you really want to. I don't understand it. But good). I always do that so that I look for the last version which still use the dead python 3.6. More is not possible. Everything else can get out of hand. In your case this should be the version 20220430.
Regards Eric
Dne 02. 12. 22 v 11:49 Eric Schirra napsal(a):
Why suse doesn't build python in leap like it does under tumbleweed is beyond me.
From Wikipedia:
SUSE Linux Enterprise Server 15 (SLES 15) beta 1 was released on
October 18, 2017, and the final version was released on July 16, 2018.
1. We didn't have the technology for building parallel packages then 2. Whole premise of SLE-15 is that it never changes for the length of its support. That's what our customers are paying us for. 3. Yes, we are aware of this problem and we are working on a solution (actually, I have just finished one planning session with my boss).
Best,
Matěj
Am Freitag, 2. Dezember 2022, 13:59:22 CET schrieb Matěj Cepl:
Dne 02. 12. 22 v 11:49 Eric Schirra napsal(a):
Why suse doesn't build python in leap like it does under tumbleweed is beyond me.
From Wikipedia:
SUSE Linux Enterprise Server 15 (SLES 15) beta 1 was released on
October 18, 2017, and the final version was released on July 16, 2018.
- We didn't have the technology for building parallel packages then
- Whole premise of SLE-15 is that it never changes for the length of
its support. That's what our customers are paying us for.
I am aware of that. But for me no excuse that there can not also be newer python version.
- Yes, we are aware of this problem and we are working on a solution
(actually, I have just finished one planning session with my boss).
Ah. Great to hear. And thus you confirm my upper statement. :-) Hope it goes through. Can we hope that in Leap 15.5 there will be newer python versions then if you want? And not only python itself, but also all modules to it?
Regards Eric
Am Freitag, 2. Dezember 2022, 14:43:16 CET schrieb Eric Schirra:
Am Freitag, 2. Dezember 2022, 13:59:22 CET schrieb Matěj Cepl:
Dne 02. 12. 22 v 11:49 Eric Schirra napsal(a):
Why suse doesn't build python in leap like it does under tumbleweed is beyond me.
From Wikipedia:
SUSE Linux Enterprise Server 15 (SLES 15) beta 1 was released on
October 18, 2017, and the final version was released on July 16, 2018.
- We didn't have the technology for building parallel packages then
- Whole premise of SLE-15 is that it never changes for the length of
its support. That's what our customers are paying us for.
...and openSUSE is suffering from that, as Leap is tied to SLE.
I am aware of that. But for me no excuse that there can not also be newer python version.
We had that lengthy discussion. I see this the same way.
Only chance is to keep the last version of $PACKAGE that still supports Python 3.6. If security issues are fixed in a version that requires newer Python, SUSE should take action and backport these. Because they are paid for it. Even if it 'only' affects Leap.
- Yes, we are aware of this problem and we are working on a solution
(actually, I have just finished one planning session with my boss).
Ah. Great to hear. And thus you confirm my upper statement. :-) Hope it goes through. Can we hope that in Leap 15.5 there will be newer python versions then if you want?
yes, but not as default python version. Unfortunately
My 2c Axel
Am Freitag, 2. Dezember 2022, 14:57:33 CET schrieb Axel Braun:
- We didn't have the technology for building parallel packages then
- Whole premise of SLE-15 is that it never changes for the length of
its support. That's what our customers are paying us for.
...and openSUSE is suffering from that, as Leap is tied to SLE.
I am aware of that. But for me no excuse that there can not also be newer python version.
We had that lengthy discussion. I see this the same way.
Only chance is to keep the last version of $PACKAGE that still supports Python 3.6. If security issues are fixed in a version that requires newer Python, SUSE should take action and backport these. Because they are paid for it. Even if it 'only' affects Leap.
- Yes, we are aware of this problem and we are working on a solution
(actually, I have just finished one planning session with my boss).
Ah. Great to hear. And thus you confirm my upper statement. :-) Hope it goes through. Can we hope that in Leap 15.5 there will be newer python versions then if you want?
yes, but not as default python version. Unfortunately
I find that now for the time being no matter, provided that I can change if I want and if all modules are available. Because with version 3.6 you can actually no longer start with current packages. One could also delete the same. Would have almost the same effect. But as I said if I can switch with everything I do not care about the standard. But only if it really works.
Regards Eric
On 02.12.22 16:42, Eric Schirra ecsos@opensuse.org wrote:
Am Freitag, 2. Dezember 2022, 14:57:33 CET schrieb Axel Braun:
- We didn't have the technology for building parallel packages then
- Whole premise of SLE-15 is that it never changes for the length of
its support. That's what our customers are paying us for.
...and openSUSE is suffering from that, as Leap is tied to SLE.
I am aware of that. But for me no excuse that there can not also be newer python version.
We had that lengthy discussion. I see this the same way.
Only chance is to keep the last version of $PACKAGE that still supports Python 3.6. If security issues are fixed in a version that requires newer Python, SUSE should take action and backport these. Because they are paid for it. Even if it 'only' affects Leap.
- Yes, we are aware of this problem and we are working on a solution
(actually, I have just finished one planning session with my boss).
Ah. Great to hear. And thus you confirm my upper statement. :-) Hope it goes through. Can we hope that in Leap 15.5 there will be newer python versions then if you want?
yes, but not as default python version. Unfortunately
I find that now for the time being no matter, provided that I can change if I want and if all modules are available. Because with version 3.6 you can actually no longer start with current packages. One could also delete the same. Would have almost the same effect. But as I said if I can switch with everything I do not care about the standard. But only if it really works.
Regards Eric
A first step could be a repository like 15.4_py310 in d:l:p and there subprojects. I thing the most packages works well already. A few packages have hard coded python3 dependencies, but these one could split in mulitbuild or change to the actual/primary python. This will be find out when these repositories are available.
The whole python maintainers did a great job already and to use a second python version beside the default one is a short leap.
Torsten
Hi,
I am cross-posting to python@lists.opensuse.org, because I think the discussion as progressed into a topic more suitable for that list. I suggest replying to that list only.
On 07.12.22 15:37, t.gruner@katodev.de wrote:
A first step could be a repository like 15.4_py310 in d:l:p and there subprojects. I thing the most packages works well already. A few packages have hard coded python3 dependencies, but these one could split in mulitbuild or change to the actual/primary python. This will be find out when these repositories are available.
The whole python maintainers did a great job already and to use a second python version beside the default one is a short leap.
Yes coinstallable python versions work great in Tumbleweed, but unfortunately the adoption to Leap is not a walk in the park. Until recently (Matej mentioned the plans for 15.5) SUSE denied multi-flavors for SLE/Leap. Thus, many specfiles assume:
%if 0%{?suse_version} < 1550 # I only have python3 (and python2 in <=15.3) and it is 3.6 and that won't change %endif
This assumption needs to be rectified in more than a few specfiles.
Another aspect: Just making 15.4_py310 in d:l:p without connections to its subprojects would fail at the first package in the dependency chain outside of d:l:p, e.g. pytest.
d:l:p:backports solves this by linking almost all python packages from Factory into one project. However, have a look at the already existing 15.4_py39 there, it is not in a good state:
https://build.opensuse.org/project/monitor/devel:languages:python:backports?...
Whoever set it up, I think they would welcome contributions to make it better. It may even be bumped to 15.4_py310 or a future 15.5_py310 for development of the announced coinstallability in 15.5 eventually.
- Ben
For anyone who finds this in the archives, I got python 3.9 builds for Leap 15.4 to work in OBS.
1) security:forensics now has approximately 50 python 3.9 packages installable for 15.4. Most required no spec file changes at all.
2) I added devel:languages:python:backports/15.4_py39 as a primary repository for security:forensics to build against. It has python 3.9 and most of the python core modules linked in.
3) I performed install testing of some of the security:forensics python modules with complex dependency chains. For some unknown reason (or unknown to me) "zypper in" didn't cause packages from devel:languages:python:backports/15.4_py39 to get pullied in during "zypper in" actions, so I branched 10 or 15 core modules from devel:languages:python to security:forensics in order to satisfy dependencies.
4) My primary test was "zypper in python39-dfVFS" on a Leap 15.4 machine. It now works.
Thanks for those that built the devel:languages:python:backports/15.4_py39 repo and for those that pointed me at it.
Greg
On Thu, Dec 1, 2022 at 7:19 PM Greg Freemyer greg.freemyer@gmail.com wrote:
All,
I'm trying to update security:forensics/python-dfVFS
I've branched it to: https://build.opensuse.org/package/show/home:gregfreemyer:Tools-for-forensic...
The new release builds fine for factory, but for LEAP it fails with:
[ 75s] + /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s' [ 76s] Unsupported Python version: 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC], version 3.7 or higher required. [ 76s] error: Bad exit status from /var/tmp/rpm-tmp.LM9LB1 (%build)
Is there a relatively easy way to get this build. I would really like to have this available for 15.4 at a minimum (in addition to factory).
Thanks Greg -- Greg Freemyer Advances are made by answering questions. Discoveries are made by questioning answers. — Bernard Haisch
On Friday 2022-12-16 23:26, Greg Freemyer wrote:
- I added devel:languages:python:backports/15.4_py39 as a primary
repository for security:forensics
- I performed install testing of some of the security:forensics
python modules with complex dependency chains. For some unknown reason (or unknown to me) "zypper in" didn't cause packages from devel:languages:python:backports/15.4_py39 to get pullied in
What you described as 'primary repo' is a build-time construct only, it does not carry over to zypper.
On Fri, Dec 16, 2022 at 6:02 PM Jan Engelhardt jengelh@inai.de wrote:
On Friday 2022-12-16 23:26, Greg Freemyer wrote:
- I added devel:languages:python:backports/15.4_py39 as a primary
repository for security:forensics
- I performed install testing of some of the security:forensics
python modules with complex dependency chains. For some unknown reason (or unknown to me) "zypper in" didn't cause packages from devel:languages:python:backports/15.4_py39 to get pullied in
What you described as 'primary repo' is a build-time construct only, it does not carry over to zypper.
It's rather annoying!
Is there a work around to not force every python module in devel:languages:python that forensics:security depends on to not have to have a branch from d:l:p?
For python-dfdatetime and python-dfVFS it's not a crazy number of modules to branch from d:l:p.
But for python-plaso, it is getting to be a long list to branch from d:l:p.
Thanks, Greg
On 17.12.2022 05:17, Greg Freemyer wrote:
On Fri, Dec 16, 2022 at 6:02 PM Jan Engelhardt jengelh@inai.de wrote:
On Friday 2022-12-16 23:26, Greg Freemyer wrote:
- I added devel:languages:python:backports/15.4_py39 as a primary
repository for security:forensics
- I performed install testing of some of the security:forensics
python modules with complex dependency chains. For some unknown reason (or unknown to me) "zypper in" didn't cause packages from devel:languages:python:backports/15.4_py39 to get pullied in
What you described as 'primary repo' is a build-time construct only, it does not carry over to zypper.
It's rather annoying!
Is there a work around to not force every python module in devel:languages:python that forensics:security depends on to not have to have a branch from d:l:p?
Have you added repository devel:languages:python:backports/15.4_py39 to zypper?
For python-dfdatetime and python-dfVFS it's not a crazy number of modules to branch from d:l:p.
But for python-plaso, it is getting to be a long list to branch from d:l:p.
Thanks, Greg
On Sat, Dec 17, 2022 at 1:58 AM Andrei Borzenkov arvidjaar@gmail.com wrote:
On 17.12.2022 05:17, Greg Freemyer wrote:
On Fri, Dec 16, 2022 at 6:02 PM Jan Engelhardt jengelh@inai.de wrote:
On Friday 2022-12-16 23:26, Greg Freemyer wrote:
- I added devel:languages:python:backports/15.4_py39 as a primary
repository for security:forensics
- I performed install testing of some of the security:forensics
python modules with complex dependency chains. For some unknown reason (or unknown to me) "zypper in" didn't cause packages from devel:languages:python:backports/15.4_py39 to get pullied in
What you described as 'primary repo' is a build-time construct only, it does not carry over to zypper.
It's rather annoying!
Is there a work around to not force every python module in devel:languages:python that forensics:security depends on to not have to have a branch from d:l:p?
Have you added repository devel:languages:python:backports/15.4_py39 to zypper?
To the best of my knowledge, yes:
zypper lr 1 Alias : devel_languages_python_backports Name : Backport builds of Python Modules (15.4) URI : https://download.opensuse.org/repositories/devel:/languages:/python:/backpor... Enabled : Yes GPG Check : (r ) Yes Priority : 99 (default priority) Autorefresh : Off Keep Packages : Off Type : rpm-md GPG Key URI : https://download.opensuse.org/repositories/devel:/languages:/python:/backpor... Path Prefix : Parent Service : Keywords : --- Repo Info Path : /etc/zypp/repos.d/devel_languages_python_backports.repo MD Cache Path : /var/cache/zypp/raw/devel_languages_python_backports
Greg
Am 18.12.22 um 03:02 schrieb Greg Freemyer:
On Sat, Dec 17, 2022 at 1:58 AM Andrei Borzenkov arvidjaar@gmail.com wrote:
On 17.12.2022 05:17, Greg Freemyer wrote:
On Fri, Dec 16, 2022 at 6:02 PM Jan Engelhardt jengelh@inai.de wrote:
On Friday 2022-12-16 23:26, Greg Freemyer wrote:
- I added devel:languages:python:backports/15.4_py39 as a primary
repository for security:forensics
- I performed install testing of some of the security:forensics
python modules with complex dependency chains. For some unknown reason (or unknown to me) "zypper in" didn't cause packages from devel:languages:python:backports/15.4_py39 to get pullied in
What you described as 'primary repo' is a build-time construct only, it does not carry over to zypper.
It's rather annoying!
Is there a work around to not force every python module in devel:languages:python that forensics:security depends on to not have to have a branch from d:l:p?
Have you added repository devel:languages:python:backports/15.4_py39 to zypper?
To the best of my knowledge, yes:
zypper lr 1 Alias : devel_languages_python_backports Name : Backport builds of Python Modules (15.4) URI : https://download.opensuse.org/repositories/devel:/languages:/python:/backpor... Enabled : Yes GPG Check : (r ) Yes Priority : 99 (default priority) Autorefresh : Off Keep Packages : Off Type : rpm-md GPG Key URI : https://download.opensuse.org/repositories/devel:/languages:/python:/backpor... Path Prefix : Parent Service : Keywords : --- Repo Info Path : /etc/zypp/repos.d/devel_languages_python_backports.repo MD Cache Path : /var/cache/zypp/raw/devel_languages_python_backports
Greg
That is backports/15.4/ but you would need backports/15.4_py39/ which, again, is NOT recommended. There is too much nonfunctional stuff in there. Make your own subrepo with just the packages you need.
- Be
On Sun, 18 Dec 2022 12:30:27 +0100 Ben Greiner wrote:
That is backports/15.4/ but you would need backports/15.4_py39/ which, again, is NOT recommended. There is too much nonfunctional stuff in there. Make your own subrepo with just the packages you need.
So far, backports/15.4_py39 has been great for getting the pytest and friends from, so I don't have to branch/build those, since there is, AFAICT, a *long* dependency chain for them :) IIRC, when I built QGIS python deps with 15.3 python3.9, tests and dependencies for them were easily the bulk of the repo (which had ~250 packages, in total), which was...fun.
Pedja
-
Andrei Borzenkov -
Axel Braun -
Ben Greiner -
Eric Schirra -
Greg Freemyer -
Jan Engelhardt -
Johannes Kastl -
Matěj Cepl -
Predrag Ivanović -
t.gruner＠katodev.de