New subject: [opensuse-packaging] Re: python singlespec: how to convert your package

17 Feb 2017

      Dear packagers,

this is an overview of how singlespec works and how to use it for your
package. First, some key concepts:

* It will allow you to build Python 2 and Python 3 (and PyPy 3 in the
future) package from a single spec file, with very little manual effort.
This means that if you're maintaining a separate python3-something in
d:l:py3, you can now switch over to the common python-something in d:l:py.

* It is designed to work on specs called "python-something". There is
some support for "python2-something" and "python3-something", but it is
mostly untested.
If your spec file name doesn't start with "python", don't even try.
(Yet.)

* (if you're building for Factory,) all your dependencies must be
converted first.
That's because the macros will BuildRequire "python2-things", which only
exist for packages that are already converted.
The devel:languages:python:singlespec already has basic packages, and
not all of them will be in d:l:py immediately, so if you want to play
now, build against this project. I'll also be accepting SRs into it.

* In general, if you're unsure, please submit to d:l:py:singlespec
first. Also examine linkdiffs of packages in d:l:py:singlespec to get an
idea of how it can look like.
https://build.opensuse.org/project/show/devel:languages:python:singlespec

* Individual macros are documented on github
https://github.com/openSUSE/python-rpm-macros

* I'm working on an autoconverter that performs most of the following
steps automatically. I intend to use it to convert most of d:l:py into
d:l:py:singlespec:staging, and then review and submit packages one by
one. So maybe if you wait, you can save yourself some work ;)

Conversion steps:

0. If you want to build for anything other than Factory, you need to
include compatibility shims.
This definition of the %python_module macro goes on top of your spec:
%{?!python_module:%define python_module() python-%1 python3-%1}
and a buildrequire:
BuildRequires: python-rpm-macros

1. Wrap all your "BuildRequires: python-something":
BuildRequires: %{python_module something}
Do not do this in Requires or any other fields.
If you have a buildrequire that should only apply to one python, say,
python-enum34 only for python 2, do not wrap it.

2. At the end of the spec preamble, before %description, place the macro:
%python_subpackages
on a line by itself

3. Replace "python setup.py build" with %python_build
and "python setup.py install" with %python_install.
These already contain the usual arguments (--prefix, --root), so usually
you don't need to add any.
If you configure CFLAGS or something like that, export them first:

CFLAGS="-fwrapv" python setup.py build
->
export CFLAGS="-fwrapv"
%python_build

If you are running any other commands based on python, use %python_exec.
"python setup.py test" -> "%python_exec setup.py test"
This works for usual test runners too:
"nosetests" -> "%python_exec %{_bindir}/nosetests"

4. If you're doing something more complicated ... say, removing files
from %python2_sitelib and %python3_sitelib, use %python_expand.
"%python_expand rm -f %{$python_sitelib}/foo/undesirable.txt"

Yes, that is "%$python_sitelib". Beware. If you put only
"%python_sitelib", it will not work, because "%python_sitelib" gets
expanded before %python_expand touches it.

In general, after %python_expand, "$python" is replaced by all pythons
even in macro names.
If you need more lines, wrap in braces:
%{python_expand firstline
secondline $python
thirdline}
Note that the first line must not be empty.

5. change "%files" to "%files %{python_files}"
If you have subpackages, say "%files foo", then
change to "%files %{python_files foo}".

6. If some files are only in one python version, mark them as
%python2_only or %python3_only in the filelist.
If there's more of them, you can use %ifpython2 ... %endif
If the entry in question is a __pycache__ directory, use %pycache_only
or %ifpycache.

7. If you are using update-alternatives, remove them.
Instead mark the executable as %python3_only:
%python3_only %{_bindir}/yourbinary
In most cases, we only need one version of the executable. If the
purpose of the tool is, e.g., reading EXIF metadata, we don't care if
python 2 or 3 reads them. (And the library files are still installed for
both.)

(If you are sure that you need executables for all python versions, look
at what python-bottle does:
https://build.opensuse.org/package/view_file/devel:languages:python:singlesp...
note the installation inside %python_expand. Usually, packages don't
install versioned executables by themselves, so you need to do it
manually. If your package does, good for you.)

8. Take the time to enable automatic tests :)

have a nice weekend
m.

[opensuse-packaging] python singlespec: how to convert your package

tags

participants (14)