Mailinglist Archive: opensuse-packaging (54 mails)

< Previous Next >
[opensuse-packaging] python single-spec progress, questions
hello packagers,

i'm happy to announce that i have a macroset that works pretty well, if
i say so myself :)
Have a look at the d:l:python:singlespec repository [1] and the github
repo [2] for details.

The gist of it: it is now very possible to build packages for both
python 2 and python 3, using a pretty straightforward set of macros. See
spec files in the linked repository, and documentation on github.

So far this only reliably works on python 2 and 3. It can work with pypy
too, but pypy is not building so I can't test against it.

Now there is a number of remaining issues to be worked out, and I'd like
your feedback on them:
1. Necessary changes to OBS configuration
2. Backwards compatibility
3. Transition to new macros
4. New policies for d:l:py
5. Naming scheme


1. Necessary changes to OBS configuration
---
The macro set only works with the one particular repository, that has
the macros in its prjconf. In particular, the %python_module macro is
problematic, because it needs to be expanded *before* the build starts,
by OBS. Usually, we would put the macro definitions inside
/etc/rpm/macros.python, as a part of the python/python3/whichever
package, but that won't work in this case.

Before we can deploy these packages to Factory, either Factory's
prjconf, or OBS global config, needs to include at least the
%python_module macro.

And if possible, I'd like to include all of them, because:

2. Backwards compatibility
---
Basically, we would need to build for old distros using new macros.
There's a number of options:
a) macro definitions in OBS and everything will build everywhere
b) macro definitions in python, everything will build with new enough
version of python
c) macro definitions in prjconf of particular projects (say d:l:py),
only packages in those projects build.
Personally, i'd go with (a). And if that's not possible, with (b) and
(c) combined.

The problem with anything other than (a) is that it is going to be
difficult to write specs that work both with and without the macros. You
could conditionally include the %python_packages macro, but you would
have to shim all the others in every spec file, and that is going to be
ugly.
Which means that by using the new macros, we would be hard-forking the
python packages from ALL currently released distributions, unless we
could get the macro definitions into them.

3. Transition to new macros
---
There is a script called 'process-spec.pl' in the github repo. I'm about
to run it first against (some small subset of) python-* in d:l:py, then
against python3-* in d:l:py3, and put the results in
d:l:py:singlespec:staging. Then it will be necessary to review the
packages and submit to d:l:py:singlespec. That's going to be a lot of
work, and I would like to ask you to help with this. I'll put up a
separate call for volunteers next week.

In the meantime, feel free to grab the macro definitions from prjconf
and play around. And if you want access rights to d:l:py:singlespec,
just send a request.

4. New policies for d:l:py
---
If the transition goes smoothly, I'd like to use the opportunity to
clean out the devel:languages:python project.

One, d:l:py is collecting applications that happen to be *written in*
Python, but have nothing to do with python development, and should
instead be placed in other topically appropriate projects. We've had
some discussions about dependencies only present in d:l:py, but here's a
policy proposal:
If your package depends on d:l:py and is not appropriate for d:l:py, you
can either push your dependencies to Factory, or link them to your
topically appropriate project.

Two, end-user applications should start to move to Python 3. This is
already happening. The relevance here is that we can now build
"dual-version" packages. This should be discouraged. There are packages
where that is appropriate (pip, nose and similar), but most packages
should only provide executables for python3. We have a great number of
packages that act as modules (= dependencies for other packages) and
also as command-line utilities.
Policy proposal: Unless we actually need the executable for all versions
of python, all executables should be %py3_only.

Finally, three, d:l:py is collecting non-python dependencies of python
modules. That should not be happening. Analogously to point one, policy
proposal: If your package depends on something that is not appropriate
for d:l:py, either get that dependency into Factory, or your package is
also not appropriate for d:l:py.

How about it?

5. Naming scheme
---
Finally some bikeshedding: names of RPM macros for python-related things
are inconsistent. We have %py_ver and %py_compile and %python_sitelib
and %ifpython2 and %py2_only.

I would like to keep "python" and "py" as *generic* versions of the
macros, not specifically referring to a particular version or flavor of
Python. So for every "py_" and "python_", equivalent "py2_" and
"python2_" should be defined.
This is partially true in the new macro set already.

Another thing is with "short" and "long" versions. I would really prefer
to use only one of them, and that's probably the short one. Keep %py_ver
and %py2_ver, but make %py2_sitelib etc.

The last thing is about executables for pypy. For instance, when
installing pip for python2 and python3, you get pip-2.7 and pip-3.5.
What should the pypy variant be called?

Thoughts, questions, comments?

thanks
m.


[1]
https://build.opensuse.org/project/show/devel:languages:python:singlespec
[2] https://github.com/openSUSE/multipython-macros

< Previous Next >
List Navigation