Hello packagers, this is a progress report on the singlespec project. I also have some things to discuss. These are way down in the e-mail, under a heading. If you don't want to read all of it, please at least scroll down to the "Remaining issues" section. 1. Changes from last time ------------------------- In the last month, I've made a number of significant improvements to the macro set. 1.1 Backwards compatibility This one is the biggest: we can now build new-style packages in old distributions without any prjconf changes. The trick consists of two parts: - placing the macro definitions in a separate python-rpm-macros package, which is then buildrequired. You can simply link the package to your project, or we might push it into SLE/Leap build base. - shimming the %python_module definition in the spec file. Since the main purpose of this macro is to avoid repetitive listing of buildrequires, we can conditionally define it to expand to "python-$1 python3-$1" -- or, if needed, just "python-$1" -- and this will do the necessary work. The d:l:py:singlespec repository now builds packages for all distros from 13.2 up. In principle, nothing is stopping us from supporting SLE11 too, but with all the other compatibility requirements, I consider it not worth the effort. This also means that you can now easily build singlespec packages in your own repository. Just link python-rpm-macros from d:l:py:Factory. 1.2 Macro naming Naming is now very much unified. Every macro name is prefixed with the "flavor" name, which is the name of the binary in /usr/bin. That means no short versions: %python2_version, %python3_version. If we ever add, say, Jython, it's going to get %jython3_version. Some short versions are retained as compatibility spellings, namely "%py_ver". I have, however, dropped the old macro set "%py_sitedir, %py_incdir" etc. I don't think they are used anywhere; if they are, we will find out soon enough. 1.3 Fedora compatibility I have based some macro definitions on Fedora [1]. I claim _partial_ compatibility: we don't have macros for wheels and eggs. They look simple enough that I'll probably add them. I also didn't copy SRPM macros, because I have no idea what they are good for. If some Fedora people want me to include them, please explain :) As stated above, the primary spelling of all macros is with long names: %py2_compile in SUSE will be an alias for %python2_compile. The latter is the preferred spelling. [1] http://pkgs.fedoraproject.org/cgit/rpms/python-rpm-macros.git/tree/ 1.4 Package naming You can name your package anything (usual scenario is "python-foo"). In older distros, the result will be python-foo, python3-foo (and possibly pypy3-foo). In newer distros (as soon as I submit the new Python 2 package that Provides "python2-*" symbols), the result will be python2-foo, python3-foo and an empty package python-foo. I don't think I can get rid of the empty package, but the python2-foo packages are created with obsoletes/provides on the name. In order to accomplish this trick, if your package is called "python-foo", you have to decorate all %files sections: "%files %python_files", or "%files %{python_files subpkg}". The build will fail if it detects an undecorated %files section. This is not necessary if your top-level package is called "python2-foo", "python3-foo" or anything else. 1.5 %pycache_only In addition to %ifpython2, %ifpython3 etc, I have added a %ifpycache section and a corresponding %pycache_only marker. This is equivalent to "if not python2", and is applicable for flavors that use the "__pycache__" directory to store .pyc files. 1.6 %python_expand This macro will repeat its arguments for every flavor, replacing "$python" with the flavor name. For example: %{python_expand cp command-%$python_bin_suffix %{buildroot}%{$python_sitelib}/foomodule/command} I'm still not sure if this is a good idea -- after all, it's introducing a minilanguage specific to Python packages. But it sure is useful. 2. Remaining issues ------------------- New-style packages, "python2-foo", rely on presence of prjconf-based definition of %python_modules. Before we can submit packages to Factory, we need to get the definitions there. This should pose no problem, however. In two packages now I have run into a situation where I want to create a subpackage that is _not_ automatically replicated. In the first case, python-pycparser has some data that can be shared between the versions; in the second, for python-Sphinx, i want to create a subpackage for the binaries that has a dependency only on the Python 3 version. Both cases can be solved in a different way, however, the need for non-replicated looks like an important feature. How to implement it? One idea is to not replicate any %package -n something. Subpackages of this type are currently buggy, changing the bugginess to straight out ignoring is easy enough :) And conversely, it is unclear how to replicate packages of this kind anyway, so declaring them to be non-replicating is sort of natural. Another option is to delineate packages by a macro, say, %python_noreplicate. What do you think? Requires, Suggests and Recommends are copied from the main package declaration and changed to match the python flavor. It is somewhat unclear what to do about Provides and Obsoletes. They should probably be handled similarly, except the autoconverter should wrap them in %python2_only by default. On a related note, the %ifpython2 and %python2_only macros were originally intended for use both in Requires and %files. However, with the new style packages, python-foo.spec generating python2-foo.rpm and empty python-foo.rpm, this will have some flaws. Namely the requires/provides will be the same in the empty python-foo as in the correct python2-foo. I foresee this causing problems: "have choice for python-provided-symbol: python-foo python2-foo". There is a number of solutions: 1. I can hack %python2_only to work properly, and invent an alternate spelling for %ifpython2. Maybe %ifpython2_requires? I can then detect whether the correct spelling is used and halt the build if it is not. I don't like this too much, for obvious reasons :) 2. I can detect said macros and halt build without alternative, forcing the packager to rename the whole package to python2-foo or python3-foo, which would sidestep the problem. 3. I can simply not care and let the empty package have the same obsoletes/provides. This would be great if we could cripple the empty package in some other way. Introduce an unsatisfiable requirement? "Requires: this-does-not-exist" or "Conflicts: aaa_base"? Would this help zypper resolve the choice? Is there a different option for saying "this package should always be preferred" from within the package? "Prefer: python2-foo" but within the spec? Unsatisfiable requirement for the empty package would be a good idea in any case, to make sure the empty packages don't sneak into the distribution and on people's computers. As usual, i'll be glad for all your input, answer questions, etc. etc. regards m.