On 10.11.2016 14:02, Neal Gompa wrote:

Why not pull in the dependency generator I upstreamed into RPM[1] into openSUSE? Fedora is using it now in Fedora 25 and Rawhide[2][3], and Mageia is using an earlier variant of it now. It establishes common names for Python packages that provide egg-info or dist-info data (egg/wheel metadata installed on the system) that can be used for standardizing how Python dependencies are referenced.

this is certainly very useful and we're looking into it, but it doesn't seem to solve my question? to restate, I need to configure BuildRequires (not install-time requirements) and pull in, as automagically as possible, all rpm packages of "python module foo" for all supported pythons (that is python2 and python3 at the moment, pypy3 in the near future (hopefully)) also, BuildRequires are evaluated by OBS before rpmbuild starts, so I can't rely on the usual ways of injecting macros into the build environment. (i'd also be interested in what Fedora has to say about the single-spec thing, [1]. ISTM you don't do anything similar at the moment? but i suppose that's for a different thread altogether) [1] https://lists.opensuse.org/opensuse-packaging/2016-10/msg00031.html

On Thu, Nov 10, 2016 at 8:16 AM, Neal Gompa wrote:

On Thu, Nov 10, 2016 at 8:11 AM, jan matejek wrote:

...

On 10.11.2016 14:02, Neal Gompa wrote:

...

Why not pull in the dependency generator I upstreamed into RPM[1] into openSUSE? Fedora is using it now in Fedora 25 and Rawhide[2][3], and Mageia is using an earlier variant of it now. It establishes common names for Python packages that provide egg-info or dist-info data (egg/wheel metadata installed on the system) that can be used for standardizing how Python dependencies are referenced.

this is certainly very useful and we're looking into it, but it doesn't seem to solve my question?

to restate, I need to configure BuildRequires (not install-time requirements) and pull in, as automagically as possible, all rpm packages of "python module foo" for all supported pythons (that is python2 and python3 at the moment, pypy3 in the near future (hopefully))

Having the standardized names means you can just read requirements.txt or whatever and generate the list and expect it to work. It's a step that'd be required for that. OBS would likely need to grow the ability to run something like pyp2rpm spec generator before starting the build, which means being able to read requirements.txt or setup.py for the deps and resolving them properly.

...

also, BuildRequires are evaluated by OBS before rpmbuild starts, so I can't rely on the usual ways of injecting macros into the build environment.

(i'd also be interested in what Fedora has to say about the single-spec thing, [1]. ISTM you don't do anything similar at the moment? but i suppose that's for a different thread altogether)

[1] https://lists.opensuse.org/opensuse-packaging/2016-10/msg00031.html

As far as I know, openSUSE was the *only* distribution *not* doing single spec Python 2/3 packaging. Fedora[1] and Mageia[2] both do.

[1]: https://fedoraproject.org/wiki/Packaging:Python#Example_common_spec_file [2]: https://wiki.mageia.org/en/Python_policy#Samples

Also, Fedora and Mageia (and other related distributions) have a shared set of macros for Python packaging[1]. Unfortunately, openSUSE does not use these, so Python module packaging differs quite a bit between openSUSE and the other RPM based distributions I work on. [1]: http://pkgs.fedoraproject.org/cgit/rpms/python-rpm-macros.git/tree/ -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

On Thu, Nov 10, 2016 at 8:24 AM, jan matejek wrote:

On 10.11.2016 14:16, Neal Gompa wrote:

...

As far as I know, openSUSE was the *only* distribution *not* doing single spec Python 2/3 packaging. Fedora[1] and Mageia[2] both do.

and as far as I can tell, we're now the only ones to (try to) automate this as opposed to writing everything in the spec file explicitly ;)

Which, for instance, would allow us to support pypy across the board with little additional effort.

Maybe that's where pyp2rpm comes in at Fedora? Does it regenerate the spec for every build?

m.

Yes. We also have it hooked up to our COPR service to autogenerate them fairly regularly[1][2][3] as a way to help make pyp2rpm even more useful and reliable. [1]: https://copr.fedorainfracloud.org/coprs/g/copr/PyPi/ [2]: https://copr.fedorainfracloud.org/coprs/g/copr/PyPI2/ [3]: https://copr.fedorainfracloud.org/coprs/g/copr/PyPI3/ -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

On 10.11.2016 14:03, Andreas Schwab wrote:

On Nov 10 2016, jan matejek wrote:

...

Is it possible to make the buildservice itself recognize this macro, or place it in some sort of global prjconf?

Project configs are inherited across repo paths, so I'm not sure what you mean with global prjconf.

I'm not sure either, that's why I'm asking here ;) What macros are supported at buildrequires evaluation time? How are these configured? I know that only single-line expansion works, so there must be some sort of special handling. An option could be to put this into prjconf of Factory and other distros? So everyone building for Factory, directly or indirectly, would also inherit the macros? However, I'm told that changing prjconf of old distributions would trigger unwanted rebuilds and we definitely don't want that. Does it mean that something like %python_module can never be supported on old distros?

Andreas.

On 10.11.2016 14:27, Andreas Schwab wrote:

On Nov 10 2016, jan matejek wrote:

...

An option could be to put this into prjconf of Factory and other distros?

Why other ditros? This is surely a factory thing.

packages using the new macros are strictly incompatible with older distros. an updated package would only build for factory, unless you put a huge conditional that lists buildrequires by hand for older distros, which kinda defeats the point.

Andreas.

On 10.11.2016 15:44, Andreas Schwab wrote:

On Nov 10 2016, jan matejek wrote:

...

On 10.11.2016 14:27, Andreas Schwab wrote:

...

On Nov 10 2016, jan matejek wrote:

...

An option could be to put this into prjconf of Factory and other distros?

Why other ditros? This is surely a factory thing.

packages using the new macros are strictly incompatible with older distros.

What's your point then?

well, the primary reason for this incompatibility is the %python_module macro, which i would need to add to said older distros. the other macros can be solved in several ways, the most promising is making and buildrequiring a "python-rpm-macros" package, that can be either submitted to older distros, or copypac'd/linked to projects as required. crucially, this does not work for macros used in BuildRequires which is the whole point of this thread? regards m.

Andreas.

On 10.11.2016 16:15, Andreas Schwab wrote:

On Nov 10 2016, jan matejek wrote:

...

On 10.11.2016 15:44, Andreas Schwab wrote:

...

On Nov 10 2016, jan matejek wrote:

...

On 10.11.2016 14:27, Andreas Schwab wrote:

...

On Nov 10 2016, jan matejek wrote:

...

An option could be to put this into prjconf of Factory and other distros?

Why other ditros? This is surely a factory thing.

packages using the new macros are strictly incompatible with older distros.

What's your point then?

well, the primary reason for this incompatibility is the %python_module macro, which i would need to add to said older distros.

Why would that be needed? Only factory-based repositories will be using it, IIUC.

devel:languages:python itself builds for distros as old as SLE11. we might drop that soon, but still there's SLE12_backports, 13.2 and both Leaps, both of which are "old distros" for the purpose of this discussion, seeing as they're already released. So do many personal / topical / private repositories. We need to support all of that, otherwise nobody will switch to the new macros.

Andreas.

On 10.11.2016 16:45, Andreas Schwab wrote:

On Nov 10 2016, jan matejek wrote:

...

devel:languages:python itself builds for distros as old as SLE11. we might drop that soon, but still there's SLE12_backports, 13.2 and both Leaps, both of which are "old distros" for the purpose of this discussion, seeing as they're already released.

So do many personal / topical / private repositories.

We need to support all of that, otherwise nobody will switch to the new macros.

No problem, set up the project config of devel:languages:python as you like. That's your anchor.

But then everyone who wants to build a python package would need to fork from d:l:py. This is not the case in practice. m.

On Thu, Nov 10, 2016 at 11:18 AM, Andreas Schwab wrote:

On Nov 10 2016, jan matejek wrote:

...

On 10.11.2016 16:45, Andreas Schwab wrote:

...

On Nov 10 2016, jan matejek wrote:

...

devel:languages:python itself builds for distros as old as SLE11. we might drop that soon, but still there's SLE12_backports, 13.2 and both Leaps, both of which are "old distros" for the purpose of this discussion, seeing as they're already released.

So do many personal / topical / private repositories.

We need to support all of that, otherwise nobody will switch to the new macros.

No problem, set up the project config of devel:languages:python as you like. That's your anchor.

But then everyone who wants to build a python package would need to fork from d:l:py. This is not the case in practice.

Any project that wants the updated python-rpm-macros package will either have to build against d:l:py (or factory) or carry over the corresponding project config settings. That includes any :Update project.

Andreas.

What percentage of projects have a python package in them? Since these are _new_ macros, what would be the danger of adding them to the base distros: 13.1, 13.2, 42.1, 42.2 and factory. If there is a concern some projects already define them for a different purpose, couldn't OBS be scanned for existing macros with the same name and then eliminate the conflicts prior to instituting these new macros. I for one, would prefer to see these macros either global to OBS, or at least global to the openSUSE repositories. Greg -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

On Thu, Nov 10, 2016 at 4:31 PM, Simon Lees wrote:

On 11/11/2016 05:43 AM, Greg Freemyer wrote:

...

On Thu, Nov 10, 2016 at 11:18 AM, Andreas Schwab wrote:

...

On Nov 10 2016, jan matejek wrote:

...

On 10.11.2016 16:45, Andreas Schwab wrote:

...

On Nov 10 2016, jan matejek wrote:

...

devel:languages:python itself builds for distros as old as SLE11. we might drop that soon, but still there's SLE12_backports, 13.2 and both Leaps, both of which are "old distros" for the purpose of this discussion, seeing as they're already released.

So do many personal / topical / private repositories.

We need to support all of that, otherwise nobody will switch to the new macros.

No problem, set up the project config of devel:languages:python as you like. That's your anchor.

But then everyone who wants to build a python package would need to fork from d:l:py. This is not the case in practice.

Any project that wants the updated python-rpm-macros package will either have to build against d:l:py (or factory) or carry over the corresponding project config settings. That includes any :Update project.

Andreas.

What percentage of projects have a python package in them?

Since these are _new_ macros, what would be the danger of adding them to the base distros: 13.1, 13.2, 42.1, 42.2 and factory.

I think as stated previously, know one has mentioned a way of doing this without triggering a rebuild of these repo's which is undesirable and unwanted.

--

Simon Lees (Simotek)

Surely someone with backend access to the files could insert the change without triggering a full rebuild. Touch can backdate files. If the trigger is in a database hash compare, a direct update of the hash field could be done, etc. Obviously its a non-standard process, but I would be very surprised if it can't be done. Greg -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org