On 10/28/2016 05:05 AM, Todd Rme wrote:

On Thu, Oct 27, 2016 at 12:51 PM, jan matejek wrote:

...

hello packagers,

i'm happy to announce that i have a macroset that works pretty well, if i say so myself :) Have a look at the d:l:python:singlespec repository [1] and the github repo [2] for details.

The gist of it: it is now very possible to build packages for both python 2 and python 3, using a pretty straightforward set of macros. See spec files in the linked repository, and documentation on github.

So far this only reliably works on python 2 and 3. It can work with pypy too, but pypy is not building so I can't test against it.

Some comments on the macros:

1. I think the version number should be explicit in all cases. So it should be "%have_python2", not "%have_python", and "%have_pypy2" and "%have_pyp3", not "%have_pypy". 2. Along those lines, I think pypy2 and pyp3 versions should be available, especially with a lot of progress now happening in pypy3 thanks to mozilla funding it.

...

4. New policies for d:l:py --- If the transition goes smoothly, I'd like to use the opportunity to clean out the devel:languages:python project.

One, d:l:py is collecting applications that happen to be *written in* Python, but have nothing to do with python development, and should instead be placed in other topically appropriate projects. We've had some discussions about dependencies only present in d:l:py, but here's a policy proposal:

When was this ever not the case? There are packages like that which haven't been updated in 4 or 5 years. I did some spot checks and the other devel:languages:___ repos also have a wide variety of software written in that language. What do you see as wrong with the current approach?

There is almost always a better repo that they could be in which is less confusing to users who care about what applications do rather then which language they live in. For example variety which is a cross desktop wallpaper fetcher and changer written in python lives in the X11:Utilites repo because its a Utility for X11 apps, similarly if the purpose of the application is to monitor servers we have a nice Server:Monitoring repo I believe. The reason its probably worth being part of this discussion is for almost all libraries should be built for python2 + python3 (Unless python2 is not supported upstream anymore). Where as applications should only be built for one or the other. As for applications requiring dependencies only in d:l:py the simple policy should be anything added to d:l:py should also then be forwarded to openSUSE:Factory where it will hopefully get picked up for the next Leap release as well. If this is enforced the required deps will always be available for all repos and it won't be an issue. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adeliade Australia, UTC+9:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 10/28/2016 08:01 AM, Todd Rme wrote:

On Thu, Oct 27, 2016 at 5:07 PM, Simon Lees wrote:

...

On 10/28/2016 05:05 AM, Todd Rme wrote:

...

On Thu, Oct 27, 2016 at 12:51 PM, jan matejek wrote:

...

hello packagers,

i'm happy to announce that i have a macroset that works pretty well, if i say so myself :) Have a look at the d:l:python:singlespec repository [1] and the github repo [2] for details.

The gist of it: it is now very possible to build packages for both python 2 and python 3, using a pretty straightforward set of macros. See spec files in the linked repository, and documentation on github.

So far this only reliably works on python 2 and 3. It can work with pypy too, but pypy is not building so I can't test against it.

Some comments on the macros:

1. I think the version number should be explicit in all cases. So it should be "%have_python2", not "%have_python", and "%have_pypy2" and "%have_pyp3", not "%have_pypy". 2. Along those lines, I think pypy2 and pyp3 versions should be available, especially with a lot of progress now happening in pypy3 thanks to mozilla funding it.

...

4. New policies for d:l:py --- If the transition goes smoothly, I'd like to use the opportunity to clean out the devel:languages:python project.

One, d:l:py is collecting applications that happen to be *written in* Python, but have nothing to do with python development, and should instead be placed in other topically appropriate projects. We've had some discussions about dependencies only present in d:l:py, but here's a policy proposal:

When was this ever not the case? There are packages like that which haven't been updated in 4 or 5 years. I did some spot checks and the other devel:languages:___ repos also have a wide variety of software written in that language. What do you see as wrong with the current approach?

There is almost always a better repo that they could be in which is less confusing to users who care about what applications do rather then which language they live in. For example variety which is a cross desktop wallpaper fetcher and changer written in python lives in the X11:Utilites repo because its a Utility for X11 apps, similarly if the purpose of the application is to monitor servers we have a nice Server:Monitoring repo I believe.

The problem is that the distinction between "application" and "library" is not very clear-cut with most python packages. In the rare case where something is purely an application that happens to be written in Python, then yes I suppose you are right. But more often the "application" contains both an executable and a python module that python scripts can make use of. What should we do in that situation?

For the pure applications, a bunch of those are python shells, python IDEs, debuggers, or other development tools that are connected in some way to the python version they were built with and thus need to have both versions available.

These are probably both valid examples of things that are not libraries that could stay.

Do you have any specific examples of python-based packages in d:l:p or d:l:p3 that you think shouldn't be there?

No I personally have not looked I was simply agreeing with the above quoted statement "One, d:l:py is collecting applications that happen to be *written in* Python, but have nothing to do with python development" and agreeing that they should be moved to more appropriate locations

...

As for applications requiring dependencies only in d:l:py the simple policy should be anything added to d:l:py should also then be forwarded to openSUSE:Factory where it will hopefully get picked up for the next Leap release as well. If this is enforced the required deps will always be available for all repos and it won't be an issue.

Are you talking about all packages or only non-python packages? Because there are a lot of niche python tools in d:l:p that I don't think belong in openSUSE:Factory.

All packages, unless they are of no interest to anyone even those doing python development. Users shouldn't need to add development repo's to there systems especially ones like d:l:p where there is a high chance that they can completely break there system by updating with it enabled. So if its useful to any openSUSE users it should go in the distro if its not there is probably a question of does it actually need to be there still? Richard has been pushing this view far more then I but its a view I agree with (you can watch his openSUSE conference talk). -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adeliade Australia, UTC+9:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 10/28/2016 01:55 PM, Todd Rme wrote:

On Thu, Oct 27, 2016 at 7:35 PM, Simon Lees wrote:

...

On 10/28/2016 08:01 AM, Todd Rme wrote:

...

On Thu, Oct 27, 2016 at 5:07 PM, Simon Lees wrote:

...

On 10/28/2016 05:05 AM, Todd Rme wrote:

...

On Thu, Oct 27, 2016 at 12:51 PM, jan matejek wrote:

...

hello packagers,

i'm happy to announce that i have a macroset that works pretty well, if i say so myself :) Have a look at the d:l:python:singlespec repository [1] and the github repo [2] for details.

The gist of it: it is now very possible to build packages for both python 2 and python 3, using a pretty straightforward set of macros. See spec files in the linked repository, and documentation on github.

So far this only reliably works on python 2 and 3. It can work with pypy too, but pypy is not building so I can't test against it.

Some comments on the macros:

1. I think the version number should be explicit in all cases. So it should be "%have_python2", not "%have_python", and "%have_pypy2" and "%have_pyp3", not "%have_pypy". 2. Along those lines, I think pypy2 and pyp3 versions should be available, especially with a lot of progress now happening in pypy3 thanks to mozilla funding it.

...

4. New policies for d:l:py --- If the transition goes smoothly, I'd like to use the opportunity to clean out the devel:languages:python project.

One, d:l:py is collecting applications that happen to be *written in* Python, but have nothing to do with python development, and should instead be placed in other topically appropriate projects. We've had some discussions about dependencies only present in d:l:py, but here's a policy proposal:

When was this ever not the case? There are packages like that which haven't been updated in 4 or 5 years. I did some spot checks and the other devel:languages:___ repos also have a wide variety of software written in that language. What do you see as wrong with the current approach?

There is almost always a better repo that they could be in which is less confusing to users who care about what applications do rather then which language they live in. For example variety which is a cross desktop wallpaper fetcher and changer written in python lives in the X11:Utilites repo because its a Utility for X11 apps, similarly if the purpose of the application is to monitor servers we have a nice Server:Monitoring repo I believe.

The problem is that the distinction between "application" and "library" is not very clear-cut with most python packages. In the rare case where something is purely an application that happens to be written in Python, then yes I suppose you are right. But more often the "application" contains both an executable and a python module that python scripts can make use of. What should we do in that situation?

For the pure applications, a bunch of those are python shells, python IDEs, debuggers, or other development tools that are connected in some way to the python version they were built with and thus need to have both versions available.

These are probably both valid examples of things that are not libraries that could stay.

That is almost all, if not all, of the "applications" in d:l:p.

...

...

Do you have any specific examples of python-based packages in d:l:p or d:l:p3 that you think shouldn't be there?

No I personally have not looked I was simply agreeing with the above quoted statement "One, d:l:py is collecting applications that happen to be *written in* Python, but have nothing to do with python development" and agreeing that they should be moved to more appropriate locations

My point is that this tends to be a very blurry line with python. We need some more specific rules to actually implement this sort of policy.

...

...

...

As for applications requiring dependencies only in d:l:py the simple policy should be anything added to d:l:py should also then be forwarded to openSUSE:Factory where it will hopefully get picked up for the next Leap release as well. If this is enforced the required deps will always be available for all repos and it won't be an issue.

Are you talking about all packages or only non-python packages? Because there are a lot of niche python tools in d:l:p that I don't think belong in openSUSE:Factory.

All packages, unless they are of no interest to anyone even those doing python development. Users shouldn't need to add development repo's to there systems especially ones like d:l:p where there is a high chance that they can completely break there system by updating with it enabled. So if its useful to any openSUSE users it should go in the distro if its not there is probably a question of does it actually need to be there still? Richard has been pushing this view far more then I but its a view I agree with (you can watch his openSUSE conference talk).

There are certainly people who hold this view, but I have seen nothing to suggest it is remotely close to a consensus view. I personally cannot emphasize how strongly I disagree with this view, and how much I think it would harm projects like d:l:p. If it becomes official openSUSE policy, then certainly d:l:p will have to follow it. But I am strongly opposed d:l:p implementing this policy on its own. Especially since it would involve eliminating hundreds of useful packages that our volunteers have been willing to package but aren't willing to go through the often months-long process of getting accepted into openSUSE:Factory. We simply don't have the manpower to get every useful package into openSUSE:Factory, even if they all belonged there (and I think many of them don't belong there).

While the legal review can take up to a month, although from personal experience its gotten better of late (Even with the haskell development project adding 1500 packages at once), It's not like its a month of hard effort required. If something is packaged reasonably well to start with and according to openSUSE's guidelines the amount of work required to get it included is pretty minimal and any issues should be resolved within 1 or 2 submit requests. If people would like help with this I and others are willing to spend time helping them fix there issues. Personally I think that packages in openSUSE development repo's should be following openSUSE guidelines and packages that are not willing to do that should be in some other sub repository like d:l:p:Playground but i'm guessing others would disagree.

And I don't think there is "a high chance that they can completely break there system by updating with it enabled", as long as we fix the issue with the linked libraries that was already brought up. System-critical python-based packages are not handled in that repo (if they are they will be moved under the new policy), nor is the packaging of python itself (that happens in an isolated sub-project).

That still doesn't rule out some users hitting some weird breakages at times (I've been helping with support on irc for at least 5 years and have seen some pretty weird breakages due to extra repo's in my time). Having said that i'm not doing most of the work, although i'm happy to review packages people push to factory before they do so what I say doesn't count that much. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adeliade Australia, UTC+9:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 10/28/2016 10:59 PM, Todd Rme wrote:

On Fri, Oct 28, 2016 at 12:22 AM, Simon Lees wrote:

...

On 10/28/2016 01:55 PM, Todd Rme wrote:

...

On Thu, Oct 27, 2016 at 7:35 PM, Simon Lees wrote:

...

On 10/28/2016 08:01 AM, Todd Rme wrote:

...

On Thu, Oct 27, 2016 at 5:07 PM, Simon Lees wrote:

...

On 10/28/2016 05:05 AM, Todd Rme wrote: As for applications requiring dependencies only in d:l:py the simple policy should be anything added to d:l:py should also then be forwarded to openSUSE:Factory where it will hopefully get picked up for the next Leap release as well. If this is enforced the required deps will always be available for all repos and it won't be an issue.

Are you talking about all packages or only non-python packages? Because there are a lot of niche python tools in d:l:p that I don't think belong in openSUSE:Factory.

All packages, unless they are of no interest to anyone even those doing python development. Users shouldn't need to add development repo's to there systems especially ones like d:l:p where there is a high chance that they can completely break there system by updating with it enabled. So if its useful to any openSUSE users it should go in the distro if its not there is probably a question of does it actually need to be there still? Richard has been pushing this view far more then I but its a view I agree with (you can watch his openSUSE conference talk).

There are certainly people who hold this view, but I have seen nothing to suggest it is remotely close to a consensus view. I personally cannot emphasize how strongly I disagree with this view, and how much I think it would harm projects like d:l:p. If it becomes official openSUSE policy, then certainly d:l:p will have to follow it. But I am strongly opposed d:l:p implementing this policy on its own. Especially since it would involve eliminating hundreds of useful packages that our volunteers have been willing to package but aren't willing to go through the often months-long process of getting accepted into openSUSE:Factory. We simply don't have the manpower to get every useful package into openSUSE:Factory, even if they all belonged there (and I think many of them don't belong there).

While the legal review can take up to a month, although from personal experience its gotten better of late (Even with the haskell development project adding 1500 packages at once) It's not like its a month of hard effort required. If something is packaged reasonably well to start with and according to openSUSE's guidelines the amount of work required to get it included is pretty minimal and any issues should be resolved within 1 or 2 submit requests. If people would like help with this I and others are willing to spend time helping them fix there issues. Personally I think that packages in openSUSE development repo's should be following openSUSE guidelines and packages that are not willing to do that should be in some other sub repository like d:l:p:Playground but i'm guessing others would disagree.

I disagree with most of what you said, but this can be discussed further in the Factory mailing list. What you are suggesting is a fundamental change to the most basic aspects of how OSC is organized. I am strongly against using d:l:p as a testing ground for such a change.

Given that most of the Desktop environments including (KDE/Gnome/Enlightenment) and other repo's such as haskell and probably others that I don't know about are already doing this so d:l:p would hardly be the first or a testing ground.

Even if I liked the policy, the repo is undergoing too many other changes right now. We just don't have the manpower to additionally sift through probably somewhere around 1000 packages, figure out which one should go into factory, clean them all up, wade through the submission process, do whatever it is you propose we do with those that don't make the cut. It is probably going to take months to just get the scripts implemented, and that process is mostly automated.

Again i'm not suggesting that 1000 packages should be fixed right now at once, but if you were to adopt this policy now would be a good time to consider it given that presumably you need to go through each package to add these new macro's anyway, while doing that you could decide 1. This spec file looks in pretty good shape I may as well put in a request to add it to factory 2. This package is a long way from meeting openSUSE packaging guidelines I probably should document that in the description so users are aware and someone can come back to do the work to get it up to standard later if they desire if not its labeled and we may choose to move it to another repo later. 3. This package is old and not maintained upstream its not worth migrating it to the new macro's i'll disable the build in factory and we will slowly phase it out. None of that is much extra effort on top of what already needs to be done. I have a list of packages I use from pip that I will add to openSUSE:Factory at some point, I haven't even got to checking if there in d:l:p yet because for me using pip is far easier and less complicated then trying to enable another repo for 1-2 packages. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adeliade Australia, UTC+9:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 10/29/2016 12:15 PM, Todd Rme wrote:

On Fri, Oct 28, 2016 at 9:31 PM, Simon Lees wrote:

...

On 10/28/2016 10:59 PM, Todd Rme wrote:

...

On Fri, Oct 28, 2016 at 12:22 AM, Simon Lees wrote:

...

On 10/28/2016 01:55 PM, Todd Rme wrote:

...

On Thu, Oct 27, 2016 at 7:35 PM, Simon Lees wrote:

...

On 10/28/2016 08:01 AM, Todd Rme wrote: > On Thu, Oct 27, 2016 at 5:07 PM, Simon Lees wrote: >> >> >> On 10/28/2016 05:05 AM, Todd Rme wrote: >> As for applications requiring dependencies only in d:l:py the simple >> policy should be anything added to d:l:py should also then be forwarded >> to openSUSE:Factory where it will hopefully get picked up for the next >> Leap release as well. If this is enforced the required deps will always >> be available for all repos and it won't be an issue. > > Are you talking about all packages or only non-python packages? > Because there are a lot of niche python tools in d:l:p that I don't > think belong in openSUSE:Factory. > All packages, unless they are of no interest to anyone even those doing python development. Users shouldn't need to add development repo's to there systems especially ones like d:l:p where there is a high chance that they can completely break there system by updating with it enabled. So if its useful to any openSUSE users it should go in the distro if its not there is probably a question of does it actually need to be there still? Richard has been pushing this view far more then I but its a view I agree with (you can watch his openSUSE conference talk).

There are certainly people who hold this view, but I have seen nothing to suggest it is remotely close to a consensus view. I personally cannot emphasize how strongly I disagree with this view, and how much I think it would harm projects like d:l:p. If it becomes official openSUSE policy, then certainly d:l:p will have to follow it. But I am strongly opposed d:l:p implementing this policy on its own. Especially since it would involve eliminating hundreds of useful packages that our volunteers have been willing to package but aren't willing to go through the often months-long process of getting accepted into openSUSE:Factory. We simply don't have the manpower to get every useful package into openSUSE:Factory, even if they all belonged there (and I think many of them don't belong there).

While the legal review can take up to a month, although from personal experience its gotten better of late (Even with the haskell development project adding 1500 packages at once) It's not like its a month of hard effort required. If something is packaged reasonably well to start with and according to openSUSE's guidelines the amount of work required to get it included is pretty minimal and any issues should be resolved within 1 or 2 submit requests. If people would like help with this I and others are willing to spend time helping them fix there issues. Personally I think that packages in openSUSE development repo's should be following openSUSE guidelines and packages that are not willing to do that should be in some other sub repository like d:l:p:Playground but i'm guessing others would disagree.

I disagree with most of what you said, but this can be discussed further in the Factory mailing list. What you are suggesting is a fundamental change to the most basic aspects of how OSC is organized. I am strongly against using d:l:p as a testing ground for such a change.

Given that most of the Desktop environments including (KDE/Gnome/Enlightenment) and other repo's such as haskell and probably others that I don't know about are already doing this so d:l:p would hardly be the first or a testing ground.

Again, discussing major changes to the basic structure of OSC is off-topic for this thread. This is just about the changes needed for python packages, not for a complete reorganization of OSC.

This is not anything to do with a complete reorganization of osc it is only discussing organising d:l:p (which is getting reshuffles anyway) with a aim to getting more of its contents into the "openSUSE Distribution" as a whole given this is the openSUSE packaging list discussing packaging of the openSUSE Distro it seems appropriate.

I would strongly prefer not to further derail this thread with this discussion. There are many reasons I don't like this policy, but they are way off topic, and getting into a lengthy discussion about them distracts from all the work that has to get done related to the actual topic of discussion. So if you wish to discuss this, I ask you again to please create a new thread.

Well then lets agree to disagree, I don't think I have anything more to add on this context anyway. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adeliade Australia, UTC+9:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On Fri, Oct 28, 2016 at 9:30 AM, Greg Freemyer wrote:

I have numerous packages (20?) I maintain that are c libraries at their core, but have python binding sub-packages. I'm slowly making them have both python- and python3- sub-packages.

They currently live in security:forensics (and factory).

Will they be able to leverage the proposed new macros? Or do they have to be moved to d:l:py?

That is one of the questions we are trying to answer. Ideally I would prefer the macros would apply to all of OSC, and Jan has said this as well, but AFAIK we haven't had anyone with any authority on the matter weigh in on the issue.

Is it current policy they should be there already?

I do not think so. From what I have seen non-python packages that have Python bindings follow the same rules as any other non-python package: they are only there (or at least should only be there) to allow python packages for older distros to build. The fact that they happen to provide python bindings is irrelevant.

I also have 4 or 5 dual python app / python library packages (python-dfVFS, etc.) and one pure python app (python-plaso).

My preference is to leave all of the above in security:forensics.

python-plaso still provides usable python modules, and according to the documentation these modules are intended for direct use [1], so I would say it is also a dual python app / library. I can't speak for anyone else, but I would think they should stay where they are, but you could (but don't need to) have them linked in d:l:p as long as they don't have any dependency issues and you think they might be interesting to people outside of your project. From my perspective, something like plaso that is used to extract file information can be useful in a wide variety of situations besides forensics (such as sorting directories or searching), so I would recommend linking it. But the decision is up to you, I am not aware of any policy requiring that all python packages always have to live in d:l:p. [1] https://github.com/log2timeline/plaso/wiki/Users-Guide#the-tools -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

Hello, Am Donnerstag, 27. Oktober 2016, 14:35:11 CEST schrieb Todd Rme:

On Thu, Oct 27, 2016 at 12:51 PM, jan matejek wrote:

...

Finally, three, d:l:py is collecting non-python dependencies of python modules. That should not be happening. Analogously to point one, policy proposal: If your package depends on something that is not appropriate for d:l:py, either get that dependency into Factory, or your package is also not appropriate for d:l:py.

A lot of these are backports for older versions of openSUSE that either have outdated versions of the library or lack the library entirely. So yes, I agree if it really has non-Factory dependencies it probably shouldn't be there. But I think the bigger problem is that many of these are (or were) needed but are building for openSUSE versions where they aren't really needed, or where the version they were needed for is no longer supported. A cleanup of these is certainly warranted. They should also probably be renamed (package, not .spec file) to include the version of openSUSE they are needed for so we can clean them up more easily later.

An (IMHO) easier way to handle this is: - disable building those packages for factory, but keep the build for released versions (42.x, 13.x) enabled - if you know that for example 42.1 doesn't need this package from d:l:py, feel free to also disable the build for 42.1 - whenever a release goes EOL, no longer build for this release - when a package has no remaining build targets (see the "Monitor" page), drop it I'm using this method in my home repo (for backporting some packages that are missing in older releases) and it works fine. I never tested this in a repo with a large number of packages, but I'd expect it to be useable there also (it's guaranteed to work on the technical side, the only question is if it is maintainer-friendly).

1. The layout of the repositories. With single spec files, does "devel:languages:python3" even make sense anymore? I think we should phase it out.

Agreed.

2. Package naming. Might this be a chance to switch from using python-foo to python2-foo?

That sounds like lots of (IMHO superfluous) package renames ;-)

Either way, I would strongly suggest pypy packages using pypy2-foo and pypy3-foo from the beginning, rather than pypy-foo.

No objections. Regards, Christian Boltz -- Opensuse-Factory is mainly for systemd infights and KDE3 legacy maintainence questions nowadays. I guess you chose the right list ;-) [Ralf Lang in opensuse-programming] -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 10/27/2016 10:51 PM, Todd Rme wrote:

On Thu, Oct 27, 2016 at 6:24 PM, Christian Boltz wrote:

...

Hello,

Am Donnerstag, 27. Oktober 2016, 14:35:11 CEST schrieb Todd Rme:

...

2. Package naming. Might this be a chance to switch from using python-foo to python2-foo?

That sounds like lots of (IMHO superfluous) package renames ;-)

I don't think there will be lots of renaming. The names of the binary packages should be handled by the macros, and the number of Python-2 only packages should be shrinking, more below.

The idea would be that the spec file would be named "python-foo.spec", while the python2 subpackage created by the macros would be automatically named "python2-foo" instead of "python-foo" as they now. So the macros would handle the naming, and that is the case no matter what naming scheme we choose. Only packages that aren't compatible with the macros would need any additional work.

Agreed the macros should produce a python2-foo binary package. But also we need to consider that there are probably still some packages that are Python-2 only, although the number is getting smaller. In any event I think we need to account for the following: - packages that only build for Python-2 - packages that only build for Python-3 So my proposal for package names, meaning the names seen when looking at d:l:p in the web UI: - python2-foo -> a package that produces only a python2-foo binary package - python-foo -> a package that may produce a python-foo and python2-foo binary package or a python-foo package only Later, Robert -- Robert Schweikert MAY THE SOURCE BE WITH YOU Public Cloud Architect LINUX rjschwei@suse.com IRC: robjo

On 10/29/2016 12:04 PM, Robert Schweikert wrote:

So let me revise this proposal and expand on it just a little bit.

So my proposal for package names, meaning the names seen when looking at d:l:p in the web UI:

- python2-foo -> a package that produces only a python2-foo binary package - python-foo -> a package that may produce a python3-foo and python2-foo binary package or a python3-foo package only

And we would drop python-foo as binary package names. We could then add Provides: python-foo to the spec files, via macro. As long as Python-2 is the default interpreter the python2-foo packages provide python-foo and when Python-3 becomes the default interpreter than the python3-foo packages will provide python-foo. This was it is all the same for the end user:

zypper in python-foo

will provide the package that matches the default interpreter for the distribution. I like this idea.

And if there's no python3 version of the package, then python-foo is provided by python3-foo or not? Sebastian -- python programming - mail server - photo - video - https://sebix.at cryptographic key at https://sebix.at/DC9B463B.asc and on public keyservers

On 29.10.2016 11:57, Robert Schweikert wrote:

Making the package names explicit with respect to the version would get people used to a disassociation of the command line name and the associated packages. Today, with the Python-2 interpreter being the default one uses "python" on the command line and installs modules with binary packages called "python-foo". Thus there is a nice match in names. However, at some point in the future the "python" command will start the Python-3 interpreter and then all of a sudden one has to install "python3-foo" to use the module. So my thought is, if we break this association now, while Python-2 is still the default, then the change when Python-3 becomes the default is not going to be as traumatic as people will already be used to not having the "python" command and the binary package names be associated by name.

At this point it seems realistic to aim to switch the default python to Python 3 in SLE 13. For that, we would need to rename "python" to "python2", and keep "python" just as a provided symbol. This might also be an opportunity to move to "python2-foo" + "python3-foo" and make "python-foo" just a provided symbol too. This way the association is never broken. (Please note that at this point there is no concrete plan for the switch. It's just what seems realistic to me personally.)

I agree I don't think we should start to evict packages from d:l:p based on some nebulous arbitrary policy decision. If an application is written in Python it should be able to live in d:l:p. Chances are the application provides some modules that are useful for other purposes and the maintenance burden is reduced on the maintainer.

OTOH, if you want to add a repository for, say, all geodesics packages, you shouldn't need to add Application:Geo ... AND d:l:py and d:l:ruby and d:l:haskell and d:l:javascript and d:l:intercal, just because some of your geodesics apps happen to be written in a particular language. Also, adding the package to, say, Application:Geo, the maintenance burden is moved from d:l:py maintainers to App:Geo maintainers. This even makes more sense in that App:Geo maintainers probably care more about keeping a "core" package up to date, as opposed to d:l:py maintainers for whom a random app is not considered core. The "provided modules" thing is also nebulous. For me, a good rule of thumb is this: if the application does not have an API documentation (or if the API is merely a way to invoke the utility without using system() ), it is probably not useful as a library. And conversely, if there *is* an extensive and documented API, it's probably a good Python library that happens to have an app bundled, and I am happy about having it in d:l:py ;)

If we evict the application from d:l:p and make it live in d:myapp then the maintainer of myapp has a good chance of having to back link a rather large number

That's why these packages should be submitted to Factory in the first place.

of packages from d:l:p in addition to having to deal with prjconfig, repo setup etc. which are things that are currently handled by the d:l:p project maintainers.

which also should not be necessary in general. If it is, the solution should be to make sure you can build a package anywhere without hassle, not to collect it in a project that did the work. Of course, the latter solution is more practical. But while we're talking about big changes, I'm aiming for the clean approach. regards m.

Later, Robert

hello, On 27.10.2016 20:35, Todd Rme wrote:

2. Along those lines, I think pypy2 and pyp3 versions should be available, especially with a lot of progress now happening in pypy3 thanks to mozilla funding it.

I would very much prefer to NOT support pypy 2 at all, now that pypy3 is making significant progress. Given that we don't really support pypy2 at this point, I don't see any benefits in starting. That's why my proposal uses unversioned "pypy". That is not going to be future-proof if we ever get pypy 4, but, well, I say we take that risk ;)

I would personally keep dlp as a generic place for all python packages. This has worked very well up to this point and I don't see any advantage to changing it.

I have always considered d:l:py to be a generic place for Python modules and tools -- that is, for software where Python is a fundamental property, as opposed to implementation detail. From your conversation with Simon, it doesn't seem like you disagree on this, but you're concerned about the "edge case" packages -- where it's unclear whether the "library" or the "utility" part is more important. Let me clarify that I don't propose to throw these out summarily. I'm more concerned about the actually-clear-cut apps. I don't know how many of those are in d:l:py now, but I have rejected several in the last year, and I'd like to make it clearer that these don't belong. (Also, being more careful about the edge cases would not hurt, IMHO ;) )

I think the rule should be that if the executable is intended for use with python files or packages, it should provide both python 2 and python 3 versions when available.

to this I would add, "and if the executable doesn't natively support multiple python versions". For instance, if pylint3 could work on python 2 by itself, I would include it in the rule and say that we will only distribute pylint3. This could very well be the case for IDEs and similar.

...

Finally, three, d:l:py is collecting non-python dependencies of python modules. That should not be happening. Analogously to point one, policy proposal: If your package depends on something that is not appropriate for d:l:py, either get that dependency into Factory, or your package is also not appropriate for d:l:py.

A lot of these are backports for older versions of openSUSE that either have outdated versions of the library or lack the library entirely. So yes, I agree if it really has non-Factory dependencies it probably shouldn't be there. But I think the bigger problem is that many of these are (or were) needed but are building for openSUSE

It doesn't make sense to me to build a binding for, say, 13.2, for a library that is not present or too old in 13.2. In such case, I think the right solution is to disable build for the older SUSE version -- and optionally link the binding to a project that provides the newer library for the older distribution.

1. The layout of the repositories. With single spec files, does "devel:languages:python3" even make sense anymore? I think we should phase it out.

Definitely. That is one of the goals.

2. Package naming. Might this be a chance to switch from using python-foo to python2-foo? Either way, I would strongly suggest pypy packages using pypy2-foo and pypy3-foo from the beginning, rather than pypy-foo.

Probably not, for technical reasons. The problem is that the macroset can't modify what's already explicitly written in the spec file. So if the spec is for python-foo, and has a %files section, then we MUST build package python-foo that includes the files listed. We could of course rename the package and spec, but, as Christian pointed out, that is a lot of renames for no clear benefit. What makes more sense is to gradually rename packages to python3-foo, and generate python2-foo subpackages (with Provides: python-foo, which could in time move to the python3 subpackage). The macroset is designed with this possibility in mind, although it's probably buggy at the moment. (as for pypy, as I wrote before, i would prefer to never even use versioned names for it)

3. Version-specific packages. There are still a lot of packages that only support python 2, and an increasing number that only support python 3. Should these use the macros or be kept simple?

python2-only, I would consider legacy and keep simple, until such a time when they move to python 3 python3-only, I'd use the macros and "%define have_python2 0". After all, these could work on pypy or other flavors that might be created in the future.

4. Repository cleanup. There are packages that haven't been updated in many years and are not under active development. Who knows what sort of security or compatibility issues they might have. It might be worthwhile to move them to a separate repository (perhaps d:l:p:archive or d:l:p:legacy) with lower build priority, or delete them outright, to free up resources for packages that are better-maintained. 5. Disabled packages. Packages that are completely disabled should almost certainly be deleted.

good ideas regards m.

On Tuesday 2016-11-01 14:51, Robert Schweikert wrote:

You can look at it this way, currently we have stuff with "python" and "python3" and we all know python really means python2. But at some point in the future python3 will become the default, does that mean that python3 turns into python and what is now python turns into python2. If that is the case the confusion and ripples will be big [...] All of this can be avoided by simply adding the version number now when we go through a transition

The Python people recognized with the python3 rollout that you cannot have huge incompatible generations anymore without harming the project. So I would like to believe in my vision that it won't be necessary to have multiple concurrent python versions in the future -- just like perl5 has demonstrated for some 10+ years in (open)SUSE. [That's something the Ruby devs should look into as well, hint hint.] -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On Mon, Oct 31, 2016 at 11:28 AM, jan matejek wrote:

hello,

On 27.10.2016 20:35, Todd Rme wrote:

...

2. Along those lines, I think pypy2 and pyp3 versions should be available, especially with a lot of progress now happening in pypy3 thanks to mozilla funding it.

I would very much prefer to NOT support pypy 2 at all, now that pypy3 is making significant progress. Given that we don't really support pypy2 at this point, I don't see any benefits in starting.

Fair enough, but I still think keeping things consistent would be preferable. So even if there is no "pypy2" commands, I think we should still call iot "pypy3" to maintain the same pattern.

That's why my proposal uses unversioned "pypy". That is not going to be future-proof if we ever get pypy 4, but, well, I say we take that risk ;)

We are getting about 1 python minor version a year, with 3.6 coming out any time now. Guido has said he that python 4 will just be python 3.10 because he doesn't like double-digit versions. This means we are probably about 4 years away from Python 4. So I would include the python 4 transition in any plans we make, it isn't far off at all. In fact it will probably happen about the same time as 2.7 is EOL.

...

I would personally keep dlp as a generic place for all python packages. This has worked very well up to this point and I don't see any advantage to changing it.

I have always considered d:l:py to be a generic place for Python modules and tools -- that is, for software where Python is a fundamental property, as opposed to implementation detail. From your conversation with Simon, it doesn't seem like you disagree on this, but you're concerned about the "edge case" packages -- where it's unclear whether the "library" or the "utility" part is more important.

Let me clarify that I don't propose to throw these out summarily. I'm more concerned about the actually-clear-cut apps. I don't know how many of those are in d:l:py now, but I have rejected several in the last year, and I'd like to make it clearer that these don't belong. (Also, being more careful about the edge cases would not hurt, IMHO ;) )

Agreed. My point is that we need to be very clear about this policy. The current python application naming policy is pretty fuzzy and I don't think it handles these "edge" cases well (scare quotes because they are by far the most common case).

...

I think the rule should be that if the executable is intended for use with python files or packages, it should provide both python 2 and python 3 versions when available.

to this I would add, "and if the executable doesn't natively support multiple python versions". For instance, if pylint3 could work on python 2 by itself, I would include it in the rule and say that we will only distribute pylint3. This could very well be the case for IDEs and similar.

Agreed. Although IDEs are a problem because the python shell they support is usually the same as they python version they were built against.

...

...

Finally, three, d:l:py is collecting non-python dependencies of python modules. That should not be happening. Analogously to point one, policy proposal: If your package depends on something that is not appropriate for d:l:py, either get that dependency into Factory, or your package is also not appropriate for d:l:py.

A lot of these are backports for older versions of openSUSE that either have outdated versions of the library or lack the library entirely. So yes, I agree if it really has non-Factory dependencies it probably shouldn't be there. But I think the bigger problem is that many of these are (or were) needed but are building for openSUSE

It doesn't make sense to me to build a binding for, say, 13.2, for a library that is not present or too old in 13.2. In such case, I think the right solution is to disable build for the older SUSE version -- and optionally link the binding to a project that provides the newer library for the older distribution.

What about, say, a general-purpose numerical library that needs some C library to do some task? What about an update to a key existing program that gets a new indirect C dependency?

...

2. Package naming. Might this be a chance to switch from using python-foo to python2-foo? Either way, I would strongly suggest pypy packages using pypy2-foo and pypy3-foo from the beginning, rather than pypy-foo.

Probably not, for technical reasons. The problem is that the macroset can't modify what's already explicitly written in the spec file. So if the spec is for python-foo, and has a %files section, then we MUST build package python-foo that includes the files listed.

Can't you have the macros based off, say "%files -n python%{default_py_ver}-foo" or something like that and have no %files section? I would very much prefer no version number in the .spec file. With python 4 likely very close, I think it would be better to be prepared for this now rather than having to do this all over again before too much longer.

...

3. Version-specific packages. There are still a lot of packages that only support python 2, and an increasing number that only support python 3. Should these use the macros or be kept simple?

python2-only, I would consider legacy and keep simple, until such a time when they move to python 3

python3-only, I'd use the macros and "%define have_python2 0". After all, these could work on pypy or other flavors that might be created in the future.

Sound reasonable. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 8.11.2016 16:40, Todd Rme wrote:

I don't think a blanket ban on non-python packages will work. I think we need to look at how the packages are used and what needs them on a case-by-case basis. I think there are a variety of scenarios that need different approaches.

Thank you, this (and the list of packages) is wonderful work. I'm about to review and delete and/or file deleterequests for most packages on your list in the first three sections

The fifth packages with a low chance of conflicts whose library is required by very important python packages. In cases where a newer version is needed, these should have a minimal version copied (not linked) into d:l:p, have the -devel package be renamed from "foo-devel" to "foo-sonum-devel" (where "sonum" is the so number), and have the OSC package name renamed from "foo" to "foo-sonom-osver" (where osver is the openSUSE version).

This doesn't look like a good idea. It is going to introduce dependencies on "non-Factory" symbols, or complicate spec files (%if suse_version == 1320; Requires: foo-1320-devel; %else; Requires: foo-devel; %endif) Renaming also does not help if there are file conflicts between the renamed and the distro-provided version. Instead of renaming, I'd keep the same name and only enable build for distributions that need them. (as discussed earlier) And, of course, never enable for Factory, instead push the required version.

apparently unused (or a dependency of an apparently unused package) gmsh mongodb openblas petsc tetgen cgns - apparently used only by gmsh blacs - apparently used only by petsc hypre - apparently used only by petsc scalapack - apparently used only by petsc

newer version not useful hdf netcdf-cxx qwt

fundamental packages with a high chance of breakage: GeoIP agg cmake doxygen freeimage lapack libgit2 libsmi lz4 mercurial opencv-qt5 openpgm snappy vtk vtkdata xrootd zeromq swig3 - this has be renamed to just "swig", packages shouldn't use it anymore

low-risk packages that can be copied if needed cfitsio fftw3 geos hdf5 hdf5-1_8 libcryptopp libsodium netcdf netcdf-cxx4 parallel-netcdf portmidi qhull qscintilla qscintilla-qt5 suitesparse

used only by python packages, should be owned by d:l:p blosc DSDP

should by be in a subject-specific repo (moved in necessary) gccxml liblouis libsvm meep pythonocc rrdtool sundials python/3-PyAVM python/3-pyMinuit2 mhash - used by meep harminv - used by meep libctl - used by meep oce - used by pythonocc geom - used by pythonocc smesh - used by pythonocc erfa - used by python/3-PyAVM wcslib - used by python/3-PyAVM python/3-meep - bindings for meep python/3-py-rrdtool - bindings for rrdtool python/3-pygccxml - bindings for gccxml python/3-pysundials - bindings for sundials

-- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 11/03/2016 06:38 AM, Todd Rme wrote:

On Mon, Oct 31, 2016 at 11:28 AM, jan matejek wrote:

...

hello,

On 27.10.2016 20:35, Todd Rme wrote:

...

2. Along those lines, I think pypy2 and pyp3 versions should be available, especially with a lot of progress now happening in pypy3 thanks to mozilla funding it.

I would very much prefer to NOT support pypy 2 at all, now that pypy3 is making significant progress. Given that we don't really support pypy2 at this point, I don't see any benefits in starting. That's why my proposal uses unversioned "pypy". That is not going to be future-proof if we ever get pypy 4, but, well, I say we take that risk ;)

...

I would personally keep dlp as a generic place for all python packages. This has worked very well up to this point and I don't see any advantage to changing it.

I have always considered d:l:py to be a generic place for Python modules and tools -- that is, for software where Python is a fundamental property, as opposed to implementation detail. From your conversation with Simon, it doesn't seem like you disagree on this, but you're concerned about the "edge case" packages -- where it's unclear whether the "library" or the "utility" part is more important.

Let me clarify that I don't propose to throw these out summarily. I'm more concerned about the actually-clear-cut apps. I don't know how many of those are in d:l:py now, but I have rejected several in the last year, and I'd like to make it clearer that these don't belong. (Also, being more careful about the edge cases would not hurt, IMHO ;) )

...

I think the rule should be that if the executable is intended for use with python files or packages, it should provide both python 2 and python 3 versions when available.

to this I would add, "and if the executable doesn't natively support multiple python versions". For instance, if pylint3 could work on python 2 by itself, I would include it in the rule and say that we will only distribute pylint3. This could very well be the case for IDEs and similar.

...

...

Finally, three, d:l:py is collecting non-python dependencies of python modules. That should not be happening. Analogously to point one, policy proposal: If your package depends on something that is not appropriate for d:l:py, either get that dependency into Factory, or your package is also not appropriate for d:l:py.

A lot of these are backports for older versions of openSUSE that either have outdated versions of the library or lack the library entirely. So yes, I agree if it really has non-Factory dependencies it probably shouldn't be there. But I think the bigger problem is that many of these are (or were) needed but are building for openSUSE

It doesn't make sense to me to build a binding for, say, 13.2, for a library that is not present or too old in 13.2. In such case, I think the right solution is to disable build for the older SUSE version -- and optionally link the binding to a project that provides the newer library for the older distribution.

...

1. The layout of the repositories. With single spec files, does "devel:languages:python3" even make sense anymore? I think we should phase it out.

Definitely. That is one of the goals.

...

2. Package naming. Might this be a chance to switch from using python-foo to python2-foo? Either way, I would strongly suggest pypy packages using pypy2-foo and pypy3-foo from the beginning, rather than pypy-foo.

Probably not, for technical reasons. The problem is that the macroset can't modify what's already explicitly written in the spec file. So if the spec is for python-foo, and has a %files section, then we MUST build package python-foo that includes the files listed.

We could of course rename the package and spec, but, as Christian pointed out, that is a lot of renames for no clear benefit.

What makes more sense is to gradually rename packages to python3-foo, and generate python2-foo subpackages (with Provides: python-foo, which could in time move to the python3 subpackage). The macroset is designed with this possibility in mind, although it's probably buggy at the moment.

(as for pypy, as I wrote before, i would prefer to never even use versioned names for it)

...

3. Version-specific packages. There are still a lot of packages that only support python 2, and an increasing number that only support python 3. Should these use the macros or be kept simple?

python2-only, I would consider legacy and keep simple, until such a time when they move to python 3

python3-only, I'd use the macros and "%define have_python2 0". After all, these could work on pypy or other flavors that might be created in the future.

...

4. Repository cleanup. There are packages that haven't been updated in many years and are not under active development. Who knows what sort of security or compatibility issues they might have. It might be worthwhile to move them to a separate repository (perhaps d:l:p:archive or d:l:p:legacy) with lower build priority, or delete them outright, to free up resources for packages that are better-maintained. 5. Disabled packages. Packages that are completely disabled should almost certainly be deleted.

good ideas

regards m.

Another issue: might this be a good time to drop support for SLES 11.4? It is the only version that doesn't support python 3, and many packages require some significant workaround to keep building. I think it is probably past due to drop support for it, or at least to disable all builds and freeze the repo.

As general info SLED is out of support and SLES will have General support until 31 Mar 2019 and LTSS support until 31 Mar 2022 which is still quite some time. I don't have a strong opinion here but dropping it seems reasonable to me unless people indicate they are using it and are willing to do the work. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adeliade Australia, UTC+9:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B