[opensuse-python] RFC: New Policy: Submitting new Python packages should not be free
Hello, At the time of writing we have 538 packages in d:l:python which are out of date and require updating. There are only two reasons why any Python package should be packaged for OpenSUSE: either it is dependency of another package in OpenSUSE, or we want to maintain it. Otherwise, if an user wants an unmaintained package, every user has pip available and they can install a package from PyPI directly. The conclusion of these two points is that every package in Factory carries with itself some (small) cost, and there is no point in trying to push all of PyPI into Factory. Therefore I suggest these limitations on putting new packages into Factory: 1. Every new package submitted to d:l:p (or any other official OpenSUSE project) SHALL include in its submit request message “business reasons” for including the package into OpenSUSE (either because it is dependency of some other package, or some other reason, why it is needed). 2. Everybody who wants to submit new package to OpenSUSE, MUST submit two updates of packages already in Factory from the list delivered to this list every week. 3. Packages which fail to build for sufficiently long time SHALL be removed from Factory and d:l:python (or moved to d:l:p:misc). John, let me address you directly, because you do by far the most work for OpenSUSE Python packages. I really do appreciate how incredibly much you do for OpenSUSE, but I just don’t think it is right, when from 30+ requests on any work day, twenty of them are submissions of your new packages. Moreover, some of those packages are really questionable. For example, do we really want to maintain in OpenSUSE complete set of packages needed for accessing Fedora infrastructure? Why anybody who wants to maintain Fedora packages on OpenSUSE systems (and that include me, I have still rights for some Fedora packages) cannot just use `pip install --user copr-cli`? Any queries and comments are welcome, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 The truth is a beautiful and terrible thing, and should therefore be treated with caution. -- Albus Dumbledore
Hi, I strongly disagree with most of this, firstly if i'm writing a script and have a missing dep, i'd much rather push it to tumbleweed then deal with pip I want to zypper dup/up not try and think of what I need to update with pip. Secondaly in openSUSE we always encourage people to do and only say no to a contribution if we have very good reason to, we also don't expect anyone to do work for us. With that in mind id suggest a policy more along these lines which is what happens elsewhere. If someone adds a new package they get added as the maintainer in obs (For now lets just talk about non core things that aren't in SLE). At that point they are expected to keep the package up to date. If the package stops building and they don't fix it then it gets dropped as per our process everywhere else in factory, if the package has a security or critical bug and doesn't get fixed then it gets dropped. If a package is otherwise out of date and no one cares then frankly that's fine it can be out of date eventually if it gets to a point above it will be fixed or dropped. What goes alongside that is no one expects you or anyone else to keep packages that you don't want to maintain up to date which means there is only a small amount of work per package in the unusual case where we need to modify some macro etc even then generally such a change can be done in a way where not every package needs to be done at once. Cheers Simon On 11/20/20 5:47 PM, Matěj Cepl wrote:
Hello,
At the time of writing we have 538 packages in d:l:python which are out of date and require updating.
There are only two reasons why any Python package should be packaged for OpenSUSE: either it is dependency of another package in OpenSUSE, or we want to maintain it. Otherwise, if an user wants an unmaintained package, every user has pip available and they can install a package from PyPI directly.
The conclusion of these two points is that every package in Factory carries with itself some (small) cost, and there is no point in trying to push all of PyPI into Factory.
Therefore I suggest these limitations on putting new packages into Factory:
1. Every new package submitted to d:l:p (or any other official OpenSUSE project) SHALL include in its submit request message “business reasons” for including the package into OpenSUSE (either because it is dependency of some other package, or some other reason, why it is needed).
2. Everybody who wants to submit new package to OpenSUSE, MUST submit two updates of packages already in Factory from the list delivered to this list every week.
3. Packages which fail to build for sufficiently long time SHALL be removed from Factory and d:l:python (or moved to d:l:p:misc).
John, let me address you directly, because you do by far the most work for OpenSUSE Python packages. I really do appreciate how incredibly much you do for OpenSUSE, but I just don’t think it is right, when from 30+ requests on any work day, twenty of them are submissions of your new packages. Moreover, some of those packages are really questionable. For example, do we really want to maintain in OpenSUSE complete set of packages needed for accessing Fedora infrastructure? Why anybody who wants to maintain Fedora packages on OpenSUSE systems (and that include me, I have still rights for some Fedora packages) cannot just use `pip install --user copr-cli`?
Any queries and comments are welcome,
Matěj
_______________________________________________ openSUSE Python mailing list -- python@lists.opensuse.org To unsubscribe, email python-leave@lists.opensuse.org List Netiquette: https://en.opensuse.org/openSUSE:Mailing_list_netiquette List Archives: https://lists.opensuse.org/archives/list/python@lists.opensuse.org
-- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Simon Lees píše v Pá 20. 11. 2020 v 18:45 +1030:
I strongly disagree with most of this, firstly if i'm writing a script and have a missing dep, i'd much rather push it to tumbleweed then deal with pip I want to zypper dup/up not try and think of what I need to update with pip.
I have absolutely no intent to stop submission of new packages. Just update two packages already in d:l:p (or any other of the checked projects, it may be as simple as just bumping the version number, rebuilding the package, and submitting to OBS), and you are very welcome to update new one. The message I just really want to send is that having package in d:l:p is not costless. The cost is small, but with 2007 packages in it (with over 500 in need of update), the bottom line is very much non-zero.
Secondly in openSUSE we always encourage people to do and only say no to a contribution if we have very good reason to, we also don't expect anyone to do work for us. With that in mind id suggest a policy more along these lines which is what happens elsewhere.
Perhaps the solution would be just send some packages to some other repos, which would remain unmaintained by the community (i.e., if you care about them enough to maintain them, be my guest, or actually the guest of OBS). So, for example, all those COPR packages belong to system:packagemanager, after all they have really not much to do with the Python ecosystem anyway. Also, I would have no problems with (non-monitored) d:l:p:PyPI project with automatically (or otherwise) generated mirror of PyPI.
If someone adds a new package they get added as the maintainer in obs (For now lets just talk about non core things that aren't in SLE).
That’s absolutely perfect, but let them do it outside of d:l:p (or other monitored projects).
What goes alongside that is no one expects you or anyone else to keep packages that you don't want to maintain up to date which means there is only a small amount of work per package in the unusual case where we need to modify some macro etc even then generally such a change can be done in a way where not every package needs to be done at once.
We have scripts which follow these repositories https://gitlab.com/mcepl/suse_cron_scripts/-/blob/master/scripts/environment... and generate the weekly email to this list. It would be very difficult to separate unmaintained packages in these projects from those we wish to update. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 In the government of this Commonwealth, the legislative department shall never exercise the executive and judicial powers, or either of them: The executive shall never exercise the legislative and judicial powers, or either of them: The judicial shall never exercise the legislative and executive powers, or either of them: to the end it may be a government of laws and not of men. -- John Adams in the Article XXXth of the Constitution of the Commonwealth of Massachusetts
On 11/20/20 10:35 PM, Matěj Cepl wrote:
Simon Lees píše v Pá 20. 11. 2020 v 18:45 +1030:
I strongly disagree with most of this, firstly if i'm writing a script and have a missing dep, i'd much rather push it to tumbleweed then deal with pip I want to zypper dup/up not try and think of what I need to update with pip.
I have absolutely no intent to stop submission of new packages. Just update two packages already in d:l:p (or any other of the checked projects, it may be as simple as just bumping the version number, rebuilding the package, and submitting to OBS), and you are very welcome to update new one.
The message I just really want to send is that having package in d:l:p is not costless. The cost is small, but with 2007 packages in it (with over 500 in need of update), the bottom line is very much non-zero.
I don't actually see a problem with 500 packages needing an update, at least if those packages are working for people. My other point is that the people who submit the packages and want to maintain them should be paying that cost for the most part not you.
Secondly in openSUSE we always encourage people to do and only say no to a contribution if we have very good reason to, we also don't expect anyone to do work for us. With that in mind id suggest a policy more along these lines which is what happens elsewhere.
Perhaps the solution would be just send some packages to some other repos, which would remain unmaintained by the community (i.e., if you care about them enough to maintain them, be my guest, or actually the guest of OBS).
So, for example, all those COPR packages belong to system:packagemanager, after all they have really not much to do with the Python ecosystem anyway.
Yeah in some cases that would make sense, I already maintain the bindings for efl in the enlightenment repo because it makes more sense.
Also, I would have no problems with (non-monitored) d:l:p:PyPI project with automatically (or otherwise) generated mirror of PyPI.
If someone adds a new package they get added as the maintainer in obs (For now lets just talk about non core things that aren't in SLE).
That’s absolutely perfect, but let them do it outside of d:l:p (or other monitored projects).
Please no, then we will just end up with d:l:p and d:l:p:nonmonitored and that would just be a world of confusion. d:l:p:core with just the packages we care about for SLE could be less confusing but not nearly as good as my suggestions down further.
What goes alongside that is no one expects you or anyone else to keep packages that you don't want to maintain up to date which means there is only a small amount of work per package in the unusual case where we need to modify some macro etc even then generally such a change can be done in a way where not every package needs to be done at once.
We have scripts which follow these repositories https://gitlab.com/mcepl/suse_cron_scripts/-/blob/master/scripts/environment... and generate the weekly email to this list. It would be very difficult to separate unmaintained packages in these projects from those we wish to update.
This is not true, obs makes it very easy to get the package maintainer `osc maintainer python-PyPDF2` for example, (using the -e flag will even provide an email address). It would be easy for you to treat packages that have a maintainer separately you could also get the script to email me when the packages I maintain explicitly are out of date (that would be really useful). [A side note is I should maintain several more python packages then I do I don't know where my rights went]. Another way would be to define "Core" and "Non Core" packages a basic definition could be any python package that uses SLE as its origin in Leap. For 15.1 that list was here https://build.opensuse.org/package/view_file/openSUSE:Leap:15.1:Update/00Met... so again easy for your script to deal with (I'm not sure where it went in 15.2). It would be reasonable to state that SUSE is committed to keeping core packages up to date, but not non core packages. Maybe it could also be reasonable to require "Non core" packages to have a maintainer listed, but that might also depend on how many of the non core packages the non SUSE project maintainers are looking after. I also generally agree with Robert, we shouldn't be making the process harder, if i'm in the middle of something the last thing I want to do is stop and update 2 random packages I don't care about (i'm more then happy to update the ones I do care about). The result of making things harder (even by a bit) is that people will just end up leaving packages in there home project and duplicating each others effort which is exactly what we as a project want to avoid. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hi, Am 21.11.20 um 09:18 schrieb Simon Lees:
This is not true, obs makes it very easy to get the package maintainer `osc maintainer python-PyPDF2` for example, (using the -e flag will even provide an email address). It would be easy for you to treat packages that have a maintainer separately you could also get the script to email me when the packages I maintain explicitly are out of date (that would be really useful).
There exists a separate bot script sent by meissner@suse.de which does exactly that already. The signature reads: Python Package Index Reader (/suse/meissner/projects/caldera-tools/packagehunter/rss/python.pl) I don't know if this is automated or sent manually. Cheers, Ben
Hi Matěji, Matěj Cepl <mcepl@cepl.eu> writes:
Hello,
At the time of writing we have 538 packages in d:l:python which are out of date and require updating.
There are only two reasons why any Python package should be packaged for OpenSUSE: either it is dependency of another package in OpenSUSE, or we want to maintain it. Otherwise, if an user wants an unmaintained package, every user has pip available and they can install a package from PyPI directly.
The conclusion of these two points is that every package in Factory carries with itself some (small) cost, and there is no point in trying to push all of PyPI into Factory.
Therefore I suggest these limitations on putting new packages into Factory:
1. Every new package submitted to d:l:p (or any other official OpenSUSE project) SHALL include in its submit request message “business reasons” for including the package into OpenSUSE (either because it is dependency of some other package, or some other reason, why it is needed).
I am against such a measure, as this is very much subjective and will result in one of the following: a) the devel project maintainer wants to encourage collaboration, so nothing really changes for them, except that they now have to read business reasons as well (and the submitter has to come up with one). b) the devel project maintainer does not want to have an overloaded project, so they either would reject such a package anyway, or they make the submitter the maintainer. Also in this case, I fail to see how a business reason helps. Imho the best solution would be: if you submit a package, then you have to maintain it. If you fail to do so, then it gets dropped. Don't add any additional (arbitrary) conditions. We already have wildly different and mostly unwritten rules for submissions to devel projects, we don't need more of that, rather less. Just my 2ct, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Dan Čermák píše v Pá 20. 11. 2020 v 10:21 +0100:
I am against such a measure, as this is very much subjective and will result in one of the following:
a) the devel project maintainer wants to encourage collaboration, so nothing really changes for them, except that they now have to read business reasons as well (and the submitter has to come up with one).
I didn’t want to change anything for them, just to make them think whether d:l:p is really the place where they want to submit their package, and whether they should submit it into OpenSUSE at all.
b) the devel project maintainer does not want to have an overloaded project, so they either would reject such a package anyway, or they make the submitter the maintainer. Also in this case, I fail to see how a business reason helps.
Well, to the best of my understanding, I am closest to being the maintainer of d:l:p, but the problem is that with over 2000 packages in the project, it is really hard to do decisions without some rules about it. Which is what we are talking about right now. And, forget the word “business”, that was unfortunate, I don’t care about money and sense, only about making the packager to think, as said above.
Imho the best solution would be: if you submit a package, then you have to maintain it. If you fail to do so, then it gets dropped. Don't add any additional (arbitrary) conditions. We already have wildly different and mostly unwritten rules for submissions to devel projects, we don't need more of that, rather less.
With the number of packages we have in d:l:p, I believe the community effort with monitoring scripts is the only way how to make at least some sense in the effort. Yes, we can effectively split whole project into little subfiefdoms and assign group of packages to each maintainer, but I think that the collective maintenance is way more powerful and more sustainable (especially considering having many volunteer packagers, who cannot be expected to be as reliable as full-time professionals due to The Real Life™ making troubles for them). Also, I want to be able to make large-scale changes without struggling in discussions with individual submaintainer. We are currently working on removal of python-nose from OpenSUSE, and whatever else may come up in future (e.g., I have my eyes set on eliminating python-mock as a third party package as well). See https://trello.com/b/WsskhdXA/opensuse-python for current TODO-list. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Quod fuimus, estis; quod sumus, vos eritis.
On 11/21/20 1:56 AM, Matěj Cepl wrote:
Dan Čermák píše v Pá 20. 11. 2020 v 10:21 +0100:
I am against such a measure, as this is very much subjective and will result in one of the following:
a) the devel project maintainer wants to encourage collaboration, so nothing really changes for them, except that they now have to read business reasons as well (and the submitter has to come up with one).
I didn’t want to change anything for them, just to make them think whether d:l:p is really the place where they want to submit their package, and whether they should submit it into OpenSUSE at all.
One of openSUSE's fundamental strengths has always been that we welcome any contribution or package that they believe will be useful for someone in the community
b) the devel project maintainer does not want to have an overloaded project, so they either would reject such a package anyway, or they make the submitter the maintainer. Also in this case, I fail to see how a business reason helps.
Well, to the best of my understanding, I am closest to being the maintainer of d:l:p, but the problem is that with over 2000 packages in the project, it is really hard to do decisions without some rules about it. Which is what we are talking about right now.
And, forget the word “business”, that was unfortunate, I don’t care about money and sense, only about making the packager to think, as said above.
Imho the best solution would be: if you submit a package, then you have to maintain it. If you fail to do so, then it gets dropped. Don't add any additional (arbitrary) conditions. We already have wildly different and mostly unwritten rules for submissions to devel projects, we don't need more of that, rather less.
With the number of packages we have in d:l:p, I believe the community effort with monitoring scripts is the only way how to make at least some sense in the effort. Yes, we can effectively split whole project into little subfiefdoms and assign group of packages to each maintainer, but I think that the collective maintenance is way more powerful and more sustainable (especially considering having many volunteer packagers, who cannot be expected to be as reliable as full-time professionals due to The Real Life™ making troubles for them).
The reality is most openSUSE packages (especially once your away from the core) are maintained by single people who cares about a certain few things, sometimes they no longer have time and we need to find a new maintainer or drop the package
Also, I want to be able to make large-scale changes without struggling in discussions with individual submaintainer. We are currently working on removal of python-nose from OpenSUSE, and whatever else may come up in future (e.g., I have my eyes set on eliminating python-mock as a third party package as well). See https://trello.com/b/WsskhdXA/opensuse-python for current TODO-list.
Needing to discuss large scale changes is a part of working in the openSUSE community, that is why we have this list and the factory list. Although rather then discussing it with each maintainer 1-1 a better approach would be to write a proposal to this list about dropping python-nose and if you get a general consensus here you can just give any maintainers a link to the thread and be done. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Dan Čermák píše v Pá 20. 11. 2020 v 10:21 +0100:
I am against such a measure, as this is very much subjective and will result in one of the following: a) the devel project maintainer wants to encourage collaboration, so nothing really changes for them, except that they now have to read business reasons as well (and the submitter has to come up with one).
If there is one argument I would like to finish next year (yes, that's how little hope in humanity I have) that this is this one. There is no such an easy packaging, which never changes. Yes, of course, we should not throw artificial obstacles in packagers way, but yes, if you want to contribute to large public distro, you have to be literate. And instead of inane abuse I now read on Factory about “News in openSUSE Packaging” I would expect some recognition of this fact. Goal of the Linux distribution is not only to be as large as possible, but also to keep high quality of its packages. If we throw quality of packages to anonymous (and not permanent) volunteer contributors, quality will suffer quickly. Ehm, just saw on Twitter https://twitter.com/sysrich/status/1330554305275957248
Imho the best solution would be: if you submit a package, then you have to maintain it.
Except: a) That is not how OpenSUSE (especially around Python) has been working … we all help to maintain each others packages, or even better ownership of packages is not much considered. You are suggesting quite radical change, a way more radical than what I am suggesting. And I am strongly against it (experienced this organization in two distros and it was always mess which had to be worked around all the time). b) We don’t have enough tools to monitor quality of packaging by others. I am afraid that the reaction of users on broken packages is not filing bugs on bugzilla (nobody files bugs there), but either switching to Ubuntu or using pip directly, which means even less testing for our packages. BTW, attached is the screenshot of my current queue of the review of submission requests, and it is pretty normal. Everybody throws new packages to d:l:p and not many actually maintain them. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 See, when the GOVERNMENT spends money, it creates jobs; whereas when the money is left in the hands of TAXPAYERS, God only knows what they do with it. Bake it into pies, probably. Anything to avoid creating jobs. -- Dave Barry
Hmm, that screenshot doesn't look recent... This e-mail was delivered by the mail system over a week after it has been sent? Anyway, Am 23.11.20 um 22:32 schrieb Matěj Cepl:
Dan Čermák píše v Pá 20. 11. 2020 v 10:21 +0100:
I am against such a measure, as this is very much subjective and will result in one of the following: a) the devel project maintainer wants to encourage collaboration, so nothing really changes for them, except that they now have to read business reasons as well (and the submitter has to come up with one). b) We don’t have enough tools to monitor quality of packaging by others. I am afraid that the reaction of users on broken packages is not filing bugs on bugzilla (nobody files bugs there), but either switching to Ubuntu or using pip directly, which means even less testing for our packages.
But how will putting additional constraints on submit requests help in this situation? If we want a better Python environment without having to switch to pip, we want *more* submit requests to fix unresolvable and broken packages, not less. Cheers, Ben
Hmm, that screenshot doesn't look recent... This e-mail was delivered by the mail system over a week after it has been sent? Anyway, Am 23.11.20 um 22:32 schrieb Matěj Cepl:
Dan Čermák píše v Pá 20. 11. 2020 v 10:21 +0100:
I am against such a measure, as this is very much subjective and will result in one of the following: a) the devel project maintainer wants to encourage collaboration, so nothing really changes for them, except that they now have to read business reasons as well (and the submitter has to come up with one). b) We don’t have enough tools to monitor quality of packaging by others. I am afraid that the reaction of users on broken packages is not filing bugs on bugzilla (nobody files bugs there), but either switching to Ubuntu or using pip directly, which means even less testing for our packages.
But how will putting additional constraints on submit requests help in this situation? If we want a better Python environment without having to switch to pip, we want *more* submit requests to fix unresolvable and broken packages, not less. Cheers, Ben
Am Montag, 23. November 2020, 22:32:40 CET schrieb Matěj Cepl:
Goal of the Linux distribution is not only to be as large as possible, but also to keep high quality of its packages. If we throw quality of packages to anonymous (and not permanent) volunteer contributors, quality will suffer quickly.
Hi Matej, fundamentally, you're saying "outdated" == "low quality". and that is not always the case. I would argue in some cases the "lets get rid of nose and drop unittest2" changes that ended up creating hundreds of downstream patches everywhere, some of them never going upstream or are submitted upstream but failing there (or having patch conflicts meanwhile) did not *improve* quality either. I'm not against such efforts per se, but doing those downstream first and never following up on getting them upstream makes life just miserable for everyone else at very little value achieved. For me quality is: "it works and is secure". There are soft quality factors, like "is recent, is performant", but those are a clear 2nd factor over "it works" and "is secure". so if we have a particular thing that is having vulnerabilities, there is no doubt that it should be fixed (updated, patch added) or dropped. So how about we draft a policy around the actual goals rather than having a unrelated discussion ("you can only add a package if you have a business reason"). Also, IMHO openSUSE is not really "large" by any means. I'd say it is by far the smallest of the popular distros, so instead of coming up with policies on our own we maybe should also learn about how others that are larger are managing that (I haven't really educated myself on that either, so I can't point it out directly, sorry).
a) That is not how OpenSUSE (especially around Python) has been working … we all help to maintain each others packages, or even better ownership of packages is not much considered. You are suggesting quite radical change, a way more radical than what I am suggesting. And I am strongly against it (experienced this organization in two distros and it was always mess which had to be worked around all the time).
In my observation, the python packaging has also always done things in a way that make things extremly painful. or in another way of stating it, it sometimes appeared like you intentionally want to hit all the obstacles on the way.
b) We don’t have enough tools to monitor quality of packaging by others. I am afraid that the reaction of users on broken packages is not filing bugs on bugzilla (nobody files bugs there), but either switching to Ubuntu or using pip directly, which means even less testing for our packages.
BTW, attached is the screenshot of my current queue of the review of submission requests, and it is pretty normal. Everybody throws new packages to d:l:p and not many actually maintain them.
As far as I remember in this mailthread, noone was against a policy of dropping unmaintained(unmaintainable) stuff. Lets just work on a refresher on the original policy draft proposal addressing also the feedback you got. Greetings, Dirk
Dirk Müller píše v Út 01. 12. 2020 v 13:52 +0100:
fundamentally, you're saying "outdated" == "low quality".
Just to be clear. No, I am not saying that. To rewrite your statement, I would say more like “many outdated packages” == “low quality of maintenance” (or to paraphrase even more, “we took more than we can chew on”).
and that is not always the case. I would argue in some cases the "lets get rid of nose and drop unittest2" changes that ended up creating hundreds of downstream patches everywhere, some of them never going upstream or are submitted upstream but failing there (or having patch conflicts meanwhile) did not *improve* quality either.
As far as I know, ALL those patches went upstream and many were accepted with gratitude (of course, many projects are dead or a maintainer is lazy, but at least the patch is known and public in such situations). We got couple of “Lovely, so at least SUSE is not only living from our work but sometimes also contributing upstream” as well. Hopefully with continuing updates back from upstream, we will be removing those patches from our packages (as it happened in many situations).
For me quality is: "it works and is secure". There are soft quality factors, like "is recent, is performant", but those are a clear 2nd factor over "it works" and "is secure".
Is it so surprising idea that most commits upstream are meant to improve that software? Is it so surprising to expect that the easiest way (in large numbers, I have neither time nor will to do per-package research) to get the best and greatest is to make sure we have the latest?
So how about we draft a policy around the actual goals rather than having a unrelated discussion ("you can only add a package if you have a business reason").
Thank you for keeping the discussion on topic and avoiding personal attacks. My actual goal, which you apparently don’t share, but it doesn’t make it less relevant, is to keep Factory updated and not ignore the fact that quarter of our packages in the supposedly latest and most raw edge distribution are outdated. That was the goal I started this thread with. You may not like this goal, you may think that you have better metric for the quality of our Python packages, but it doesn’t give you the right to refuse it as ignorable.
Also, IMHO openSUSE is not really "large" by any means. I'd say it is by far the smallest of the popular distros, so instead of coming up with policies on our own we maybe should also learn about how others that are larger are managing that (I haven't really educated myself on that either, so I can't point it out directly, sorry).
I have no idea about Debian (and related distros), but from my (very) past experience they don’t care much, and some parts of their waste repository are not very well maintained. Also, their solution to most of problems is to have endless number of people on the job. I don’t have exact numbers, but combining Debian, Ubuntu, Mint, and other derived distros, they have substantial share of all Linux users and thus also developers. From my personal experience with Fedora (I was working at Red Hat for eleven years), their solution is simple: exactly what I was suggesting. Keeping the number of packages low, keeps the situation somehow stable. Let me present some numbers: * Debian unstable https://repology.org/repository/debian_unstable * Fedora Rawhide https://repology.org/repository/fedora_rawhide * OpenSUSE Tumbleweed https://repology.org/repository/opensuse_tumbleweed Ignoring small difference in distributions (yes, we have QA, Fedora effectively doesn’t, I am not sure about the current situation in Debian) and ignoring unknown methodology of indicating “outdated” packages (and just by brief looks at their graphs, I am not full of confidence in it), I see that * Debian has 33,015 packages in total, 6,879 of them outdated, which is 20.84 %. * Fedora has 22,489 packages, 4,349 of them outdated, which is 19.34 % * We have the smallest number of packages, 14,152, but 3,052 of them are outdated, which is the highest share, 21.57 %. * Situation in d:l:p is even worse, 538 out of 2,027 packages (i.e., 26.54 %) need update. And that’s probably the best situation from all d:l:p projects (I see some life in d:l:p:numeric, but otherwise it is really silent there).
In my observation, the python packaging has also always done things in a way that make things extremly painful. or in another way of stating it, it sometimes appeared like you intentionally want to hit all the obstacles on the way.
From my experience packaging both on Debian and Fedora, Python packaging in OpenSUSE is by far the easiest and our macros are actually helpful. Without them (and some of our ultra-dedicated maintainers … hi, Markéta! hi, John!, hi, Benjamin!) we would be completely lost.
As far as I remember in this mailthread, noone was against a policy of dropping unmaintained(unmaintainable) stuff. Lets just work on a refresher on the original policy draft proposal addressing also the feedback you got.
Well, the container has two ends: one where the new packages flow in, and the other where unmaintained packages are kicked out of repo. It is by far easier to stop inflow of suspicious packages, then to indicate and eliminate them on the outflow. Control of inflow was what I was trying to achieve. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 There is no reason to suppose that most human beings are engaged in maximizing anything unless it be unhappiness, and even this with incomplete success. -- Ronald Coase Introduction to “The Firm, the Market, and the Law”
Am Donnerstag, 3. Dezember 2020, 21:33:11 CET schrieb Matěj Cepl: Hi Matej,
fundamentally, you're saying "outdated" == "low quality". Just to be clear. No, I am not saying that. To rewrite your statement, I would say more like “many outdated packages” == “low quality of maintenance” (or to paraphrase even more, “we took more than we can chew on”).
yes, I agree, updating python packages by itself is not hard, and I'm doing that regularly. I see two major timesinks: * finding the changelog. often this is not in the tarball itself, so it requires searching where the F this particular piece of software publishes its changelog to add it to changes. I spent more than 80% of the time on this * rebasing off-upstream patches. This is mostly like those "general cleanups" patches, like unittest2 removals that did not go upstream or the like. everything else, is probably like 2-3 minutes of work per package. Not that I'm saying I am happy to do 500+ package updates a week, but I am happy to do the ones that I care about once a week or so, especially if there are no off- upstream patches added.
As far as I know, ALL those patches went upstream and many were accepted with gratitude (of course, many projects are dead or a maintainer is lazy, but at least the patch is known and public in such situations). We got couple of “Lovely, so at least SUSE is not only living from our work but sometimes also contributing upstream” as well. Hopefully with continuing updates back from upstream, we will be removing those patches from our packages (as it happened in many situations).
Yes, I get that (and I did that). the packages that I care about (need to search for the concrete examples), I hit like 5 packages in a row where the patch never went upstream, or where the patch was posted upstream but in conflicts and never rebased. This is very frustrating. I think being able to drop those patches and accelerating the work on getting it upstream would be a solution. Its not the one you prefer, but that leads to updating packages become slow and more painful, not so much more a side job. One of the packages that I care about for example is eventlet. Now for reasons that I don't fully know we have updated dnspython to 2.x, which is incompatible (this is documented upstream). I tried to fork a dnspython1 package, which was deleted. patches to add dnspython 2.x support were added instead, which did not yet go upstream, and while they passed some of the unittests, it actually caused a lot of pain in real functionality. so effectively it broke eventlet. Now, there were newer releases, and maybe there is a release now that supports the newer dnspython, but lets generalize the concern: every other distribution seems to stick dnspython on the older version if it breaks something major, and we instead have the conversation of "its outdated, needs to be updated". There was one package (python-cmd2 iirc) that had only one (!) dependency. This one depencency was requiring a specific old version. Yet, I think 6 or 7 times the python team updated cmd2 to the latest release, breaking every time the single depending pacakge. every time I went in, reverted to the old version, and less than two days later someone came in, without checking the changes file, and updated the package again to the latest version, which again broke everything. I do feel like this was frustrating for both sides, and this example should very clearly demonstrate that there is no value whatsoever to always ship the newest package. I am suggesting to by policy aim for shipping the newest *working* package. Working needs to be more important than "latest". I don't say "latest" isn't what we should strive for, but if "latest" breaks, "outdated but working" should be fine, while the regression is being resolved.
Is it so surprising idea that most commits upstream are meant to improve that software?
I have no issue whatsoever for anyone doing stuff upstream. what I have an issue with is aptches that did not *go* upstream (e.g. got merged in $main/ $master of some future release) being added to packages, because they just add drag.
So how about we draft a policy around the actual goals rather than having a unrelated discussion ("you can only add a package if you have a business reason"). Thank you for keeping the discussion on topic and avoiding personal attacks.
Sorry, I hope you didn't mean this ironically. I'm not trying to attack you personally in any way. I see you have found a problem, and it is worth tackling and I'm happy to help. lets put the policy in some shared etherpad and I'm happy to edit or improve it further and hopefully we can come up something that everyone can tolerate and support.
and most raw edge distribution are outdated. That was the goal I started this thread with. You may not like this goal, you may think that you have better metric for the quality of our Python packages, but it doesn’t give you the right to refuse it as ignorable.
I do like to have tumbleweed always up to date. I am regularly working towards that goal (not only with python packages). however I *strongly* prefer working software over newest software.
I have no idea about Debian (and related distros), but from my (very) past experience they don’t care much, and some parts of their waste repository are not very well maintained. Also, their solution to most of problems is to have endless number of people on the job. I don’t have exact numbers, but combining Debian, Ubuntu, Mint, and other derived distros, they have substantial share of all Linux users and thus also developers.
its a spectrum. Debian is very much more on the spectrum towards "stable" and tumbleweed is very much towards the spectrum to "latest". all I'm saying we can take one step back from the rightmost corner of the spectrum of "latest" and still not devalue the goal of a rolling release distro.
From my personal experience with Fedora (I was working at Red Hat for eleven years), their solution is simple: exactly what I was suggesting. Keeping the number of packages low, keeps the situation somehow stable.
I am also not against dropping packages that break often, noone uses, noone maintains. Thats totally fine. I just don't want to make this by making initial contribution difficult. initial contribution needs to be easy. if contributions slow down, or a package goes problematic and noone steps up, I'm with you we should drop it. Thats all I'm trying ot say.
* We have the smallest number of packages, 14,152, but 3,052 of them are outdated, which is the highest share, 21.57 %.
yes. I'm aware of that. "latest" is not necessarily the most important dimension. it is not unimportant at all. if you have a range of 1 to 10, where 1 is "stable working" and "10" is latest, I'm saying we should aim for 8 or 9. Thats all I'm trying to say.
From my experience packaging both on Debian and Fedora, Python packaging in OpenSUSE is by far the easiest and our macros are actually helpful. Without them (and some of our ultra-dedicated maintainers … hi, Markéta! hi, John!, hi, Benjamin!) we would be completely lost.
I am very regularly comparing the work done in fedora with the opensuse packages, because I can often find fixes there and I see the part of the benefit of working in the open that we can collaborate, even if we work on different distros. I don't disagree that the ppython packaging has made things easier and more standardized. thats great. I am not at all saying we should stop that. on the other hand, the macros are also adding a lot of secret sauce that is often not easy to understand.
Well, the container has two ends: one where the new packages flow in, and the other where unmaintained packages are kicked out of repo. It is by far easier to stop inflow of suspicious packages, then to indicate and eliminate them on the outflow. Control of inflow was what I was trying to achieve.
Right, my point is that contributions should be encouraged, and unmaintained stuff should be discouraged. your proposal for me was trying to control "inflow". I would prefer to make "egress" easier. Greetings, Dirk
Dirk Müller píše v Čt 10. 12. 2020 v 16:42 +0100:
Yes, I get that (and I did that). the packages that I care about (need to search for the concrete examples), I hit like 5 packages in a row where the patch never went upstream, or where the patch was posted upstream but in conflicts and never rebased.
Either mistakes happen, or just I am not responsible for anybody's else work. Sorry.
One of the packages that I care about for example is eventlet. Now for reasons that I don't fully know we have updated dnspython to 2.x, which is incompatible (this is documented upstream). I tried to fork a dnspython1 package, which was deleted. patches to add dnspython 2.x support were added instead, which did not yet go upstream, and while they passed some of the unittests, it actually caused a lot of pain in real functionality. so effectively it broke eventlet.
There is probably one solution to this. Take over maintenance of packages you care about, and when you see submit request coming which is stupid or hurtful, make your opinion known. We are certainly not forcing the latest updates for packages which are known to break universe. What would break if we revert dnspython back to 1.* (although, I have built https://build.opensuse.org/project/show/home:mcepl:branches:devel:tools:scm with dnspython 2.0.0 and it seems to work)? Are there any patches whcih would help eventlet? I don't know, but I am very willing to be told.
Now, there were newer releases, and maybe there is a release now that supports the newer dnspython, but lets generalize the concern: every other distribution seems to stick dnspython on the older version if it breaks something major, and we instead have the conversation of "its outdated, needs to be updated".
Yes, because we don't have anybody who would actually be specialist for dnspython, eventlet, or many other crucial packages, and we who try to keep together d:l:p make a lot of mistakes from ignorance.
I do feel like this was frustrating for both sides, and this example should very clearly demonstrate that there is no value whatsoever to always ship the newest package. I am suggesting to by policy aim for shipping the newest *working* package.
Again, I don't remember repeatedly reverting cmd2, that sounds truly bad, but I ended up making comments in SPEC files in sense "Do not upgrade until you are certain you are not breaking python-foo!". Just above the Version: line. Perhaps, that could help.
Working needs to be more important than "latest". I don't say "latest" isn't what we should strive for, but if "latest" breaks, "outdated but working" should be fine, while the regression is being resolved.
I completely agree.
lets put the policy in some shared etherpad and I'm happy to edit or improve it further and hopefully we can come up something that everyone can tolerate and support.
https://en.opensuse.org/openSUSE:Packaging_Python is a wiki. It has Discussion page. Or we have this email list. Let's not make the third avenue for discussion.
your proposal for me was trying to control "inflow". I would prefer to make "egress" easier.
I think we have to do both. Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Oh, to be young, and to feel love’s keen sting. -- Albus Dumbledore
Hi, Am 13.12.20 um 09:37 schrieb Matěj Cepl:
Dirk Müller píše v Čt 10. 12. 2020 v 16:42 +0100:
I only seem to receive half of the conversation. Dirk did you send to Matej directly and only he replied back to the list (with two addresses, so I get that message twice)?
One of the packages that I care about for example is eventlet. Now for reasons that I don't fully know we have updated dnspython to 2.x, which is incompatible (this is documented upstream). I tried to fork a dnspython1 package, which was deleted. patches to add dnspython 2.x support were added instead, which did not yet go upstream, and while they passed some of the unittests, it actually caused a lot of pain in real functionality. so effectively it broke eventlet.
Regarding that particular case: dnspython1 is still present in d:l:p. Why was it deleted from Factory (?). You describe a very valid reason to have it: Packages need it for painless functionality. We have more packages in a similar situation: pytest, tornado, Sphinx, etc. We probably need a better guide how to set up the names, prjconf files and Provides/Obsoletes/Conflicts tags so that (in my opinion) ill-advised changes like https://build.opensuse.org/request/show/852863 don't happen.
Now, there were newer releases, and maybe there is a release now that supports the newer dnspython, but lets generalize the concern: every other distribution seems to stick dnspython on the older version if it breaks something major, and we instead have the conversation of "its outdated, needs to be updated".
[...]
I do feel like this was frustrating for both sides, and this example should very clearly demonstrate that there is no value whatsoever to always ship the newest package. I am suggesting to by policy aim for shipping the newest *working* package.
I'd rather state it as: "If there is a stable, working, newer package, provide it" in the sense that TW provides the greatest and latest. But it should go along with "If an outdated package is needed in order not to break something major, provide it as alternative. As long as it does not pose a security risk.
Again, I don't remember repeatedly reverting cmd2, that sounds truly bad, but I ended up making comments in SPEC files in sense "Do not upgrade until you are certain you are not breaking python-foo!". Just above the Version: line. Perhaps, that could help.
That's why we need activated unit tests and at the very latest notice breaking stuff in Staging. Ben
On 12/13/20 9:37 AM, Matěj Cepl wrote:
What would break if we revert dnspython back to 1.*
Actually dnspython is a very good example to look at. In my case I was forced to change the code in my own upstream software regardless what's present in downstream package repos. Why? Because if people do a pip install they'll get dnspython 2.x. Yes, I could pin to dnspython <= 1.6.x but then nobody will receive security updates if upstream author decides not to maintain the 1.6 release anymore. Also dnspython 1.6 could be removed downstream. So pinning in such a way is bad practice. It could be done temporarily but there's no valid definition for "temporarily" and thus in practice the pinning would last forever until another dependency breaks it. => Bite the bullet and fail forward. Ciao, Michael.
Michael Ströder píše v Ne 13. 12. 2020 v 18:00 +0100:
Actually dnspython is a very good example to look at.
In my case I was forced to change the code in my own upstream software regardless what's present in downstream package repos.
Why? Because if people do a pip install they'll get dnspython 2.x.
Yes, I could pin to dnspython <= 1.6.x but then nobody will receive security updates if upstream author decides not to maintain the 1.6 release anymore. Also dnspython 1.6 could be removed downstream. So pinning in such a way is bad practice. It could be done temporarily but there's no valid definition for "temporarily" and thus in practice the pinning would last forever until another dependency breaks it.
=> Bite the bullet and fail forward.
Which hopefully at least in some cases we help by patching the offended packages. Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 As long as we are thinking of natural values we must say that the sun looks down on nothing half so good as a household laughing together over a meal, or two friends talking over a pint of beer, or a man alone reading a book that interests him; and that all economies, politics, laws, armies, and institutions, save insofar as they prolong and multiply such scenes, are a mere ploughing the sand and sowing the ocean, a meaningless vanity and vexation of the spirit. Collective activities are, of course, necessary, but this is the end to which they are necessary. -- C.S. Lewis, “Membership” in “The Weight of Glory”
Am Freitag, 20. November 2020, 08:17:08 CET schrieb Matěj Cepl: Hi Matej,
At the time of writing we have 538 packages in d:l:python which are out of date and require updating.
sometimes having them out of date is a feature (when the newer version breaks everything else). But that should be a minority.
There are only two reasons why any Python package should be packaged for OpenSUSE: either it is dependency of another package in OpenSUSE, or we want to maintain it. Otherwise, if an user wants an unmaintained package, every user has pip available and they can install a package from PyPI directly.
"We want to maintain it" is a very vague term.. is that more than the "the initial submitter wants to maintain it"? if so, what is it?
1. Every new package submitted to d:l:p (or any other official OpenSUSE project) SHALL include in its submit request message “business reasons” for including the package into OpenSUSE (either because it is dependency of some other package, or some other reason, why it is needed).
Whats a "business reason" for a community project?
2. Everybody who wants to submit new package to OpenSUSE, MUST submit two updates of packages already in Factory from the list delivered to this list every week.
Huh? Why? I get the idea, you want people to help contributing also in other areas, but imho thats why we had all the splitting of dlp into subprojects (with the corresponding pain for people that just want to use a set of working and updated packages on older distros)
include me, I have still rights for some Fedora packages) cannot just use `pip install --user copr-cli`?
Thats the same reason for every of those python-packages, right? lets assume I want to have certbot. I can totally install that from pip, so why is it packaged in openSUSE? I'm not against defining a policy for inclusion, but we need to chew a bit more on the rules for that. Personally, I can live without the set of python packages, as well as I can live with packaging more. I can also live with outdated packages that I don't care about, as long as they still build and don't cause my packages to fail building. There are just a few packages that are incredibly painful to maintain, as they are breaking frequently when its dependencies are updated. maybe we should try to do something about them first (e.g. Twisted is one that came accross). Greetings, Dirk
Dirk Müller píše v Pá 20. 11. 2020 v 11:22 +0100:
sometimes having them out of date is a feature (when the newer version breaks everything else). But that should be a minority.
Certainly, but as you said, these should be minority (I have just downgraded python-Pygments).
"We want to maintain it" is a very vague term.. is that more than the "the initial submitter wants to maintain it"? if so, what is it?
See below, its updates are monitored and the package should be updated as soon as possible (with stress on the word “possible”).
Whats a "business reason" for a community project?
I haven’t meant business reasons with the bottom line calculated, just to make submitter stop and think, whether he has really good reasons why to push the package to d:l:p. Do we really need python-buttplug in d:l:p, shouldn't it be better to have it some special project supporting exotic hardware? Is this really important to have it as a part of the Python ecosystem (perhaps yes, we have other Python bindings for other C libraries in d:l:p as well)? Perhaps, but all what I ask is that the submitter stops for a minute and thinks about it (and writes his reasons down, so we can follow his logic). d:l:p shouldn't be just dumpster where people throw anything they can come up with.
Huh? Why? I get the idea, you want people to help contributing also in other areas, but imho thats why we had all the splitting of dlp into subprojects (with the corresponding pain for people that just want to use a set of working and updated packages on older distros)
And yet, we still have over 2000 packages in d:l:p (and 500 of them needs update).
Thats the same reason for every of those python-packages, right? lets assume I want to have certbot. I can totally install that from pip, so why is it packaged in openSUSE?
a) certbot has C modules, doesn't it? Those are always a bitch to package, so having it done well in OpenSUSE makes a lot of sense. b) how many people have OpenSUSE/SLE on their websites and they want to use Letsencrypt for their certificates versus how many people use OpenSUSE to create packages for Fedora/RHEL? c) thinking about these things could stop you long enough to recognize, that these packages don’t belong to d:l:p, but to the system:packagemanager project.
I'm not against defining a policy for inclusion, but we need to chew a bit more on the rules for that.
That’s why this is RFC. I mean it, I really want to get a feedback.
There are just a few packages that are incredibly painful to maintain, as they are breaking frequently when its dependencies are updated. maybe we should try to do something about them first (e.g. Twisted is one that came accross).
And these should unfortunately certainly be part of our packages which we support, because they are foundation of so many other packages, and exactly because they are difficult to maintain. And yes, I would love to have some volunteer dedicating his/her time to actually really understand well Twisted and all problems related to its packaging. Even if we separate all python*twisted* packages to separate subproject (which may be a good idea), we certainly want to monitor it and keep it supported. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 It seemed that every spring the daffodils came out, every summer the cherries ripened and every autumn William Wilberforce would present his bill to the House. -- said to be written either by his (later?) wife Barbara Spooner, or perhaps by Samuel Johnson, but I could not find evidence for either.
Hi, On 11/20/20 8:41 AM, Matěj Cepl wrote:
Dirk Müller píše v Pá 20. 11. 2020 v 11:22 +0100:
sometimes having them out of date is a feature (when the newer version breaks everything else). But that should be a minority.
Certainly, but as you said, these should be minority (I have just downgraded python-Pygments).
"We want to maintain it" is a very vague term.. is that more than the "the initial submitter wants to maintain it"? if so, what is it?
See below, its updates are monitored and the package should be updated as soon as possible (with stress on the word “possible”).
Whats a "business reason" for a community project?
I haven’t meant business reasons with the bottom line calculated, just to make submitter stop and think, whether he has really good reasons why to push the package to d:l:p. Do we really need python-buttplug in d:l:p, shouldn't it be better to have it some special project supporting exotic hardware? Is this really important to have it as a part of the Python ecosystem (perhaps yes, we have other Python bindings for other C libraries in d:l:p as well)? Perhaps, but all what I ask is that the submitter stops for a minute and thinks about it (and writes his reasons down, so we can follow his logic).
d:l:p shouldn't be just dumpster where people throw anything they can come up with.
Huh? Why? I get the idea, you want people to help contributing also in other areas, but imho thats why we had all the splitting of dlp into subprojects (with the corresponding pain for people that just want to use a set of working and updated packages on older distros)
And yet, we still have over 2000 packages in d:l:p (and 500 of them needs update).
But this problem is not solved by making up some rules that make it harder for people to contribute and ask for a "business reason" in an open source project that is community driven. "business reasons" apply to SLE not to openSUSE. The way to fix the "it doesn't build" or "no one cares and it's way old" problem is to have a policy to evict packages that fall into those categories, define the policy and automate it. Don't create a policy that makes it harder for people to contribute, that's counter productive to what we are trying to achieve. We want to attract contributors not scare them away. Simple things like "When a package fails to build the original contributor or maintainer on record will be notified. If the package build is not fixed after $SUITABLE_GRACE_PERIOD the package will be removed from d:l:p" "The goal of the d:l:p project is to stay reasonably close to upstream where we define "reasonably close" as a version spread of no more than $SUITABLE_VERSION_DIFFERENCE". If a package falls outside this reasonably close condition the original contributor or maintainer on record will be notified. If the package is not updated after $SUITABLE_GRACE_PERIOD the package will be removed from d:l:p" There, simple, can be fully automated and tells people that when they submit a package they have a responsibility. Later, Robert
Thats the same reason for every of those python-packages, right? lets assume I want to have certbot. I can totally install that from pip, so why is it packaged in openSUSE?
a) certbot has C modules, doesn't it? Those are always a bitch to package, so having it done well in OpenSUSE makes a lot of sense. b) how many people have OpenSUSE/SLE on their websites and they want to use Letsencrypt for their certificates versus how many people use OpenSUSE to create packages for Fedora/RHEL? c) thinking about these things could stop you long enough to recognize, that these packages don’t belong to d:l:p, but to the system:packagemanager project.
I'm not against defining a policy for inclusion, but we need to chew a bit more on the rules for that.
That’s why this is RFC. I mean it, I really want to get a feedback.
There are just a few packages that are incredibly painful to maintain, as they are breaking frequently when its dependencies are updated. maybe we should try to do something about them first (e.g. Twisted is one that came accross).
And these should unfortunately certainly be part of our packages which we support, because they are foundation of so many other packages, and exactly because they are difficult to maintain. And yes, I would love to have some volunteer dedicating his/her time to actually really understand well Twisted and all problems related to its packaging. Even if we separate all python*twisted* packages to separate subproject (which may be a good idea), we certainly want to monitor it and keep it supported.
Best,
Matěj
_______________________________________________ openSUSE Python mailing list -- python@lists.opensuse.org To unsubscribe, email python-leave@lists.opensuse.org List Netiquette: https://en.opensuse.org/openSUSE:Mailing_list_netiquette List Archives: https://lists.opensuse.org/archives/list/python@lists.opensuse.org
-- Robert Schweikert MAY THE SOURCE BE WITH YOU Distinguished Engineer LINUX Technical Team Lead Public Cloud rjschwei@suse.com IRC: robjo
Robert Schweikert píše v Pá 20. 11. 2020 v 09:04 -0500:
But this problem is not solved by making up some rules that make it harder for people to contribute and ask for a "business reason" in an open source project that is community driven. "business reasons" apply to SLE not to openSUSE.
Sorry, but how many times I need to repeat that “business reason” was unfortunate phrasing before you stop quoting it again me?
The way to fix the "it doesn't build" or "no one cares and it's way old" problem is to have a policy to evict packages that fall into those categories, define the policy and automate it.
So, the only bugs we are interested in is that the package is so broken it doesn't even build. Are there any other bugs we care about (and which would require updating)? And yes, exactly this policy we have, and the result is that quarter of packages in d:l:p need updating. Of course, there is an alternative to switch off monitoring, and just let packages rot without any concern from us as long as they at least build, and touch anything only if somebody bothers to file a bug. That will make certainly OpenSUSE the paragon of relevant and up-to- date Linux distro. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 In the course of my life, I have seen Frenchmen, Italians, Russians, etc.; I am even aware, thanks to Montesquieu, that one can be a Persian. But, as for Man, I declare that I have never met him in my life. If he exists, I certainly have no knowledge of him. -- Joseph de Maistre, Considerations on France, 1797
On 11/22/20 2:54 AM, Matěj Cepl wrote:
The way to fix the "it doesn't build" or "no one cares and it's way old" problem is to have a policy to evict packages that fall into those categories, define the policy and automate it.
So, the only bugs we are interested in is that the package is so broken it doesn't even build. Are there any other bugs we care about (and which would require updating)?
Yes, security bugs above a certain severity, bugs that cause data loss or applications using the lib to crash or bugs that mean the library no longer functions in the way it was designed to (you might be able to add 1-2 more things to that list). Often a new version might just be a couple of minor fixes that only fix issues that very few people might see or might only affect certain platforms, equally it might just add a minor feature few people care about. If no one is reporting having issues with the bugs or caring enough about the new features to do the update does the package really need updating? I say this from the perspective of someone who maintains a couple of SLE packages that haven't changed in 15-20 years (Serial modems don't change alot). -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On 11/21/20 11:24 AM, Matěj Cepl wrote:
Robert Schweikert píše v Pá 20. 11. 2020 v 09:04 -0500:
But this problem is not solved by making up some rules that make it harder for people to contribute and ask for a "business reason" in an open source project that is community driven. "business reasons" apply to SLE not to openSUSE.
Sorry, but how many times I need to repeat that “business reason” was unfortunate phrasing before you stop quoting it again me?
The way to fix the "it doesn't build" or "no one cares and it's way old" problem is to have a policy to evict packages that fall into those categories, define the policy and automate it.
So, the only bugs we are interested in is that the package is so broken it doesn't even build. Are there any other bugs we care about (and which would require updating)?
And yes, exactly this policy we have,
Apparently we do not have that policy, or it is not enforced.
and the result is that quarter of packages in d:l:p need updating. Of course, there is an alternative to switch off monitoring, and just let packages rot without any concern from us as long as they at least build,
And what is wrong with not having the "latest and greatest"? If version 1 works for the people that use it and the package at version 1 does not cause a dependency problem why do we need to have version 2, 3, 4 or 5 ?
and touch anything only if somebody bothers to file a bug. That will make certainly OpenSUSE the paragon of relevant and up-to- date Linux distro.
Sounds like you feel we have to chase the tip, which is fair. But does that mean everyone else has to feel the same way and act accordingly? Later, Robert
Best,
Matěj
_______________________________________________ openSUSE Python mailing list -- python@lists.opensuse.org To unsubscribe, email python-leave@lists.opensuse.org List Netiquette: https://en.opensuse.org/openSUSE:Mailing_list_netiquette List Archives: https://lists.opensuse.org/archives/list/python@lists.opensuse.org
-- Robert Schweikert MAY THE SOURCE BE WITH YOU Distinguished Engineer LINUX Technical Team Lead Public Cloud rjschwei@suse.com IRC: robjo
Robert Schweikert píše v Po 23. 11. 2020 v 09:17 -0500:
Apparently we do not have that policy, or it is not enforced.
There are messages regularly send about failing packages.
And what is wrong with not having the "latest and greatest"?
We are talking about d:l:p and Factory (the word “development” is a hint). These are supposed to be the latest and greatest (without the bleeding edge with the stress on “bleeding” as was the situation with Fedora Rawhide). If you want mature and secure, then it is Leap (or SLE) we are talking about.
Sounds like you feel we have to chase the tip, which is fair. But does that mean everyone else has to feel the same way and act accordingly?
Yes, when we are talking about Factory. You are free to use SLE- 12 (sorry, Leap 42 is EOS), if you are conservative and don't want change. It is still supported. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 I didn't attend the funeral, but I sent a nice letter saying I approved of it. -- Mark Twain
On 11/23/20 3:18 PM, Matěj Cepl wrote:
Robert Schweikert píše v Po 23. 11. 2020 v 09:17 -0500:
Apparently we do not have that policy, or it is not enforced.
There are messages regularly send about failing packages.
Please read my message suggesting a policy again. I pointed out that we should have a clear direction about when stuff gets evicted. The messages about failing builds addresses only 1/2 the problem of "we have too many failing builds".
And what is wrong with not having the "latest and greatest"?
We are talking about d:l:p and Factory (the word “development” is a hint). These are supposed to be the latest and greatest (without the bleeding edge with the stress on “bleeding” as was the situation with Fedora Rawhide). If you want mature and secure, then it is Leap (or SLE) we are talking about.
And again, my previous message where I proposed a simple policy had some provision for "$SOME_ACCEPTABLE_TO_US_VERSION_DRIFT_TO_LATEST_UPSTREAM" Later, Robert
Sounds like you feel we have to chase the tip, which is fair. But does that mean everyone else has to feel the same way and act accordingly?
Yes, when we are talking about Factory. You are free to use SLE- 12 (sorry, Leap 42 is EOS), if you are conservative and don't want change. It is still supported.
Best,
Matěj
-- Robert Schweikert MAY THE SOURCE BE WITH YOU Distinguished Engineer LINUX Technical Team Lead Public Cloud rjschwei@suse.com IRC: robjo
On 11/24/20 7:04 AM, Robert Schweikert wrote:
On 11/23/20 3:18 PM, Matěj Cepl wrote:
Robert Schweikert píše v Po 23. 11. 2020 v 09:17 -0500:
Apparently we do not have that policy, or it is not enforced.
There are messages regularly send about failing packages.
Please read my message suggesting a policy again. I pointed out that we should have a clear direction about when stuff gets evicted.
The messages about failing builds addresses only 1/2 the problem of "we have too many failing builds".
And what is wrong with not having the "latest and greatest"?
We are talking about d:l:p and Factory (the word “development” is a hint). These are supposed to be the latest and greatest (without the bleeding edge with the stress on “bleeding” as was the situation with Fedora Rawhide). If you want mature and secure, then it is Leap (or SLE) we are talking about.
And again, my previous message where I proposed a simple policy had some provision for "$SOME_ACCEPTABLE_TO_US_VERSION_DRIFT_TO_LATEST_UPSTREAM"
The way I see it is we should actively encourage having the latest version, however, in my opinion having an older working version in most cases is better then having no version. Then having no version is better then having a "Broken" version, where no longer building is a pretty good indication of brokenness (especially where we have tests enabled). The other easy measurable indicator of brokenness is reports in bugzilla. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
-1 Am 20.11.20 um 08:17 schrieb Matěj Cepl:
Hello,
At the time of writing we have 538 packages in d:l:python which are out of date and require updating.
There are only two reasons why any Python package should be packaged for OpenSUSE: either it is dependency of another package in OpenSUSE, or we want to maintain it. Otherwise, if an user wants an unmaintained package, every user has pip available and they can install a package from PyPI directly.
They can do it in any case. Whether there is an outdated package in openSUSE or not. But the more you push people to PyPI (or Anaconda for d:l:python:numeric), those users will have unmaintainable ecosystems with all sorts of clutter - https://xkcd.com/1987/ . This is why I have always preferred to have an up to date, zypper managed, python sitelib. And it is the reason why I am a big proponent of the upcoming coinstallable python3 flavors.
1. Every new package submitted to d:l:p (or any other official OpenSUSE project) SHALL include in its submit request message “business reasons” for including the package into OpenSUSE (either because it is dependency of some other package, or some other reason, why it is needed).
Other than creating additional workload for both submitter and reviewers , I see no point. There is always a reason to submit a package: Someone want's to use it on their system and is so kind to share their work with the community.
2. Everybody who wants to submit new package to OpenSUSE, MUST submit two updates of packages already in Factory from the list delivered to this list every week.
I thought openSUSE was about encouraging volunteer work.
3. Packages which fail to build for sufficiently long time SHALL be removed from Factory and d:l:python (or moved to d:l:p:misc).
At least for Factory, this policy is already in place, isn't it? If I remember correctly, one of my first packages submitted ever (python-photutils) got removed from Factory a few weeks later, because it started to fail and I did not set up my e-mail notifications correctly. Ben
Ben Greiner píše v Pá 20. 11. 2020 v 12:19 +0100:
1. Every new package submitted to d:l:p (or any other official
OpenSUSE project) SHALL include in its submit request message “business reasons” for including the package into OpenSUSE (either because it is dependency of some other package, or some other reason, why it is needed).
Other than creating additional workload for both submitter and reviewers , I see no point.
500+ packages which need update. See my other email for more detailed discussion.
2. Everybody who wants to submit new package to OpenSUSE, MUST
submit two updates of packages already in Factory from the list delivered to this list every week.
I thought openSUSE was about encouraging volunteer work.
See other email for difference between monitored and non- monitored repositories. If you want to have your package in the monitored repos (like d:l:p, complete list is https://gitlab.com/mcepl/suse_cron_scripts/-/blob/master/scripts/environment... ), then you should put in some greasy oil and contribute to its maintenance.
At least for Factory, this policy is already in place, isn't it?
Except, I am not certain how much it is enforced. Best, and thank you very much for huge amount of work you did for us on python-rpm-macros. Very much appreciated. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 в чужой монастырь со своим уставом не ходят. -- Russian proverb (this time actually checked by a native Russian)
Hi, I have already submitted https://paste.opensuse.org/view/4252871, so shortly: I don't like the first two rules. IMO it does not correspond very well with the "open" philosophy we are trying to mantain, although they (on the other hand) reduce the cost and increase the benefit in the same time. And the second rule sounds openly hostile to me, although the condition is fairly easy to pass. What would you do if some package from d:l:p needded some other package "from fan" (which could be already in Factory, but does not have to)? Marketa
participants (9)
-
Ben Greiner -
Benjamin Greiner -
Dan Čermák -
Dirk Müller -
Marketa Machova -
Matěj Cepl -
Michael Ströder -
Robert Schweikert -
Simon Lees