Hi All,
It likely comes as no surprise to anyone one that the groups in spec file issue was raised with the board (one of these was on a public list). In this specific email we are acting in our role of "helping resolve conflicts" complaints were also made directly about the behavior of certain individuals (again including on this list). The board has chosen to deal with these issues in private as is our general approach to such issues.
Firstly the board generally agrees that in the following thread [1] there was a plan and general agreement to follow it for the removal of groups if certain other issues were resolved. As such we believe they acted with good intentions when starting to remove group support from various places such as Yast, rpmlint and spec cleaner.
The board decided to wait until the discussion on new proposals settled down, before going forward with the issue, at the time we had the discussion we believed this to be the case.
Of all the proposals discussed the board agreed that the one submitted by Ludwig [2], to move groups into the repository meta data is technically the best solution and the one most broadly supported by contributors. As such we welcome any contributions from the community towards getting this working and integrated into tooling where it makes sense such as Yast and software.opensuse.org
Given that our core tooling no longer displays groups and that the board has a copy of all the existing groups that can be used as a starting point for the new system the board has decided that including groups in spec files should now be optional with the final decision resting with the maintainer as some maintainers may still use there spec files in distro's that still support groups although we believe most packages going into factory wont.
We realize that this decision will not make everyone happy but hopefully it eventually leads to better group support in the future.
1. https://lists.opensuse.org/opensuse-factory/2018-05/msg00460.html 2. https://lists.opensuse.org/opensuse-factory/2019-10/msg00263.html
Am Mittwoch, 30. Oktober 2019, 06:06:39 CET schrieb Simon Lees:
Hi All,
It likely comes as no surprise to anyone one that the groups in spec file issue was raised with the board (one of these was on a public list). In this specific email we are acting in our role of "helping resolve conflicts" complaints were also made directly about the behavior of certain individuals (again including on this list). The board has chosen to deal with these issues in private as is our general approach to such issues.
Firstly the board generally agrees that in the following thread [1] there was a plan and general agreement to follow it for the removal of groups if certain other issues were resolved. As such we believe they acted with good intentions when starting to remove group support from various places such as Yast, rpmlint and spec cleaner.
The board decided to wait until the discussion on new proposals settled down, before going forward with the issue, at the time we had the discussion we believed this to be the case.
Of all the proposals discussed the board agreed that the one submitted by Ludwig [2], to move groups into the repository meta data is technically the best solution and the one most broadly supported by contributors. As such we welcome any contributions from the community towards getting this working and integrated into tooling where it makes sense such as Yast and software.opensuse.org
Given that our core tooling no longer displays groups and that the board has a copy of all the existing groups that can be used as a starting point for the new system the board has decided that including groups in spec files should now be optional with the final decision resting with the maintainer as some maintainers may still use there spec files in distro's that still support groups although we believe most packages going into factory wont.
We realize that this decision will not make everyone happy but hopefully it eventually leads to better group support in the future.
Let me translate this to colloquial language:
The people, that didn't care a fuzz about grouping packages and even tried to create sneaky precedents to get rid of them, decide to bury the whole topic in the backyard, where those, that care, won't dig it up again.
It's a semi smart way to phase out an uncomfortable discussion (to put it politely).
An external management of package grouping is doomed to die, as the amount of work will surpass it's usefulness.
Well, in fact, if you reread the answers to Ludwig's proposal, you find mostly refusals or warnings to take that route.
In the Pro-Grouping-Tag camp, we tried to find an easy to provide solution, that actually would *add* value to the distribution as a whole, and would set it apart for others *without* *much* *fuzz*.
With a tag based grouping scheme in the rpm spec, rpmlint could *recommend* adding tags to the packager, if missing or in the old format. A tag based grouping would allow YaST to restore the group view in a much more useful way, that actually would help our users, especially the *new* ones, to find their way though the overwhelming number of packages.
I remember times in the past, where I browsed through these groups in YaST to explore the hidden features, that were all in this *magical* box. And now, that we talk about, the existing group view is still great. The "special" groups are especially nice.
What you're trying to enforce is something we mostly have already: patterns, but they depend on preferences of a few maintainers. The group view entries could be extended to trees from the tags, that could lead to a nice way to navigate though *all* packages in a smart way, while keeping the grouping where it belongs, in the hand of packagers (that care). I'm sure, that most packagers will welcome a polite notice from rpmlint, if it helps users to locate their packages.
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
Regards, Pete
Am 31.10.19 um 14:41 schrieb Hans-Peter Jansen:
Let me translate this to colloquial language:
The people, that didn't care a fuzz about grouping packages and even tried to create sneaky precedents to get rid of them, decide to bury the whole topic in the backyard, where those, that care, won't dig it up again.
Thanks Pete, I could not have worded this in a similar polite way.
It's a semi smart way to phase out an uncomfortable discussion (to put it politely).
An external management of package grouping is doomed to die, as the amount of work will surpass it's usefulness.
Well, in fact, if you reread the answers to Ludwig's proposal, you find mostly refusals or warnings to take that route.
In the Pro-Grouping-Tag camp, we tried to find an easy to provide solution, that actually would *add* value to the distribution as a whole, and would set it apart for others *without* *much* *fuzz*.
[ snipped lots of IMHO correct points ]
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
I wholeheartedly agree.
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
Am Mittwoch, 30. Oktober 2019, 06:06:39 CET schrieb Simon Lees:
Hi All,
It likely comes as no surprise to anyone one that the groups in spec file issue was raised with the board (one of these was on a public list). In this specific email we are acting in our role of "helping resolve conflicts" complaints were also made directly about the behavior of certain individuals (again including on this list). The board has chosen to deal with these issues in private as is our general approach to such issues.
Firstly the board generally agrees that in the following thread [1] there was a plan and general agreement to follow it for the removal of groups if certain other issues were resolved. As such we believe they acted with good intentions when starting to remove group support from various places such as Yast, rpmlint and spec cleaner.
The board decided to wait until the discussion on new proposals settled down, before going forward with the issue, at the time we had the discussion we believed this to be the case.
Of all the proposals discussed the board agreed that the one submitted by Ludwig [2], to move groups into the repository meta data is technically the best solution and the one most broadly supported by contributors. As such we welcome any contributions from the community towards getting this working and integrated into tooling where it makes sense such as Yast and software.opensuse.org
Given that our core tooling no longer displays groups and that the board has a copy of all the existing groups that can be used as a starting point for the new system the board has decided that including groups in spec files should now be optional with the final decision resting with the maintainer as some maintainers may still use there spec files in distro's that still support groups although we believe most packages going into factory wont.
We realize that this decision will not make everyone happy but hopefully it eventually leads to better group support in the future.
Let me translate this to colloquial language:
The people, that didn't care a fuzz about grouping packages and even tried to create sneaky precedents to get rid of them, decide to bury the whole topic in the backyard, where those, that care, won't dig it up again.
It's a semi smart way to phase out an uncomfortable discussion (to put it politely).
An external management of package grouping is doomed to die, as the amount of work will surpass it's usefulness.
Many consider that the state the internal management of groups would equally indicate that it had failed.
Well, in fact, if you reread the answers to Ludwig's proposal, you find mostly refusals or warnings to take that route.
Between them there are suggestions from people who are either heavily involved or that have been heavily involved in release management agreeing that it would be a better solution for them to be external, including reasons that it would be technically Superior such as translations.
Beyond that we have years of evidence that leaving it to maintainers, many of who don't strongly care has lead to a big mess.
In the Pro-Grouping-Tag camp, we tried to find an easy to provide solution, that actually would *add* value to the distribution as a whole, and would set it apart for others *without* *much* *fuzz*.
Well I am also in the "Pro-Grouping-Tag camp" I just don't think doing it in spec files is the best way.
With a tag based grouping scheme in the rpm spec, rpmlint could *recommend* adding tags to the packager, if missing or in the old format. A tag based grouping would allow YaST to restore the group view in a much more useful way, that actually would help our users, especially the *new* ones, to find their way though the overwhelming number of packages.
The proposed external solution would also allow yast to display groups in this equally useful way. However the decision was made over a year back that with the state groups were in then the group view was actually more harmful confusing to users because often groups were missing programs that someone would reasonably expect to be there. Not to mention the significant number of packages such as system libraries that users should never need to install on there own.
I remember times in the past, where I browsed through these groups in YaST to explore the hidden features, that were all in this *magical* box. And now, that we talk about, the existing group view is still great. The "special" groups are especially nice.
I equally remember the feature being useful at some pont.
What you're trying to enforce is something we mostly have already: patterns, but they depend on preferences of a few maintainers. The group view entries could be extended to trees from the tags, that could lead to a nice way to navigate though *all* packages in a smart way, while keeping the grouping where it belongs, in the hand of packagers (that care). I'm sure, that most packagers will welcome a polite notice from rpmlint, if it helps users to locate their packages.
As someone who maintains a number of patterns, I see them as very different things, for example we no longer have patterns for things like media players or text editors, because most users only want one installed. A pattern that installs 10 text editors is not useful where as a group view that shows text editors is. Having said that I am always after suggestions on how to improve patterns so if you have them let me know.
As such I'd also like to see groups implemented in a way that is useful for users but currently there are also many other things i'd like to see and they are much higher on my todo list.
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
Well the plan would be to store the meta data in external resources because it makes more technical sense there due to the limitation of spec files, but then intergrate the display of the data into the right parts of openSUSE such as yast and software.opensuse.org
On Thu, Oct 31, 2019 at 8:13 PM Simon Lees sflees@suse.de wrote:
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
Well the plan would be to store the meta data in external resources because it makes more technical sense there due to the limitation of spec files, but then intergrate the display of the data into the right parts of openSUSE such as yast and software.opensuse.org
I guess we could use the comps.xml rpm metadata extension and define groups that way? We'd even get an easy way to translate them...
Am 01.11.19 um 01:12 schrieb Simon Lees:
Many consider that the state the internal management of groups would equally indicate that it had failed.
[...]
Beyond that we have years of evidence that leaving it to maintainers, many of who don't strongly care has lead to a big mess.
Die the idea ever come to your mind that this crazy rpmlint check that forced people to select totally useless groups might have a slight impact on the creation of this "big mess" and the resulting "failure"?
I guess that not many people will argue that the previous group system was good. Especially not the selection of the available and usable groups.
Jan / we are discussing a different approach now (just in case you have not noticed yet).
On 11/1/19 6:11 PM, Stefan Seyfried wrote:
Am 01.11.19 um 01:12 schrieb Simon Lees:
Many consider that the state the internal management of groups would equally indicate that it had failed.
[...]
Beyond that we have years of evidence that leaving it to maintainers, many of who don't strongly care has lead to a big mess.
Die the idea ever come to your mind that this crazy rpmlint check that forced people to select totally useless groups might have a slight impact on the creation of this "big mess" and the resulting "failure"?
Yep
I guess that not many people will argue that the previous group system was good. Especially not the selection of the available and usable groups.
Jan / we are discussing a different approach now (just in case you have not noticed yet).
I am well aware of the discussions of new approaches, and it seemed to us that the new approaches would be best implemented if they stored there data somewhere else that among other things is translatable. But I get the feeling that i'm starting to have to repeat myself here.
Am Freitag, 1. November 2019, 01:12:55 CET schrieb Simon Lees:
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
An external management of package grouping is doomed to die, as the amount of work will surpass it's usefulness.
Many consider that the state the internal management of groups would equally indicate that it had failed.
I'm inclined to disagree.
Well, in fact, if you reread the answers to Ludwig's proposal, you find mostly refusals or warnings to take that route.
Between them there are suggestions from people who are either heavily involved or that have been heavily involved in release management agreeing that it would be a better solution for them to be external, including reasons that it would be technically Superior such as translations.
Well, managing groups outside of specs, I can see a slight improvement in translation handling, but the comes with an even higher prize of disjoint group management.
What technical reasons hinders the translation of a collection of tags exactly?
Beyond that we have years of evidence that leaving it to maintainers, many of who don't strongly care has lead to a big mess.
Again, I think, you devaluate the existing system. It is lacking, but the existing system isn't in a such a bad shape, that some of you want to color it. As shown in this discussion, more than 90% of group specs are fine.
In the Pro-Grouping-Tag camp, we tried to find an easy to provide solution, that actually would *add* value to the distribution as a whole, and would set it apart for others *without* *much* *fuzz*.
Well I am also in the "Pro-Grouping-Tag camp" I just don't think doing it in spec files is the best way.
The thing, that matters most here is, the existing system can be easily transformed into something hugely useful. Pushing it outside the packages itself creates a load to already overloaded package maintainers. Combined with explicitly and implicitly expressed general desinterest is the reason for me claiming this approach being "doomed to die".
With a tag based grouping scheme in the rpm spec, rpmlint could *recommend* adding tags to the packager, if missing or in the old format. A tag based grouping would allow YaST to restore the group view in a much more useful way, that actually would help our users, especially the *new* ones, to find their way though the overwhelming number of packages.
The proposed external solution would also allow yast to display groups in this equally useful way. However the decision was made over a year back that with the state groups were in then the group view was actually more harmful confusing to users because often groups were missing programs that someone would reasonably expect to be there. Not to mention the significant number of packages such as system libraries that users should never need to install on there own.
Well, the value of grouping tags reaches way beyond simple installation tasks. In fact, I often want to lookup libraries for dependency resolution, where such a library tag could improve lookups, because one never no, is this library called libsomething or something, or something totally different.. All of us were snared by that trap already...
BTW, I really dream from a more specific/versatile search in OBS and grouping tags could help here as well.
[...]
What you're trying to enforce is something we mostly have already: patterns, but they depend on preferences of a few maintainers. The group view entries could be extended to trees from the tags, that could lead to a nice way to navigate though *all* packages in a smart way, while keeping the grouping where it belongs, in the hand of packagers (that care). I'm sure, that most packagers will welcome a polite notice from rpmlint, if it helps users to locate their packages.
As someone who maintains a number of patterns, I see them as very different things, for example we no longer have patterns for things like media players or text editors, because most users only want one installed. A pattern that installs 10 text editors is not useful where as a group view that shows text editors is. Having said that I am always after suggestions on how to improve patterns so if you have them let me know.
Don't take me wrong here, patterns are fine as such! They just don't show the full picture. With an improved grouping, we might even be able to reduce the sheer number of them, making them even more useful..
As such I'd also like to see groups implemented in a way that is useful for users but currently there are also many other things i'd like to see and they are much higher on my todo list.
Since we all suffer from limited resources, this is also the most critical argument for a pragmatic approach here.
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
Well the plan would be to store the meta data in external resources because it makes more technical sense there due to the limitation of spec files, but then intergrate the display of the data into the right parts of openSUSE such as yast and software.opensuse.org
Okay Simon, thanks for the valuable discussion.
Let's get a bit more technical. Where is this data supposed to be stored? What's the proposed workflow for an average packager (that cares)? How will this being made available in the different areas, where it's needed? (YaST, OBS, software.o.o, ...)
A typical case: I realize, that one of my packages is missing in a group. A single commit later, I'm done. With the external approach, many more hops needs to be taken, before a package is classified correctly.
The most outstanding argument against grouping, as it exists today, is the single valuedness.. Why don't we combine the proposals to centralize the management of tags, easing the work of the translation "department", providing an interface for creating proposals, and assigning these standardized tags in the spec. Translation is a none issue, it doesn't need any rpm bending, but the web interface, which can be bridged with some wiki list meanwhile.
I've created a lot of packages myself (approx. approaching 3 digit numbers), and just copying a line in a spec file is *way* more convenient than anything else (external or even just using an additional file in the package).
The reason, why Ludwig suggested the external approach was mainly avoiding the "fight" over tags. But Christian rightfully turned out, that this would be a good thing, because it would show, that people care, and I agree here.
We need to tie those loose ends together, that makes groups great again, with minimal load to the involved people, because we need to accept, that we have very limited resources to do so, only.
Cheers, Pete
On 11/1/19 11:18 PM, Hans-Peter Jansen wrote:
Am Freitag, 1. November 2019, 01:12:55 CET schrieb Simon Lees:
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
An external management of package grouping is doomed to die, as the amount of work will surpass it's usefulness.
Many consider that the state the internal management of groups would equally indicate that it had failed.
I'm inclined to disagree.
Well, in fact, if you reread the answers to Ludwig's proposal, you find mostly refusals or warnings to take that route.
Between them there are suggestions from people who are either heavily involved or that have been heavily involved in release management agreeing that it would be a better solution for them to be external, including reasons that it would be technically Superior such as translations.
Well, managing groups outside of specs, I can see a slight improvement in translation handling, but the comes with an even higher prize of disjoint group management.
What technical reasons hinders the translation of a collection of tags exactly?
Generally translations are stored in the binary alongside the original strings or in a separate file that ships with the binary that it loads and uses rpm doesn't really have support for handling translations in its metadata in any form and if it did our the software our translation team use isn't set up to search across obs to find translation strings in all the spec files. Where as if we do groups externally its easy to intergrate into our existing translation software.
In the Pro-Grouping-Tag camp, we tried to find an easy to provide solution, that actually would *add* value to the distribution as a whole, and would set it apart for others *without* *much* *fuzz*.
Well I am also in the "Pro-Grouping-Tag camp" I just don't think doing it in spec files is the best way.
The thing, that matters most here is, the existing system can be easily transformed into something hugely useful. Pushing it outside the packages itself creates a load to already overloaded package maintainers. Combined with explicitly and implicitly expressed general desinterest is the reason for me claiming this approach being "doomed to die".
It has become quite clear that most package maintainers aren't really interested in maintaining groups, having it externally means that more people who care about groups can help without having to learn about packaging.
With a tag based grouping scheme in the rpm spec, rpmlint could *recommend* adding tags to the packager, if missing or in the old format. A tag based grouping would allow YaST to restore the group view in a much more useful way, that actually would help our users, especially the *new* ones, to find their way though the overwhelming number of packages.
The proposed external solution would also allow yast to display groups in this equally useful way. However the decision was made over a year back that with the state groups were in then the group view was actually more harmful confusing to users because often groups were missing programs that someone would reasonably expect to be there. Not to mention the significant number of packages such as system libraries that users should never need to install on there own.
Well, the value of grouping tags reaches way beyond simple installation tasks. In fact, I often want to lookup libraries for dependency resolution, where such a library tag could improve lookups, because one never no, is this library called libsomething or something, or something totally different.. All of us were snared by that trap already...
Well the whole reason we have a package management system is so that general users don't need to deal with manual dependency resolution, for developers stuff like `rpm -q --whatprovides "pkgconfig(dbus-1)"` is a far quicker and easier way then searching via groups.
BTW, I really dream from a more specific/versatile search in OBS and grouping tags could help here as well.
[...]
What you're trying to enforce is something we mostly have already: patterns, but they depend on preferences of a few maintainers. The group view entries could be extended to trees from the tags, that could lead to a nice way to navigate though *all* packages in a smart way, while keeping the grouping where it belongs, in the hand of packagers (that care). I'm sure, that most packagers will welcome a polite notice from rpmlint, if it helps users to locate their packages.
As someone who maintains a number of patterns, I see them as very different things, for example we no longer have patterns for things like media players or text editors, because most users only want one installed. A pattern that installs 10 text editors is not useful where as a group view that shows text editors is. Having said that I am always after suggestions on how to improve patterns so if you have them let me know.
Don't take me wrong here, patterns are fine as such! They just don't show the full picture. With an improved grouping, we might even be able to reduce the sheer number of them, making them even more useful..
As such I'd also like to see groups implemented in a way that is useful for users but currently there are also many other things i'd like to see and they are much higher on my todo list.
Since we all suffer from limited resources, this is also the most critical argument for a pragmatic approach here.
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
Well the plan would be to store the meta data in external resources because it makes more technical sense there due to the limitation of spec files, but then intergrate the display of the data into the right parts of openSUSE such as yast and software.opensuse.org
Okay Simon, thanks for the valuable discussion.
Let's get a bit more technical. Where is this data supposed to be stored? What's the proposed workflow for an average packager (that cares)?
This will be for those implementing the solution to decide.
How will this being made available in the different areas, where it's needed? (YaST, OBS, software.o.o, ...)
Again it will be up to those interested in having the feature to work with those teams to get it implemented or do the work themselves. It should also be pointed out that currently none of those places have support for the old way of doing groups so this would need to be done regardless.
On Sat, Nov 02, 2019 at 10:41:37AM +1030, Simon Lees wrote:
On 11/1/19 11:18 PM, Hans-Peter Jansen wrote:
Am Freitag, 1. November 2019, 01:12:55 CET schrieb Simon Lees:
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
An external management of package grouping is doomed to die, as the amount of work will surpass it's usefulness.
Many consider that the state the internal management of groups would equally indicate that it had failed.
I'm inclined to disagree.
Well, in fact, if you reread the answers to Ludwig's proposal, you find mostly refusals or warnings to take that route.
Between them there are suggestions from people who are either heavily involved or that have been heavily involved in release management agreeing that it would be a better solution for them to be external, including reasons that it would be technically Superior such as translations.
Well, managing groups outside of specs, I can see a slight improvement in translation handling, but the comes with an even higher prize of disjoint group management.
What technical reasons hinders the translation of a collection of tags exactly?
Generally translations are stored in the binary alongside the original strings or in a separate file that ships with the binary that it loads and uses rpm doesn't really have support for handling translations in its metadata in any form and if it did our the software our translation team use isn't set up to search across obs to find translation strings in all the spec files. Where as if we do groups externally its easy to intergrate into our existing translation software.
But we can have data on all binaries collected so cnf can do its job.
Can't we have package tags collected in the same way so that software that works with individual packages can use the pacakge metadata and software that works with the repositories and uninstalled packages the repository metadata?
Surely that will make translation possible as well.
In the Pro-Grouping-Tag camp, we tried to find an easy to provide solution, that actually would *add* value to the distribution as a whole, and would set it apart for others *without* *much* *fuzz*.
Well I am also in the "Pro-Grouping-Tag camp" I just don't think doing it in spec files is the best way.
The thing, that matters most here is, the existing system can be easily transformed into something hugely useful. Pushing it outside the packages itself creates a load to already overloaded package maintainers. Combined with explicitly and implicitly expressed general desinterest is the reason for me claiming this approach being "doomed to die".
It has become quite clear that most package maintainers aren't really interested in maintaining groups, having it externally means that more people who care about groups can help without having to learn about packaging.
The size of the out-of-package group index we have now tells quite different story as has been pointed out several times already. Will removing the group information from the spec files magically create hosts of people who do care about external metadata?
I don't think so.
With a tag based grouping scheme in the rpm spec, rpmlint could *recommend* adding tags to the packager, if missing or in the old format. A tag based grouping would allow YaST to restore the group view in a much more useful way, that actually would help our users, especially the *new* ones, to find their way though the overwhelming number of packages.
The proposed external solution would also allow yast to display groups in this equally useful way. However the decision was made over a year back that with the state groups were in then the group view was actually more harmful confusing to users because often groups were missing programs that someone would reasonably expect to be there. Not to mention the significant number of packages such as system libraries that users should never need to install on there own.
Well, the value of grouping tags reaches way beyond simple installation tasks. In fact, I often want to lookup libraries for dependency resolution, where such a library tag could improve lookups, because one never no, is this library called libsomething or something, or something totally different.. All of us were snared by that trap already...
Well the whole reason we have a package management system is so that general users don't need to deal with manual dependency resolution, for developers stuff like `rpm -q --whatprovides "pkgconfig(dbus-1)"` is a far quicker and easier way then searching via groups.
Only if you know the name of the package. If you want a library that does something and want to pick one of the available options groups are useful.
BTW, I really dream from a more specific/versatile search in OBS and grouping tags could help here as well.
[...]
What you're trying to enforce is something we mostly have already: patterns, but they depend on preferences of a few maintainers. The group view entries could be extended to trees from the tags, that could lead to a nice way to navigate though *all* packages in a smart way, while keeping the grouping where it belongs, in the hand of packagers (that care). I'm sure, that most packagers will welcome a polite notice from rpmlint, if it helps users to locate their packages.
As someone who maintains a number of patterns, I see them as very different things, for example we no longer have patterns for things like media players or text editors, because most users only want one installed. A pattern that installs 10 text editors is not useful where as a group view that shows text editors is. Having said that I am always after suggestions on how to improve patterns so if you have them let me know.
Don't take me wrong here, patterns are fine as such! They just don't show the full picture. With an improved grouping, we might even be able to reduce the sheer number of them, making them even more useful..
As such I'd also like to see groups implemented in a way that is useful for users but currently there are also many other things i'd like to see and they are much higher on my todo list.
Since we all suffer from limited resources, this is also the most critical argument for a pragmatic approach here.
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
Well the plan would be to store the meta data in external resources because it makes more technical sense there due to the limitation of spec files, but then intergrate the display of the data into the right parts of openSUSE such as yast and software.opensuse.org
Okay Simon, thanks for the valuable discussion.
Let's get a bit more technical. Where is this data supposed to be stored? What's the proposed workflow for an average packager (that cares)?
This will be for those implementing the solution to decide.
So it'a all vaporware. You trade something not perfect for something completely non-existent. That does not sound like a good tradeoff to me.
Thanks
Michal
On Sun, Nov 3, 2019 at 16:42, Michal Suchánek msuchanek@suse.de wrote:
On Sat, Nov 02, 2019 at 10:41:37AM +1030, Simon Lees wrote:
On 11/1/19 11:18 PM, Hans-Peter Jansen wrote:
Am Freitag, 1. November 2019, 01:12:55 CET schrieb Simon Lees:
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
An external management of package grouping is doomed to die, as
the amount
of work will surpass it's usefulness.
Many consider that the state the internal management of groups
would
equally indicate that it had failed.
I'm inclined to disagree.
Well, in fact, if you reread the answers to Ludwig's proposal,
you find
mostly refusals or warnings to take that route.
Between them there are suggestions from people who are either
heavily
involved or that have been heavily involved in release management agreeing that it would be a better solution for them to be
external,
including reasons that it would be technically Superior such as translations.
Well, managing groups outside of specs, I can see a slight
improvement in
translation handling, but the comes with an even higher prize of
disjoint
group management.
What technical reasons hinders the translation of a collection of
tags
exactly?
Generally translations are stored in the binary alongside the original strings or in a separate file that ships with the binary that it loads and uses rpm doesn't really have support for handling translations in its metadata in any form and if it did our the software our translation team use isn't set up to search across obs to find translation strings in all the spec files. Where as if we do groups externally its easy to intergrate into our existing translation software.
But we can have data on all binaries collected so cnf can do its job.
That's not really needed long term, we can use the createrepo_c's filelist file (which is automatically generated for every obs repository) to see what package has files in /usr/bin or /bin (+sbin). We don't use that anywhere really, if we did, we could use packagekit's cnf instead of scout, and search any and all arbitrary files within remote and local packages. Currently the only files we can search are capabilities and locally installed rpms, which is very limiting.
LCP [Stasiek] https://lcp.world
On 11/4/19 2:12 AM, Michal Suchánek wrote:
On Sat, Nov 02, 2019 at 10:41:37AM +1030, Simon Lees wrote:
On 11/1/19 11:18 PM, Hans-Peter Jansen wrote:
Am Freitag, 1. November 2019, 01:12:55 CET schrieb Simon Lees:
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
In the Pro-Grouping-Tag camp, we tried to find an easy to provide solution, that actually would *add* value to the distribution as a whole, and would set it apart for others *without* *much* *fuzz*.
Well I am also in the "Pro-Grouping-Tag camp" I just don't think doing it in spec files is the best way.
The thing, that matters most here is, the existing system can be easily transformed into something hugely useful. Pushing it outside the packages itself creates a load to already overloaded package maintainers. Combined with explicitly and implicitly expressed general desinterest is the reason for me claiming this approach being "doomed to die".
It has become quite clear that most package maintainers aren't really interested in maintaining groups, having it externally means that more people who care about groups can help without having to learn about packaging.
The size of the out-of-package group index we have now tells quite different story as has been pointed out several times already. Will removing the group information from the spec files magically create hosts of people who do care about external metadata?
I don't think so.
Still maybe it will be more then the limited number of packagers interested in it at the moment, making things easier for others is only going to help and the reality is if there isn't enough people to maintain the groups externally there likely isn't enough to maintain it internally which is why this discussion started 15 months back.
With a tag based grouping scheme in the rpm spec, rpmlint could *recommend* adding tags to the packager, if missing or in the old format. A tag based grouping would allow YaST to restore the group view in a much more useful way, that actually would help our users, especially the *new* ones, to find their way though the overwhelming number of packages.
The proposed external solution would also allow yast to display groups in this equally useful way. However the decision was made over a year back that with the state groups were in then the group view was actually more harmful confusing to users because often groups were missing programs that someone would reasonably expect to be there. Not to mention the significant number of packages such as system libraries that users should never need to install on there own.
Well, the value of grouping tags reaches way beyond simple installation tasks. In fact, I often want to lookup libraries for dependency resolution, where such a library tag could improve lookups, because one never no, is this library called libsomething or something, or something totally different.. All of us were snared by that trap already...
Well the whole reason we have a package management system is so that general users don't need to deal with manual dependency resolution, for developers stuff like `rpm -q --whatprovides "pkgconfig(dbus-1)"` is a far quicker and easier way then searching via groups.
Only if you know the name of the package. If you want a library that does something and want to pick one of the available options groups are useful.
As I said in an earlier discussion if I want to pick a development library I need to know stuff about API's, documentation etc which makes google a far better starting place then group tags.
This decision on the other hand will lead to a less approachable product in the end. It "sells" under a fair value already, because many non mainstream abilities are hard to find without resorting to external resources.
Well the plan would be to store the meta data in external resources because it makes more technical sense there due to the limitation of spec files, but then intergrate the display of the data into the right parts of openSUSE such as yast and software.opensuse.org
Okay Simon, thanks for the valuable discussion.
Let's get a bit more technical. Where is this data supposed to be stored? What's the proposed workflow for an average packager (that cares)?
This will be for those implementing the solution to decide.
So it'a all vaporware. You trade something not perfect for something completely non-existent. That does not sound like a good tradeoff to me.
Well given that none of openSUSE's integrated tools currently have integration with the old group system id say that its equally vaporware in its current state.
Am Sonntag, 3. November 2019, 16:42:29 CET schrieb Michal Suchánek:
On Sat, Nov 02, 2019 at 10:41:37AM +1030, Simon Lees wrote:
It has become quite clear that most package maintainers aren't really interested in maintaining groups, having it externally means that more people who care about groups can help without having to learn about packaging.
Again, as repeated a couple of times, packager didn't care, because: * the single tree value scheme was lacking * rpmlint enforcements suck rocks (ask Seife!) * the former issue excellerates, if it's based on non essential reasons (technical or other)
Well the whole reason we have a package management system is so that general users don't need to deal with manual dependency resolution, for developers stuff like `rpm -q --whatprovides "pkgconfig(dbus-1)"` is a far quicker and easier way then searching via groups.
Only if you know the name of the package. If you want a library that does something and want to pick one of the available options groups are useful.
And extends to queries, where you don't know *anything* about the lib{,s existence} beforehand. (Does this lib exist, or do I need to build myself?)
I claim, that simple tag based group information in specs, integrated into OBS advanced search, would raise their value massively (and provides a further reason for packagers to fill this simple field adequately).
OTOH, relocating/transforming them into something external, non-existing by now and in the near future, it *will never fly*. Sorry, but it's that simple, Simon.
Jan's et.al. approach is coming from the practicability side of things, while the Board tries to establish an artificial top down approach, that depend on a lot of voluntary work, but nobody indicated any interest to do that. Doomed...
It *will* work the other way around:
Provide a simple, yet flexible group tag scheme (DONE). *Suggest* filling the field with rpmlint (TBD, easy). Explain packagers, why it is *advantageous* to fill this field (even if those services doesn't exist, yet) (TBD, easy). Everything else will follow, because the basics are sound. This is a cooperative approach, that has a chance to work, with a low impact on the workflow of packagers.
I don't see translations of those tags as something critical, I would even leave that to later generations, but they're possible, of course. Groups, if they survive this dispute, will keep to be a "professional tool". Translations don't *add* much value here, it might even decrease their usefulness, if queries doesn't look up the English terms as well! They have cosmetic value at most.
Cheers, Pete
Simon Lees schrieb:
On 11/1/19 12:11 AM, Hans-Peter Jansen wrote:
[...] Well, in fact, if you reread the answers to Ludwig's proposal, you find mostly refusals or warnings to take that route.
Between them there are suggestions from people who are either heavily involved or that have been heavily involved in release management agreeing that it would be a better solution for them to be external, including reasons that it would be technically Superior such as translations.
Me mentioning translations was related to the way how we can add data to repo xml that was not extracted from RPMs but collected from external data sources. It has nothing to do with technical superiority. There assets and drawbacks either way.
In fact the group tags are and always have been translated: https://l10n.opensuse.org/projects/yast-rpm-groups/master/
The extraction mechanism was stale at times and the yast implementation broken but the data is there. Conveniently split at slashes so to translators this looked like individual tags already.
cu Ludwig
Em qua, 30 de out de 2019 às 02:07, Simon Lees sflees@suse.de escreveu:
We realize that this decision will not make everyone happy but hopefully it eventually leads to better group support in the future.
Hi,
Is this kind of technical decision in the scope of the Board?
[0] says: "The board should provide guidance and support existing governance structures, but shouldn't direct or control development, since community mechanisms exist to accomplish the goals of the project. The board should document decisions and policies."
Regards,
Luiz
[0] https://en.opensuse.org/openSUSE:Board
On 11/1/19 1:29 AM, Luiz Fernando Ranghetti wrote:
Em qua, 30 de out de 2019 às 02:07, Simon Lees sflees@suse.de escreveu:
We realize that this decision will not make everyone happy but hopefully it eventually leads to better group support in the future.
Hi,
Is this kind of technical decision in the scope of the Board?
[0] says: "The board should provide guidance and support existing governance structures, but shouldn't direct or control development, since community mechanisms exist to accomplish the goals of the project. The board should document decisions and policies."
If the matter had not been raised with the board then it would not be within it's scope to deal with. However given that this issue was raised with the board by parties with strong views on both sides as well as people caught in the middle unable to do there job "resolving the conflict" fits within the board's role. In this case the board decided cleanest solution to the conflict was to recommend a technical solution.
The board is still somewhat limited in its power here, in that we can not compel people to do work, we can only tell them not to do certain things which generally is something the board tries to avoid. For example we can state that groups shall be optional and up to the maintainer because that is not forcing anyone to do work, likewise if we felt it was the best solution we could have told maintainers that submissions that drop the group field won't be accepted.
What we can't do though Is tell everyone that they need to go through and remove groups likewise we can't tell the yast team that they need to re add support for groups or tell someone that they must go and implement something in a certain way. All of these actions would force someone to do work and we don't have that power.
What we can do though is recommend that if people are interested in groups, "the following would be a good way" but whether anyone chooses to go away and implement that is a completely different thing and they may choose to do something different which would also be fine. But in this case the fact that there is a better solution played a role in our (well at least my decision to support the proposal the board put forward).
On Friday 2019-11-01 00:38, Simon Lees wrote:
The board is still somewhat limited in its power here, in that we can not compel people to do work, we can only tell them not to do certain things which generally is something the board tries to avoid. For example we can state that groups shall be optional and up to the maintainer because that is not forcing anyone to do work,
Well, you could spin the argument either way: Removing the Group line and moving it to a separate dataset is work to construct this new format.
On 11/1/19 7:13 PM, Jan Engelhardt wrote:
On Friday 2019-11-01 00:38, Simon Lees wrote:
The board is still somewhat limited in its power here, in that we can not compel people to do work, we can only tell them not to do certain things which generally is something the board tries to avoid. For example we can state that groups shall be optional and up to the maintainer because that is not forcing anyone to do work,
Well, you could spin the argument either way: Removing the Group line and moving it to a separate dataset is work to construct this new format.
Yep it is work, in the same way that keeping the data up to date is work or maintaining a package is work, people can choose not to do this work like they can choose not to do any other work.
On Wed, 2019-10-30 at 15:36 +1030, Simon Lees wrote:
Hi All,
It likely comes as no surprise to anyone one that the groups in spec file issue was raised with the board (one of these was on a public list). In this specific email we are acting in our role of "helping resolve conflicts" complaints were also made directly about the behavior of certain individuals (again including on this list). The board has chosen to deal with these issues in private as is our general approach to such issues.
Firstly the board generally agrees that in the following thread [1] there was a plan and general agreement to follow it for the removal of groups if certain other issues were resolved. As such we believe they acted with good intentions when starting to remove group support from various places such as Yast, rpmlint and spec cleaner.
The board decided to wait until the discussion on new proposals settled down, before going forward with the issue, at the time we had the discussion we believed this to be the case.
Of all the proposals discussed the board agreed that the one submitted by Ludwig [2], to move groups into the repository meta data is technically the best solution and the one most broadly supported by contributors. As such we welcome any contributions from the community towards getting this working and integrated into tooling where it makes sense such as Yast and software.opensuse.org
Given that our core tooling no longer displays groups and that the board has a copy of all the existing groups that can be used as a starting point for the new system the board has decided that including groups in spec files should now be optional with the final decision resting with the maintainer as some maintainers may still use there spec files in distro's that still support groups although we believe most packages going into factory wont.
We realize that this decision will not make everyone happy but hopefully it eventually leads to better group support in the future.
Hi Simon,
Thanks to you and the Board for this decision.
In the light of this decision, is something going to be done about submit requests which are being artificually held back by manual reviews injected by Factory Manitainers who objected to the removal of Group tags?
There are SR's like https://build.opensuse.org/request/show/738893 which are needed for important container networking stacks like cilium but cannot progress until Jan removes his objection.
Regards,
Richard
On Thu, 2019-10-31 at 16:45 +0100, Richard Brown wrote:
In the light of this decision, is something going to be done about submit requests which are being artificually held back by manual reviews injected by Factory Manitainers who objected to the removal of Group tags?
s/Factory Maintainers/openSUSE Review team members/g
There are no reviews added by the Factory Maintainers on this dispute.
Cheers, Dominique
On 2019-10-31 16:45, Richard Brown wrote:
On Wed, 2019-10-30 at 15:36 +1030, Simon Lees wrote:
Of all the proposals discussed the board agreed that the one submitted by Ludwig [2], to move groups into the repository meta data [...]
move = copy_to_dst + remove_from_src
In the light of this decision, is something going to be done about submit requests which are being artificually held back by manual reviews [...]
To get a 'move', that information should first show up in the "repository meta data" before it can be be deleted from "Groups:".
TBH I didn't follow the discussions from a certain point, but neither the referenced proposal nor the board decision does enlighten me enough to see what the actual plan is:
Technically we can generate additional information into the repo metadata.
A package is part of a development repo, and gets SR'ed to Factory, or other projects. So where is that information supposed to be originally stored? In the meta data of the package? And then propagated to the projects that package is forwarded to? How?
While I respect the decision of the board (and find it good that a decision was finally made), I'm still confused what it means in detail.
From Richard's demanding I see only the second half of the actions,
i.e., that the information should disappear from the RPM Group tag (or maybe not - depending on the maintainer). Yet - the first action, i.e., where to go with the information, is not clear to me. What are the next actions following the decision wrt/ the 1st part? Would someone please explain?
Have a nice day, Berny
On 11/1/19 8:24 AM, Bernhard Voelker wrote:
On 2019-10-31 16:45, Richard Brown wrote:
On Wed, 2019-10-30 at 15:36 +1030, Simon Lees wrote:
Of all the proposals discussed the board agreed that the one submitted by Ludwig [2], to move groups into the repository meta data [...]
move = copy_to_dst + remove_from_src
In the light of this decision, is something going to be done about submit requests which are being artificually held back by manual reviews [...]
TBH I didn't follow the discussions from a certain point, but neither the referenced proposal nor the board decision does enlighten me enough to see what the actual plan is:
Technically we can generate additional information into the repo metadata.
A package is part of a development repo, and gets SR'ed to Factory, or other projects. So where is that information supposed to be originally stored? In the meta data of the package? And then propagated to the projects that package is forwarded to? How?
While I respect the decision of the board (and find it good that a decision was finally made), I'm still confused what it means in detail. From Richard's demanding I see only the second half of the actions, i.e., that the information should disappear from the RPM Group tag (or maybe not - depending on the maintainer). Yet - the first action, i.e., where to go with the information, is not clear to me. What are the next actions following the decision wrt/ the 1st part? Would someone please explain?
The board has only really suggested that this would be a good way to see groups implemented we can't really force anyone to implement it nor do we feel like implementing it ourselves but we would welcome someone taking it forward because groups if done right can be really useful. As such we would expect the people who take this idea up to sort out all the technical details.
On 11/1/19 2:15 AM, Richard Brown wrote:
On Wed, 2019-10-30 at 15:36 +1030, Simon Lees wrote:
Hi Simon,
Thanks to you and the Board for this decision.
In the light of this decision, is something going to be done about submit requests which are being artificually held back by manual reviews injected by Factory Manitainers who objected to the removal of Group tags?
This issue has already been raised to the board and can be considered to fall in the group of things that I mentioned would be considered in private
There are SR's like https://build.opensuse.org/request/show/738893 which are needed for important container networking stacks like cilium but cannot progress until Jan removes his objection.
It seems in this case the review has now been accepted, but I wouldn't be surprised if some people have or will just resubmit requests in a similar state.
-
Bernhard Voelker -
Dominique Leuenberger / DimStar -
Hans-Peter Jansen -
Jan Engelhardt -
Ludwig Nussel -
Luiz Fernando Ranghetti -
Michal Suchánek -
Neal Gompa -
Richard Brown -
Simon Lees -
Stasiek Michalski -
Stefan Seyfried