On Wednesday 18 December 2013 12:00:41 Richard Brown wrote:
On Wed, 2013-12-18 at 11:45 +0100, Ilias R.(Zoumpis) wrote:
These number can be easily "correlated" . For instance we can correlate the number of installations with the number of features and the bug ratio. If in 12.1 we had 100k downloads , 300 features and Bug ratio = 0.60 and in the next release the name of features increases 10 % , the number of downloads decreases 5 % but the bug ratio raises up (e.g 0.70) that should mean that people did not install/upgrade to 12.3 due to the fact that more bugs are opened than closed. More calculations and metrics can be gathered (you can have a look on Eclipse example [1]) but the point is how willing are to use these number in order to decide a possible change in the release cycle of openSUSE...
I think your logic is based on false assumptions
'Bug ratio' is a useful measure of product quality only when the number of "Bug finders" and "Bug fixers" and their productivity is roughly stable - ie. It works fine in a commercial company where these variables can be controlled and/or measured and therefore factored into your metrics.
As we're a volunteer project, that's impossible in our case. Any value in looking at the Bug Ratio is going to be lost in the variable nature of our contributors and their available time to find and/or fix bugs. Bug Ratio goes up? It's just as safe to conclude that "More people are finding bugs rather than fixing them". That does not lead to a safe conclusion that "Product is more buggy"
Features also I think is relatively meaningless, as our 'features' are often something we inherit from chosen upstream packages we've been including for a long while. The only 'valid' features I'd think are useful to the kind of analysis you propose are the sort we rarely have, original, openSUSE originated "I have a new idea for a Feature, lets do this".
I don't think an Community-driven Open Source project can be effectively modelled by numbers, and I value 'human intelligence', gut feeling, and honest feedback from real people far more than cold hard numbers.
Too much numerical navel gazing can often lead to very warped and inappropriate conclusions, we've seen a lot of them lately. I'd like to think we can move forward in a way that reflects we're a community of human beings and not a collection of numbers on a graph :)
I don't want to piss on numbers TOO MUCH, though - there is information you can get from them. As long as you keep in mind that you have to apply some intelligence to interpreting them to avoid the pitfalls you point at... Things are usually gray. remember ;-) On a personal note, I really appreciate the insights that Ilias has produced in his thesis and with other numbers. I'm sure you and others agree that a graph like these is at the very least motivating: http://susepaste.org/36783018 This one is the direct result of Ilias' hard work, both alone when it comes to gathering the statistics and together with many happy geekos managing our social media accounts http://susepaste.org/54640315 The second graph shows that we've got our marketing in order for the releases - AND that we've reached a plateau, imho, in how much we can improve our marketing for the release. It is time to move marketing efforts in other area's. I've shown the very sad numbers for merchandising and (non-release related) marketing in general. Luckily, our awesome marketing team is working on planning and such (see the ML) - while at SUSE we're preparing a booth box. Hopefully we can start sending materials again in the 2nd or 3rd quarter next year. Better late than never! But now I'm going TERRIBLY off-topic :D