Mailinglist Archive: opensuse-project (539 mails)

< Previous Next >
Re: [opensuse-project] Crazy idea - use bugzilla metrics to gauge quality?
  • From: Per Jessen <per@xxxxxxxxxxxx>
  • Date: Tue, 15 Mar 2011 20:26:46 +0100
  • Message-id: <iloehm$m6q$1@saturn.local.net>
Stephan Barth wrote:

Hi,

On Mon, Mar 14, 2011 at 07:54:56AM +0100, Per Jessen wrote:
On opensuse-factory, Marco Calistri wrote:

P.S. Too many bug reports, perhaps should be better awaiting a bit
more for a GM release.

Let me throw in a crazy idea - could we use bugzilla metrics to gauge
the quality of an upcoming release? Could we perhaps even use them
to delay if the numbers don't meet our requirements?

Some possible metrics:

# of bugs in status [x,y,z] in subsystem [a,b,c].
# of bugs with priority or severity [x]


Comments?

This is already done Novell internally by the support department, but
I think I am not allowed to share the measurements here.

If you're keeping track of data that is purely openSUSE, why not?

However I am not a fan of measuring quality with bugzilla bugs,

I use the word "quality" with great care, I completely agree that
quality is difficult to measure, not to mention by simply counting
bugs. Perhaps this should be seen more as a way to manage quality?

although I can think that this type of statistics can give a hint if
something is utterly wrong. I would say a difference of less than 10%
would not mean much.

I was hoping to avoid diving straight into the numbers/metrics, my post
was really intended as "here's a crazy idea, please shoot it down". I
would prefer to wait a few days before we start looking at what kind of
metrics and criteria we might want to apply. However, here's perhaps
something to fuel the debate:

track number of changes per subsystem, as well as number of bugs
reported. Could be further qualified with severities, amount of
activity, reason for closing etc.

Depending on the subsystem (some are more prone to errors, some more to
testing, I suspect), prior to a scheduled release candidate, we could
evaluate status (cnages vs testing vs bugfixing) and determine if we
would want to postpone due to insufficient testing or insufficient
bug-attention, for instance.

Enough fuel for now.


--
Per Jessen, Zürich (10.1°C)

--
To unsubscribe, e-mail: opensuse-project+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-project+help@xxxxxxxxxxxx

< Previous Next >
Follow Ups