OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
It seems that we have a few areas that we should be looking at:
1. There should be some sort of "screening" that takes place before a bug is accepted - to ensure that the devs aren't seeing multiple copies of the same bugs. I think it's fair to say that those who write code would far rather *write code* than try to sort through duplicate bug entries. While it's likely there still would be dupes with a screening process, the incidence should be reduced.
2. There needs to be more of a "human" reply, especially on older bugs, that are being addressed. Looking at the current Bugzilla stats, I see that (for example) 12.1 has 183 bugs in a "NEEDINFO" state - there needs to be some sort of consistent follow-up to ensure that the info needed to fix the issues is provided, and if it isn't and the information can't be obtained (because the issue cannot be duped, those looking at it don't have the necessary hardware, etc), then the bug needs to be closed.
3. For older bugs, (such as on 10.3-11.3, which there are still open bugs on), a determination needs to be made as to whether the current supported releases have those issues, and if they do, the bugs need to be updated. If they don't, then the bugs need to be flagged in an appropriate way.
4. It would be extremely useful, I think - especially for the non-techie audience - to put some specific training together on things like (a) how to submit a bug and include the relevant information, (b) what to expect with your bug and how to monitor progress, (c) how to provide additional information when requested (and what to do if you've upgraded/moved to another version/changed distributions/decided to go back to Windows/MacOS/ whatever) so the bug can continue to be evaluated by those evaluating it.
Ultimately, the goal of this project is to increase resolution rates, reduce the time to resolution as much as possible, and to visibly improve the bug resolution process so more people don't perceive (correct or not) that reporting bugs is useless because they don't get resolved (and actually, looking at the stats on bug status, I think it's pretty unfair to say bugs aren't getting resolved. I see, for example, that of the reported bugs on 11.1, for example, over 3/4 of the bugs are marked as resolved.)
And overall across all releases of openSUSE, and SUSE Linux Professional/ SUSE Linux from 8.2 on, the bugs marked resolved are 13026/18285, which is nearly 3/4.
75% isn't bad (the Pareto rule is 80/20, and on 11.1 we're actually around 78% "resolved"), but I'm sure that if we can guide the process and streamline it, we can increase that.
Of course those are just rough statistics and the reason bugs are marked as 'resolved' isn't included in that - and there is a further breakdown that can be done that might help identify trends that would be useful.
There's clearly more that we should be looking at, but these four areas are a good place to start. For my own part, I'm willing to do more than talk and discuss the ideas, but it's going to take more than just one person working in his spare time to move this along.
I also think something similar to this should be done for FATE, but that's perhaps a topic for another day. :)
Jim
On Wed, 2012-05-16 at 03:04 +0000, Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
It seems that we have a few areas that we should be looking at:
- There needs to be more of a "human" reply, especially on older bugs,
that are being addressed. Looking at the current Bugzilla stats, I see that (for example) 12.1 has 183 bugs in a "NEEDINFO" state - there needs to be some sort of consistent follow-up to ensure that the info needed to fix the issues is provided, and if it isn't and the information can't be obtained (because the issue cannot be duped, those looking at it don't have the necessary hardware, etc), then the bug needs to be closed.
I think response times play a factor here. Obviously, a lot of people probably expect that as soon as they file a bug, someone is looking at it and responding to it. But that's not always the case, mainly because of limited manpower and prioritization. It can be demotivating to a bug filer to get a NEEDINFO response quite some time later and not feel they care about the bug anymore because they've found an alternative solution or whatever.
I also think something similar to this should be done for FATE, but that's perhaps a topic for another day. :)
It may be more prudent to first talk about the tool itself. There's been talk from time to time by people wanting to see FATE and Bugzilla functionality merged because its too confusing going to two different places and not clear to some people the distinction between the two tools.
Bryen
Jim
Jim Henderson Please keep on-topic replies on the list so everyone benefits
On Tue, 15 May 2012 22:28:32 -0500, Bryen M Yunashko wrote:
I think response times play a factor here. Obviously, a lot of people probably expect that as soon as they file a bug, someone is looking at it and responding to it. But that's not always the case, mainly because of limited manpower and prioritization. It can be demotivating to a bug filer to get a NEEDINFO response quite some time later and not feel they care about the bug anymore because they've found an alternative solution or whatever.
Yes, I think this is absolutely true.
That's one of the reasons I think it's important to approach this in a very methodical and predictable way. If response times are generally predictable and there's visible movement (such as moving from "new" to "assigned" or "needinfo"), then people are more likely (not always, but more likely) to be patient. But it's hard to be patient if you can't predict whether it'll be a day, week, month, or several months before you hear back on it.
I also think something similar to this should be done for FATE, but that's perhaps a topic for another day. :)
It may be more prudent to first talk about the tool itself. There's been talk from time to time by people wanting to see FATE and Bugzilla functionality merged because its too confusing going to two different places and not clear to some people the distinction between the two tools.
Yeah, you might be right about that. Looking at the stats on FATE, it seems that there's a lot of "waiting for votes", but with relatively little engagement in the voting process from the community (and I don't know what those stats are, haven't yet worked out a good way to pull more detailed reports out of FATE) as well as perhaps no solid guidelines on how many votes something needs before it's considered valid to investigate (and again, maybe those are there, I'm not as close to FATE myself as I probably should be), it's difficult to predict outcomes. What I do see there is a whole bunch of "unconfirmed" (~60% IIRC on 'openSUSE Distribution') feature requests.
But I'd like to stay focused on bugzilla here. Maybe as my current contract wraps up over the next couple of days, I'll have some time to (a) dig a bit more into FATE, and (b) start a separate discussion on that.
Jim
Jim Henderson wrote:
On Tue, 15 May 2012 22:28:32 -0500, Bryen M Yunashko wrote:
It may be more prudent to first talk about the tool itself. There's been talk from time to time by people wanting to see FATE and Bugzilla functionality merged because its too confusing going to two different places and not clear to some people the distinction between the two tools.
Yeah, you might be right about that. Looking at the stats on FATE, it seems that there's a lot of "waiting for votes", but with relatively little engagement in the voting process from the community (and I don't know what those stats are, haven't yet worked out a good way to pull more detailed reports out of FATE) as well as perhaps no solid guidelines on how many votes something needs before it's considered valid to investigate
Only zero votes really - all a request needs is an interested person who will go ahead and implement it. I don't think more votes means any higher chance of such a person turning up :-(
Le 16/05/2012 05:04, Jim Henderson a écrit :
It seems that we have a few areas that we should be looking at:
You are right
I add a thing: do not ask everybody to report bugs for non factory distribution if not major.
I just closed all my reports (30?), because after very few days I do not even remember what they are about and often can't give any more info
I just closed as wontfix and "no time to work on this"
jdd
jdd wrote:
Le 16/05/2012 05:04, Jim Henderson a écrit :
It seems that we have a few areas that we should be looking at:
You are right
I add a thing: do not ask everybody to report bugs for non factory distribution if not major.
The issue is here: "what is major"? I don't think it is wise to ask people to make that decision themselves.
I just closed all my reports (30?), because after very few days I do not even remember what they are about and often can't give any more info
That is a much worse problem, yes.
On Wed, 16 May 2012 11:02:58 +0200, jdd wrote:
Le 16/05/2012 05:04, Jim Henderson a écrit :
It seems that we have a few areas that we should be looking at:
You are right
I add a thing: do not ask everybody to report bugs for non factory distribution if not major.
I just closed all my reports (30?), because after very few days I do not even remember what they are about and often can't give any more info
I just closed as wontfix and "no time to work on this"
Per brings up a good point - "Major" is much in the eye of the beholder, but also not everyone runs factory, but people do run into issues with the released versions.
So one thing we may need to identify is a workflow that can accommodate a way of testing the most recent code in factory to see if the reported bug has been corrected.
But I don't think it's as simple as telling the reporter "try the version in factory" because I /think/ it's possible that pulling a single package in from factory might well cause additional libraries/dependencies to be brought in as well that /may/ break other things (true statement? Can someone confirm?)
Jim
Jim Henderson wrote:
But I don't think it's as simple as telling the reporter "try the version in factory" because I /think/ it's possible that pulling a single package in from factory might well cause additional libraries/dependencies to be brought in as well that /may/ break other things (true statement? Can someone confirm?)
We should most certainly not be recommending or using Factory to users as a part of resolving a bug.
Le 16/05/2012 20:38, Per Jessen a écrit :
We should most certainly not be recommending or using Factory to users as a part of resolving a bug.
sure, but the factory bugs are the most likely to be solved
jdd
jdd wrote:
Le 16/05/2012 20:38, Per Jessen a écrit :
We should most certainly not be recommending or using Factory to users as a part of resolving a bug.
sure, but the factory bugs are the most likely to be solved
That is a pity. I would have thought Factory bugs were the most likely to be closed with "please upgrade and see if it works".
On Wed, 16 May 2012 20:38:39 +0200, Per Jessen wrote:
Jim Henderson wrote:
But I don't think it's as simple as telling the reporter "try the version in factory" because I /think/ it's possible that pulling a single package in from factory might well cause additional libraries/dependencies to be brought in as well that /may/ break other things (true statement? Can someone confirm?)
We should most certainly not be recommending or using Factory to users as a part of resolving a bug.
No, but if it's fixed in factory, then the question becomes about backporting the fix and/or properly identifying that it has been fixed in factory.
It's not a productive use of a developer's time to fix something that's already been fixed. :)
Jim
Jim Henderson wrote:
On Wed, 16 May 2012 20:38:39 +0200, Per Jessen wrote:
Jim Henderson wrote:
But I don't think it's as simple as telling the reporter "try the version in factory" because I /think/ it's possible that pulling a single package in from factory might well cause additional libraries/dependencies to be brought in as well that /may/ break other things (true statement? Can someone confirm?)
We should most certainly not be recommending or using Factory to users as a part of resolving a bug.
No, but if it's fixed in factory, then the question becomes about backporting the fix and/or properly identifying that it has been fixed in factory.
If it's confirmed fixed in Factory, we assign the bug to maintenance@opensuse.org with NEEDINFO - I don't know who determines the need for an update, but that is the process.
On Wed, 16 May 2012 21:47:15 +0200, Per Jessen wrote:
Jim Henderson wrote:
On Wed, 16 May 2012 20:38:39 +0200, Per Jessen wrote:
Jim Henderson wrote:
But I don't think it's as simple as telling the reporter "try the version in factory" because I /think/ it's possible that pulling a single package in from factory might well cause additional libraries/dependencies to be brought in as well that /may/ break other things (true statement? Can someone confirm?)
We should most certainly not be recommending or using Factory to users as a part of resolving a bug.
No, but if it's fixed in factory, then the question becomes about backporting the fix and/or properly identifying that it has been fixed in factory.
If it's confirmed fixed in Factory, we assign the bug to maintenance@opensuse.org with NEEDINFO - I don't know who determines the need for an update, but that is the process.
That's good to know.
Jim
Hello,
Am Mittwoch, 16. Mai 2012 schrieb Per Jessen:
If it's confirmed fixed in Factory, we assign the bug to maintenance@opensuse.org with NEEDINFO -
I don't know who determines the need for an update,
I could answer "the package maintainer", and that's correct in most cases because he/she has to do the work.
For major security issues, the security team will "force-decide" ;-) to release an update, but they still rely on cooperation by the package maintainer.
I don't think maintenance@opensuse.org gets lots of questions like "do you think this is worth an update" (correct me if I'm wrong ;-)
They are often NEEDINFO'd when the update is already ready, and usually they'll accept the updates - except if something in them is broken, QA fails, or maybe if there's a big risk of breakage/regressions by fixing a minor bug.
Regards,
Christian Boltz
Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
It seems that we have a few areas that we should be looking at:
- There should be some sort of "screening" that takes place before a
bug is accepted - to ensure that the devs aren't seeing multiple copies of the same bugs. I think it's fair to say that those who write code would far rather *write code* than try to sort through duplicate bug entries. While it's likely there still would be dupes with a screening process, the incidence should be reduced.
There is already some sort of screening/triaging happening, but I don't know the process nor the people ionvolved. Can anyone from SUSE or Novell help? My reports are frequently forwarded/rerouted/assigned by someone called Kun Kun Zhang.
- It would be extremely useful, I think - especially for the
non-techie audience - to put some specific training together on things like (a) how to submit a bug and include the relevant information, (b) what to expect with your bug and how to monitor progress, (c) how to provide additional information when requested (and what to do if you've upgraded/moved to another version/changed distributions/decided to go back to Windows/MacOS/ whatever) so the bug can continue to be evaluated by those evaluating it.
We have got to have something like this already - it's not exactly new stuff. Maybe in the support data base?
On Wed, 16 May 2012 13:51:53 +0200, Per Jessen wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
It seems that we have a few areas that we should be looking at:
- There should be some sort of "screening" that takes place before a
bug is accepted - to ensure that the devs aren't seeing multiple copies of the same bugs. I think it's fair to say that those who write code would far rather *write code* than try to sort through duplicate bug entries. While it's likely there still would be dupes with a screening process, the incidence should be reduced.
There is already some sort of screening/triaging happening, but I don't know the process nor the people ionvolved. Can anyone from SUSE or Novell help? My reports are frequently forwarded/rerouted/assigned by someone called Kun Kun Zhang.
I think where we start with this is, as you've said here, to identify what the current processes is. Any process improvement project typically involves discovery of what's currently happening.
So I'd second your RFI here.
- It would be extremely useful, I think - especially for the
non-techie audience - to put some specific training together on things like (a) how to submit a bug and include the relevant information, (b) what to expect with your bug and how to monitor progress, (c) how to provide additional information when requested (and what to do if you've upgraded/moved to another version/changed distributions/decided to go back to Windows/MacOS/ whatever) so the bug can continue to be evaluated by those evaluating it.
We have got to have something like this already - it's not exactly new stuff. Maybe in the support data base?
It isn't new stuff, but I wonder if it is out there. Anyone know?
Jim
On 05/16/2012 01:51 PM, Per Jessen wrote:
Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
It seems that we have a few areas that we should be looking at:
- There should be some sort of "screening" that takes place before a
bug is accepted - to ensure that the devs aren't seeing multiple copies of the same bugs. I think it's fair to say that those who write code would far rather *write code* than try to sort through duplicate bug entries. While it's likely there still would be dupes with a screening process, the incidence should be reduced.
There is already some sort of screening/triaging happening, but I don't know the process nor the people ionvolved. Can anyone from SUSE or Novell help? My reports are frequently forwarded/rerouted/assigned by someone called Kun Kun Zhang.
For each component in bugzilla (and this is configurable per product/version), we have a default assignee. This is in many cases bnc-team-screening - and some guys in China handle these and assign it forward.
Anybody can look at bugs assigned to bnc-team-screening and triaging them.
There are also other default assignees.
In each case, the default assignees look at the bug and triage - and then assign to the maintainer of the package (or solve the bug themselves).
We for sure could need more people to do the initial screening.
- It would be extremely useful, I think - especially for the
non-techie audience - to put some specific training together on things like (a) how to submit a bug and include the relevant information, (b) what to expect with your bug and how to monitor progress, (c) how to provide additional information when requested (and what to do if you've upgraded/moved to another version/changed distributions/decided to go back to Windows/MacOS/ whatever) so the bug can continue to be evaluated by those evaluating it.
We have got to have something like this already - it's not exactly new stuff. Maybe in the support data base?
Btw. have you seen the documentation on bugs.opensuse.org?
Andreas
On Fri, 18 May 2012 16:06:39 +0200, Andreas Jaeger wrote:
For each component in bugzilla (and this is configurable per product/version), we have a default assignee. This is in many cases bnc-team-screening - and some guys in China handle these and assign it forward.
Anybody can look at bugs assigned to bnc-team-screening and triaging them.
There are also other default assignees.
In each case, the default assignees look at the bug and triage - and then assign to the maintainer of the package (or solve the bug themselves).
We for sure could need more people to do the initial screening.
Do we have any documentation with guidelines/instructions for how people can get involved in the initial screening?
If we need some help with this screening, what we should probably do is put together a set of desired skills for doing this screening, some guidelines on how to do it, and then make announcements in appropriate venues asking for volunteers to help out.
I don't know that the average user of openSUSE knows how to get involved in doing something like this.
We have got to have something like this already - it's not exactly new stuff. Maybe in the support data base?
Btw. have you seen the documentation on bugs.opensuse.org?
I haven't, but I'll take a look at it - looks like it redirects to the wiki.
Jim
We for sure could need more people to do the initial screening.
Do we have any documentation with guidelines/instructions for how people can get involved in the initial screening?
If we need some help with this screening, what we should probably do is put together a set of desired skills for doing this screening, some guidelines on how to do it, and then make announcements in appropriate venues asking for volunteers to help out.
Part of the reason that I was thinking of using the forum for initial Newbie bug screening is that people are already there and involved; and/or, using a nicer front-end to Bugzilla - getting more people involved is always a huge problem. The trick is to find those that are hovering about, not sure how to contribute or already quite talkative on the forum, and just give them a little sideways nudge into Bugzilla stuff.
Having said that, the biggest issue is with the initial release when it needs to be 'all hands on deck' and an invitation to a temporary bug-squashing event, facilitated by this type of documentation, might work well to help mobilize people.
I really need a place to start gathering information off list, so we can look at the information we have and not be going off on tangents. Even a piratepad would do, or do you have a wiki page I can start adding material to? -- IRC: helen_au helen.south@opensuse.org helensouth.com
On Sat, 19 May 2012 12:17:59 +1000, Helen South wrote:
We for sure could need more people to do the initial screening.
Do we have any documentation with guidelines/instructions for how people can get involved in the initial screening?
If we need some help with this screening, what we should probably do is put together a set of desired skills for doing this screening, some guidelines on how to do it, and then make announcements in appropriate venues asking for volunteers to help out.
Part of the reason that I was thinking of using the forum for initial Newbie bug screening is that people are already there and involved;
I think once we get the login/infrastructure issues dealt with, this makes sense. Until then, it's likely to cause frustration from Newbie/ non-technical users because you have to sometimes jump through all sorts of hoops clearing cache/cookies to get the login process to work without the session timing out on you.
I'm hoping to meet with the people involved next week in Provo (my first chance as I've been doing contract work for the past several months) to find out what we need to do to get them the information necessary to resolve those issues.
It's been frustrating to use for months - especially for those who use it daily (staff and regular contributors).
and/or, using a nicer front-end to Bugzilla - getting more people involved is always a huge problem.
I've also asked questions of people I know on the inside to find out what would be involved in setting a "basic" interface up for submitting bugs - something less intimidating than the "full" bugzilla submission interface.
Ideally, I think a simple interface for reporting would be good - something with some defaults (especially the description), similar to what Cyanogenmod does with theirs (they don't use Bugzilla, but rather Google Code IIRC, but the principle is the same - the default value for the description is "here's what you need to provide" and includes the instructions.)
The trick is to find those that are hovering about, not sure how to contribute or already quite talkative on the forum, and just give them a little sideways nudge into Bugzilla stuff.
Yep. And a part of that is getting the back-end resolution process streamlined so they can see relatively quick resolution on (especially) issues that end up being simple to fix.
Having said that, the biggest issue is with the initial release when it needs to be 'all hands on deck' and an invitation to a temporary bug-squashing event, facilitated by this type of documentation, might work well to help mobilize people.
Yes. If we could also do something - no matter how small - for people who participate in such an event. Some kind of little spiff or a credit in a "thank you" page included in the installer, that kind of recognition could go a ways towards getting people involved.
I really need a place to start gathering information off list, so we can look at the information we have and not be going off on tangents. Even a piratepad would do, or do you have a wiki page I can start adding material to?
I haven't set a page up yet on the wiki (got buried under some stuff that had been piling up during the week).
I have my own server here at home, so I can set up either logins on my Vibe installation or I have a barebones Mediawiki installation we can use.
I've got a "Todo Task" manager plugin installed in the MW installation - it's a very stripped down task manager (but it does e-mail notifications and I was surprised to see that working). I'm game for either tool, and since it's my own server, I can easily pull data out and manipulate it to suit our needs if/when we move to the openSUSE hosted site/solution.
If you want to use Vibe, just e-mail me off-list and I'll set up a user account for you. MW has self-registration enabled on it, and I do have the server configured to generate e-mails and relay them through my ISP. The wiki is at http://hendersj.dyndns.org/wiki (it's a dynamic IP, but my router takes care of updating it if/when the address changes, so availability isn't an issue). I'll set things up so whichever is used, I'm backing up to a secondary system.
Jim
Jim Henderson wrote:
Do we have any documentation with guidelines/instructions for how people can get involved in the initial screening?
If we need some help with this screening, what we should probably do is put together a set of desired skills for doing this screening, some guidelines on how to do it, and then make announcements in appropriate venues asking for volunteers to help out.
I don't know that the average user of openSUSE knows how to get involved in doing something like this.
Getting involved is fairly easy in fact - there is a mailing list that receives all bugzilla updates. I subscribed maybe two years ago, I can't be certain. Filtering out all new bugs is easy.
opensuse-bugs@opensuse.org
On Sat, 19 May 2012 09:27:45 +0200, Per Jessen wrote:
Jim Henderson wrote:
Do we have any documentation with guidelines/instructions for how people can get involved in the initial screening?
If we need some help with this screening, what we should probably do is put together a set of desired skills for doing this screening, some guidelines on how to do it, and then make announcements in appropriate venues asking for volunteers to help out.
I don't know that the average user of openSUSE knows how to get involved in doing something like this.
Getting involved is fairly easy in fact - there is a mailing list that receives all bugzilla updates. I subscribed maybe two years ago, I can't be certain. Filtering out all new bugs is easy.
On the one hand, yeah, that makes it easy - but understanding what to do after you've subscribed to the list may not be so clear, especially to those who are new to the process.
I think that's where we need to do some discovery and adopt some "best practices", even if they're just guidelines for those getting involved.
Jim
On 05/19/2012 04:08 AM, Jim Henderson wrote:
On Fri, 18 May 2012 16:06:39 +0200, Andreas Jaeger wrote:
For each component in bugzilla (and this is configurable per product/version), we have a default assignee. This is in many cases bnc-team-screening - and some guys in China handle these and assign it forward.
Anybody can look at bugs assigned to bnc-team-screening and triaging them.
There are also other default assignees.
In each case, the default assignees look at the bug and triage - and then assign to the maintainer of the package (or solve the bug themselves).
We for sure could need more people to do the initial screening.
Do we have any documentation with guidelines/instructions for how people can get involved in the initial screening?
If we need some help with this screening, what we should probably do is put together a set of desired skills for doing this screening, some guidelines on how to do it, and then make announcements in appropriate venues asking for volunteers to help out.
I don't know that the average user of openSUSE knows how to get involved in doing something like this.
We had some bug days where triage was done and for those some documentation was written. I don't know where that documentation is, I suggest you search for the latest bug days...
We have got to have something like this already - it's not exactly new stuff. Maybe in the support data base?
Btw. have you seen the documentation on bugs.opensuse.org?
I haven't, but I'll take a look at it - looks like it redirects to the wiki.
Yes, sure ;) It points to the information that we have about bugreporting which is at http://en.opensuse.org/openSUSE:Submitting_bug_reports
If you really want to enhance this, let's meet on IRC next week (just ping me on opensuse-project) and let's discuss any questions you have and I'll help with answering...
Andreas
On Sat, 19 May 2012 14:56:52 +0200, Andreas Jaeger wrote:
Btw. have you seen the documentation on bugs.opensuse.org?
I haven't, but I'll take a look at it - looks like it redirects to the wiki.
Yes, sure ;) It points to the information that we have about bugreporting which is at http://en.opensuse.org/openSUSE:Submitting_bug_reports
Sounds good. :)
If you really want to enhance this, let's meet on IRC next week (just ping me on opensuse-project) and let's discuss any questions you have and I'll help with answering...
I do think enhancing it would be good (as well as having something that has to do with what happens after submission - both from the reporter's perspective and from the handling perspective).
Happy to chat on IRC or wherever - once I know more about my schedule, I'll drop you an e-mail and we can coordinate something. :)
Jim
Hi,
Le mercredi 16 mai 2012, à 03:04 +0000, Jim Henderson a écrit :
- For older bugs, (such as on 10.3-11.3, which there are still open
bugs on), a determination needs to be made as to whether the current supported releases have those issues, and if they do, the bugs need to be updated. If they don't, then the bugs need to be flagged in an appropriate way.
In GNOME Bugzilla, we do have a OBSOLETE resolution. I think that would help in such cases.
I've been looking yesterday and today at some old bugs filed against GNOME. To be honest, it's just a big mess. Right now, I see:
+ 28 bugs in 11.0 + 84 bugs in 11.1 + 140 bugs in 11.2 + 73 bugs in 11.3 => 325 bugs
(I already closed something between 50 and 100 bugs)
The vast majority of those 325 bugs are most likely obsolete, bugs that should just go upstream, or bugs that are actually feature requests. That is: bugs that we really should not track anymore.
There are various reasons explaining those high number of bugs:
a) only 2-3 people are looking at the bugs, and those are the same 2-3 people doing all of the remaining work for GNOME in Factory. So we don't have a lot of time to look at bugs in general, and even less to look at bugs in old distros.
b) a lot of bugs are just missing details, and as you pointed out, developers tend to prefer to develop than request missing details in tons of bugs.
c) reporters tend to not go back to their bugs to close them when they're fixed/obsolete.
I'm happy to invest a few days to look at all the bugs and move the ones I feel are still valid for Factory to 12.2, but before doing that, I'd like us to know what's the right way to proceed for the bugs we don't care about.
I'd really love to have a OBSOLETE resolution, so we could just mass-close those bugs with a comment saying "We believe this bug is most likely fixed in openSUSE 12.1 (or Factory). If you can still reproduce the bug there, please reopen the bug and move it to openSUSE 12.1." Alternatively, we can do a mass NEEDINFO-ing of all the bugs, and wait a few weeks before closing the bugs that didn't get a reply...
Cheers,
Vincent
On Wed, 2012-05-16 at 14:11 +0200, Vincent Untz wrote:
I'd really love to have a OBSOLETE resolution, so we could just mass-close those bugs with a comment saying "We believe this bug is most likely fixed in openSUSE 12.1 (or Factory). If you can still reproduce the bug there, please reopen the bug and move it to openSUSE 12.1." Alternatively, we can do a mass NEEDINFO-ing of all the bugs, and wait a few weeks before closing the bugs that didn't get a reply...
While I get the whole "Hey, its fixed in future release" thing, I think some people find this offputting. Again, its an issue of understanding the nature of how things work. We need a clearer explanation of why we WONTFIX it now in the current release. Some people want the fix now rather than feel forced to wait for the final release of next version. Nor might they actually plan on upgrading with each new release.
Not saying I agree with their thinking, but I can see how they might feel they're not being cared about if they're told "next release." Too many people out there are still accustomed to the concept of Service Packs to fix their current release.
And yes, I know we can't make everyone happy. :-)
Bryen
On Wed, May 16, 2012 at 07:22:13AM -0500, Bryen M Yunashko wrote:
On Wed, 2012-05-16 at 14:11 +0200, Vincent Untz wrote:
I'd really love to have a OBSOLETE resolution, so we could just mass-close those bugs with a comment saying "We believe this bug is most likely fixed in openSUSE 12.1 (or Factory). If you can still reproduce the bug there, please reopen the bug and move it to openSUSE 12.1." Alternatively, we can do a mass NEEDINFO-ing of all the bugs, and wait a few weeks before closing the bugs that didn't get a reply...
While I get the whole "Hey, its fixed in future release" thing, I think some people find this offputting. Again, its an issue of understanding the nature of how things work. We need a clearer explanation of why we WONTFIX it now in the current release. Some people want the fix now rather than feel forced to wait for the final release of next version. Nor might they actually plan on upgrading with each new release.
Not saying I agree with their thinking, but I can see how they might feel they're not being cared about if they're told "next release." Too many people out there are still accustomed to the concept of Service Packs to fix their current release.
And yes, I know we can't make everyone happy. :-)
This perhaps needs more education on how fixes can be pushed out via online updates or tumbleweed.
Of course its not always possible and always a bit more work to do.
Ciao, Marcus
On Wed, 16 May 2012 14:23:34 +0200, Marcus Meissner wrote:
This perhaps needs more education on how fixes can be pushed out via online updates or tumbleweed.
Perhaps a separate project that comes out of this one is to more clearly document our update policy and procedures. What's on the wiki now is pretty good, but it seems that a lot of users don't go to the wiki (and perhaps aren't even aware of it) and so don't know where to find this information.
That might also be something that can be addressed in future releases by having a landing page that includes a link to a FAQ that addresses update policy questions and other similar questions that new users ask (I could see if the forums staff could help pull something like this together from the forums - we do tend towards the non-technical userbase over there; and we could have someone who spends a lot of time on the user mailing list help assemble a list of questions from there as well).
The FAQ should be short Q&A, and the answers should link to the fuller answers on the respective wiki pages.
Jim
On Wed, 16 May 2012 16:14:13 +0000 (UTC) Jim Henderson hendersj@gmail.com wrote:
...
The FAQ should be short Q&A, and the answers should link to the fuller answers on the respective wiki pages.
Whole problem is that FAQ and wiki and bugzilla and forums and mail lists must be up to date, all pointing to a solution. Right now with dispersed and manually connected services we can't provide accurate path to the solution that user needs when it encounters problem. How far user has to go before solving a problem, depends on luck, ie. entry point and does that point has any path to the solution.
On Sat, 19 May 2012 12:41:39 -0500, Rajko wrote:
On Wed, 16 May 2012 16:14:13 +0000 (UTC) Jim Henderson hendersj@gmail.com wrote:
...
The FAQ should be short Q&A, and the answers should link to the fuller answers on the respective wiki pages.
Whole problem is that FAQ and wiki and bugzilla and forums and mail lists must be up to date, all pointing to a solution. Right now with dispersed and manually connected services we can't provide accurate path to the solution that user needs when it encounters problem. How far user has to go before solving a problem, depends on luck, ie. entry point and does that point has any path to the solution.
I think if we can put together a recommended process, though, then this will naturally follow on. Having a repeatable process makes it much easier to put together a FAQ that addresses the process.
Jim
Whole problem is that FAQ and wiki and bugzilla and forums and mail lists must be up to date, all pointing to a solution. R- Regards, Rajko --
Perhaps not the whole problem, but certainly a big chunk.
As Jim wrote "I think that's where we need to do some discovery and adopt some "best practices"...". Discovery is important, gathering all these experiences and ideas in one place.
I liked Yaloki's suggestion to use the Wiki - decision making on project management etc will take time, and I've got a lot of material here.
I actually started to create a wiki page, but was unsure about namespaces and format, there's messages about QA and approval so I was concerned about doing the wrong thing. At this point it's primarily information-gathering and needs to hold lists of key points, ideas, quotes from messages (with links to archived mailing list message), and TO DO list. In a 'work in progress' format. I'm not even too worried about the 'ToDo' plugin at this point, while it's a great idea. A point list will do the job for now.
If someone could begin a page with the appropriate structure in place, I can start populating it. Does a namespace need to be created for 'workspace' documents?
If the Wiki turns out not to be appropriate we can easily extract the information, we just need a place to start working on this.
On Sun, 20 May 2012 07:51:58 +1000, Helen South wrote:
If the Wiki turns out not to be appropriate we can easily extract the information, we just need a place to start working on this.
Feel free to use the wiki I set up on my server here if the openSUSE wiki is slowing you down. Both are MediaWiki, so moving info back and forth will be trivial, and I'm happy to do the legwork on that if necessary.
Jim
On Sun, May 20, 2012 at 8:39 AM, Jim Henderson hendersj@gmail.com wrote:
On Sun, 20 May 2012 07:51:58 +1000, Helen South wrote:
Feel free to use the wiki I set up on my server here if the openSUSE wiki is slowing you down. Both are MediaWiki, so moving info back and forth will be trivial, and I'm happy to do the legwork on that if necessary.
Thanks Jim, that is a possible option - but I think in the interests of transparency and participation, using the more publicly accessible tool would probably be a good thing. I already have a Novell login so I have access. The main thing is the initial namespace/url - a Wiki person needs to approve that. From my point of view it's not too critical since it's a workspace rather than user-facing documentation, ideally it should reflect the overall purpose (planning/workspace) and focus (bugzilla).
If you like, email me off-list login information and if there isn't an openSUSE page when I'm ready to do some work thisevening, I'll make a start there instead. (it's early AM in Oz as I write.)
By the way, I'm just diving in feet first as always, so please feel free to offer guidance or correction as needed :)
On Sun, 20 May 2012 08:56:20 +1000, Helen South wrote:
On Sun, May 20, 2012 at 8:39 AM, Jim Henderson hendersj@gmail.com wrote:
On Sun, 20 May 2012 07:51:58 +1000, Helen South wrote:
Feel free to use the wiki I set up on my server here if the openSUSE wiki is slowing you down. Both are MediaWiki, so moving info back and forth will be trivial, and I'm happy to do the legwork on that if necessary.
Thanks Jim, that is a possible option - but I think in the interests of transparency and participation, using the more publicly accessible tool would probably be a good thing. I already have a Novell login so I have access. The main thing is the initial namespace/url - a Wiki person needs to approve that. From my point of view it's not too critical since it's a workspace rather than user-facing documentation, ideally it should reflect the overall purpose (planning/workspace) and focus (bugzilla).
If you like, email me off-list login information and if there isn't an openSUSE page when I'm ready to do some work thisevening, I'll make a start there instead. (it's early AM in Oz as I write.)
It's actually set up so anyone can create an account - it's not wrapped in SSL (there's a note to that effect), so transparency isn't an issue. I just don't want us to get bogged down trying to work out the QA page issue and whatnot in the official wiki and lose some momentum - if we can get that resolved quickly, then that's great.
I've got a very basic structure in place (basically a link to a single page for this project). The wiki is otherwise completely empty apart from a few templates.
The URL is http://hendersj.dyndns.org/wiki
By the way, I'm just diving in feet first as always, so please feel free to offer guidance or correction as needed :)
That's great. :) I'm just heading out to grab some dinner with a friend who's in town on business, when I get back I'll see what else I need to do at this point.
Jim
I've got a very basic structure in place (basically a link to a single page for this project). The wiki is otherwise completely empty apart from a few templates.
The URL is http://hendersj.dyndns.org/wiki
Jim
Just made a rough start; couldn't find a template so borrowed from openSUSE wiki.
http://hendersj.dyndns.org/wiki/index.php/Overview
Just figuring out the formatting to begin with, will put some real data in later.
Feel free to edit at will!
On Sun, 20 May 2012 10:06:31 +1000, Helen South wrote:
I've got a very basic structure in place (basically a link to a single page for this project). The wiki is otherwise completely empty apart from a few templates.
The URL is http://hendersj.dyndns.org/wiki
Jim --
Just made a rough start; couldn't find a template so borrowed from openSUSE wiki.
http://hendersj.dyndns.org/wiki/index.php/Overview
Just figuring out the formatting to begin with, will put some real data in later.
Feel free to edit at will!
Great, thanks! I'll have some time tomorrow and will dig in. :)
Jim
Rajko kindly created an appropriate page on openSUSE wiki, you can find it here.
http://en.opensuse.org/openSUSE:Bugreport_process
I've started making some very rough notes, please feel free to completely reconstruct the page along proper 'project management' lines!
On Sun, 20 May 2012 22:29:10 +1000, Helen South wrote:
Rajko kindly created an appropriate page on openSUSE wiki, you can find it here.
http://en.opensuse.org/openSUSE:Bugreport_process
I've started making some very rough notes, please feel free to completely reconstruct the page along proper 'project management' lines!
Great, thanks again. :)
Jim
On Sun, May 20, 2012 at 6:29 AM, Helen South helen.south@opensuse.org wrote:
Rajko kindly created an appropriate page on openSUSE wiki, you can find it here.
http://en.opensuse.org/openSUSE:Bugreport_process
I've started making some very rough notes, please feel free to completely reconstruct the page along proper 'project management' lines!
Just to let you (and the list) know, I had a couple days that I had to dedicate to another task (some forum infrastructure work), and now it seems that gmane's NNTP interface is messed up and I'm not seeing new posts through it. I'll catch up and get directly on the list so this can keep going.
I've had a look at the tool Henne pointed me at, and from a PM perspective, that'll meet my needs, so I'll get a project set up there so I can track it, and I'll spend time today working on this wiki page as well. Once it's set up, I'll post the link here so anyone who wants to follow along can.
Sorry for vanishing abruptly - it just looked like the list had gone quiet until I tried to follow-up. I've notified the gmane site admin that there's an issue, so hopefully they'll get that resolved soon.
Jim
On Thu, May 24, 2012 at 1:39 AM, Jim Henderson hendersj@gmail.com wrote:
I've had a look at the tool Henne pointed me at, and from a PM perspective, that'll meet my needs, so I'll get a project set up there so I can track it, and I'll spend time today working on this wiki page as well. Once it's set up, I'll post the link here so anyone who wants to follow along can.
Jim
Great. I'd been waiting out for further discussion about the use (or not) of the wiki as a workspace. I'm going to be a little bit snowed under for the next week or two, starting a new job on monday plus two exams.
cheers Helen
On Wed, May 23, 2012 at 3:08 PM, Helen South helen.south@opensuse.org wrote:
On Thu, May 24, 2012 at 1:39 AM, Jim Henderson hendersj@gmail.com wrote:
I've had a look at the tool Henne pointed me at, and from a PM perspective, that'll meet my needs, so I'll get a project set up there so I can track it, and I'll spend time today working on this wiki page as well. Once it's set up, I'll post the link here so anyone who wants to follow along can.
Jim
Great. I'd been waiting out for further discussion about the use (or not) of the wiki as a workspace. I'm going to be a little bit snowed under for the next week or two, starting a new job on monday plus two exams.
Sounds good. I have updated the page on the wiki with a little more information, and Andreas and I had a good IRC chat earlier today about the current process that I'm using to help guide what I'm doing.
It appears that the reason I've lost access via gmane is self-inflicted - I was grabbing headers from the NNTP interface to do a traffic analysis for another project I'm involved in, and it was detected as an "address harvesting activity" (shame on me for not thinking about that) and the system blocked my entire DNS subdomain out as a result.
Whoops. :( Hopefully I can get them to unblock me, otherwise I'll be catching up from the archives and getting the list sent via e-mail instead (which I find less convenient).
Jim
Hey,
On 20.05.2012 00:56, Helen South wrote:
From my point of view it's not too critical since it's a workspace rather than user-facing documentation, ideally it should reflect the overall purpose (planning/workspace) and focus (bugzilla).
I guess Rajko already explained this but the "workspace" for planing or any other purposes of the openSUSE project is the namespace openSUSE:
I hope that helps you in you endeavors :)
Henne
On Sun, 2012-05-20 at 07:51 +1000, Helen South wrote:
Whole problem is that FAQ and wiki and bugzilla and forums and mail lists must be up to date, all pointing to a solution. R- Regards, Rajko --
Perhaps not the whole problem, but certainly a big chunk.
As Jim wrote "I think that's where we need to do some discovery and adopt some "best practices"...". Discovery is important, gathering all these experiences and ideas in one place.
I liked Yaloki's suggestion to use the Wiki - decision making on project management etc will take time, and I've got a lot of material here.
I actually started to create a wiki page, but was unsure about namespaces and format, there's messages about QA and approval so I was concerned about doing the wrong thing. At this point it's primarily information-gathering and needs to hold lists of key points, ideas, quotes from messages (with links to archived mailing list message), and TO DO list. In a 'work in progress' format. I'm not even too worried about the 'ToDo' plugin at this point, while it's a great idea. A point list will do the job for now.
If someone could begin a page with the appropriate structure in place, I can start populating it. Does a namespace need to be created for 'workspace' documents?
If the Wiki turns out not to be appropriate we can easily extract the information, we just need a place to start working on this.
If I recall, I saw that problem in the past when creating something in the wrong namespace. Try creating a page at en.opensuse.org/openSUSE:<name_your_page>
Lemme know if you still get that error.
Bryen
-- IRC: helen_au helen.south@opensuse.org helensouth.com
If I recall, I saw that problem in the past when creating something in the wrong namespace. Try creating a page at en.opensuse.org/openSUSE:<name_your_page>
Lemme know if you still get that error.
Bryen
I didn't get any error messages, but I didn't see any way to name the page: it defaulted to OpenSUSE; then I moved the page to Workspace: planning, which redirected that page to http://en.opensuse.org/Workspace:_planning
complete with the space, yuck. This is why I didn't want to go making the page... messy when you don't know what you're doing. I hope I haven't broken something.
On Sun, 2012-05-20 at 09:11 +1000, Helen South wrote:
complete with the space, yuck. This is why I didn't want to go making the page... messy when you don't know what you're doing. I hope I haven't broken something.
But breaking things is how you get things fixed that you didn't know needed fixing. :-)
Okay, let's wait for Rajko to jump in on this one. he's the resident guru.
Bryen
Le mercredi 16 mai 2012, à 07:22 -0500, Bryen M Yunashko a écrit :
On Wed, 2012-05-16 at 14:11 +0200, Vincent Untz wrote:
I'd really love to have a OBSOLETE resolution, so we could just mass-close those bugs with a comment saying "We believe this bug is most likely fixed in openSUSE 12.1 (or Factory). If you can still reproduce the bug there, please reopen the bug and move it to openSUSE 12.1." Alternatively, we can do a mass NEEDINFO-ing of all the bugs, and wait a few weeks before closing the bugs that didn't get a reply...
While I get the whole "Hey, its fixed in future release" thing, I think some people find this offputting. Again, its an issue of understanding the nature of how things work. We need a clearer explanation of why we WONTFIX it now in the current release. Some people want the fix now rather than feel forced to wait for the final release of next version. Nor might they actually plan on upgrading with each new release.
Not saying I agree with their thinking, but I can see how they might feel they're not being cared about if they're told "next release." Too many people out there are still accustomed to the concept of Service Packs to fix their current release.
And yes, I know we can't make everyone happy. :-)
I understand that too. But at this point it's really a choice between status quo (bugzilla is a black hole for many bugs), and try to clean up most of the old cruft so we can start fresh and attempt to avoid a similar black hole in the future.
(To do a proper clean up, we'd also need to significantly reduce the number of open 11.4 and 12.1 bugs, though...)
Vincent
Vincent Untz wrote:
Le mercredi 16 mai 2012, à 07:22 -0500, Bryen M Yunashko a écrit :
On Wed, 2012-05-16 at 14:11 +0200, Vincent Untz wrote:
I'd really love to have a OBSOLETE resolution, so we could just mass-close those bugs with a comment saying "We believe this bug is most likely fixed in openSUSE 12.1 (or Factory). If you can still reproduce the bug there, please reopen the bug and move it to openSUSE 12.1." Alternatively, we can do a mass NEEDINFO-ing of all the bugs, and wait a few weeks before closing the bugs that didn't get a reply...
While I get the whole "Hey, its fixed in future release" thing, I think some people find this offputting. Again, its an issue of understanding the nature of how things work. We need a clearer explanation of why we WONTFIX it now in the current release. Some people want the fix now rather than feel forced to wait for the final release of next version. Nor might they actually plan on upgrading with each new release.
Not saying I agree with their thinking, but I can see how they might feel they're not being cared about if they're told "next release." Too many people out there are still accustomed to the concept of Service Packs to fix their current release.
And yes, I know we can't make everyone happy. :-)
I understand that too. But at this point it's really a choice between status quo (bugzilla is a black hole for many bugs), and try to clean up most of the old cruft so we can start fresh and attempt to avoid a similar black hole in the future.
(To do a proper clean up, we'd also need to significantly reduce the number of open 11.4 and 12.1 bugs, though...)
Avoiding a future "black hole" and cleaning up the current one are really two separate, independent items. I think we have or can quickly conjure up a reasonable way for clearing up the current situation, but we have not yet really addressed how to avoid it in the future.
On Wed, 16 May 2012 15:29:00 +0200, Per Jessen wrote:
Avoiding a future "black hole" and cleaning up the current one are really two separate, independent items. I think we have or can quickly conjure up a reasonable way for clearing up the current situation, but we have not yet really addressed how to avoid it in the future.
Absolutely agreed 100%. What's more, I think that while the two efforts can be done in parallel, we should probably be a step ahead on prevention measures to keep it from happening in the future, otherwise we end up having to go through a cleanup exercise a second time (and possibly that's unavoidable).
Jim
On Wed, 16 May 2012 14:11:26 +0200, Vincent Untz wrote:
Hi,
Le mercredi 16 mai 2012, à 03:04 +0000, Jim Henderson a écrit :
- For older bugs, (such as on 10.3-11.3, which there are still open
bugs on), a determination needs to be made as to whether the current supported releases have those issues, and if they do, the bugs need to be updated. If they don't, then the bugs need to be flagged in an appropriate way.
In GNOME Bugzilla, we do have a OBSOLETE resolution. I think that would help in such cases.
I like this idea in concept. I'm not sure what it would take to get a new resolution added because of the constraints that were in place when I left Novell a year ago and the complexity of this particular bugzilla implementation.
If I can track down who owns it now, I'll see if I can find out. I still have some internal contacts who are indirectly involved in bugzilla (unless someone else here is closer to the bugzilla implementation team).
I'm happy to invest a few days to look at all the bugs and move the ones I feel are still valid for Factory to 12.2, but before doing that, I'd like us to know what's the right way to proceed for the bugs we don't care about.
I think the use of an OBSOLETE resolution is something that would be useful, but we have to be careful in actually using it - obviously there are some fixes (security in particular) where backporting a fix is generally considered a good idea. So we need to be extremely clear that the reason it's flagged obsolete is because (a) it's been resolved in a later release, and (b) a decision has been made (and that decision should at least be explained in the comments, IMHO) to not backport because of the effort/impact/whatever.
Comes back to being aware that our audience isn't always a technical audience.
I'd really love to have a OBSOLETE resolution, so we could just mass-close those bugs with a comment saying "We believe this bug is most likely fixed in openSUSE 12.1 (or Factory). If you can still reproduce the bug there, please reopen the bug and move it to openSUSE 12.1." Alternatively, we can do a mass NEEDINFO-ing of all the bugs, and wait a few weeks before closing the bugs that didn't get a reply...
Jim
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2012-05-16 14:11, Vincent Untz wrote:
c) reporters tend to not go back to their bugs to close them when they're fixed/obsolete.
No, I don't close my bugs because I expect an answer.
- -- Cheers / Saludos,
Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar)
On Thu, 17 May 2012 03:24:24 +0200, Carlos E. R. wrote:
On 2012-05-16 14:11, Vincent Untz wrote:
c) reporters tend to not go back to their bugs to close them when they're fixed/obsolete.
No, I don't close my bugs because I expect an answer.
Even after they've been fixed and confirmed to be fixed?
That's what Vincent's talking about - reporters don't tend to flag a bug as closed after it's fixed - the bug is marked fixed and that's the end of it.
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
Jim
Le 17/05/2012 03:33, Jim Henderson a écrit :
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
It was never clear for me to know who have to close the bug.
My feeling was that it is the people that fix the bug that close it
jdd
On Thu, 17 May 2012 10:41:46 +0200, jdd wrote:
Le 17/05/2012 03:33, Jim Henderson a écrit :
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
It was never clear for me to know who have to close the bug.
My feeling was that it is the people that fix the bug that close it
I think that's one of the things that needs to be clearly stated. I've seen it done both ways, where the original reporter closes the bug, and where the original reporter marks it as 'verified' and the developer or change control board closes it.
Jim
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thursday, 2012-05-17 at 01:33 -0000, Jim Henderson wrote:
On Thu, 17 May 2012 03:24:24 +0200, Carlos E. R. wrote:
On 2012-05-16 14:11, Vincent Untz wrote:
c) reporters tend to not go back to their bugs to close them when they're fixed/obsolete.
No, I don't close my bugs because I expect an answer.
Even after they've been fixed and confirmed to be fixed?
I understand that it is the bug owner who has to close them, not the reporter.
That's what Vincent's talking about - reporters don't tend to flag a bug as closed after it's fixed - the bug is marked fixed and that's the end of it.
It is never the procedure as was explained to me years ago, nor as I have seen similar procedures in the enterprise. The reporter simply accepts or rejects the closure (by reopening).
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
But not by the reporter! If this has changed, I don't know about that. And the same I don't know about that, many reporters do not know that, and do not expect that.
- -- Cheers, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar)
I understand that it is the bug owner who has to close them, not the reporter.
That is the usual way, but it needn't be. Whoever deems the bug solved can close a bug. What few if any folks do is mark the bug as verified to explicitly state that the bug has been tested t be fixed.
The reporter simply accepts or rejects the closure (by reopening).
No, he/she is free to close the bug.
And the same I don't know about that, many reporters do not know that, and do not expect that.
So what we need is a wiki page describing the possible flows of states.
Philipp
So what we need is a wiki page describing the possible flows of states.
Philipp
Fedora has quite a detailed set of pages including a nice flowchart: http://fedoraproject.org/wiki/BugZappers/BugStatusWorkFlow
(bearing in mind, again, that it's always an issue to actually get people to Read The Flippin Manual. I know, I'm guilty :) )
One nice thing I just saw on Firefox: When updating, there was a one-question survey on the front page. Just one. A quick click. I think we need to be opportunistic about conveying information in bite-size chunks. Install-time message screens, facebook updates....
On Thu, 17 May 2012 20:45:57 +1000 Helen South helen.south@opensuse.org wrote:
I think we need to be opportunistic about conveying information in bite-size chunks. Install-time message screens, facebook updates....
My thought too.
Also, help is often written in a language that requires much more further reading then average user is willing to spend, so when thinking of concise help we should consider requirements for additional reading and at least give pointers to articles that we consider relevant and accurate.
On Sat, 19 May 2012 16:09:36 -0500, Rajko wrote:
On Thu, 17 May 2012 20:45:57 +1000 Helen South helen.south@opensuse.org wrote:
I think we need to be opportunistic about conveying information in bite-size chunks. Install-time message screens, facebook updates....
My thought too.
Also, help is often written in a language that requires much more further reading then average user is willing to spend, so when thinking of concise help we should consider requirements for additional reading and at least give pointers to articles that we consider relevant and accurate.
Part of making this work is going to be getting feedback from the target audience about what works and what doesn't.
Jim
On Thursday 17 of May 2012 01:33EN, Jim Henderson wrote:
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
Bugzilla documentation says something different, according to it the RESOLVED -> VERIFIED -> CLOSED path is for QA, not for the reporter. See https://bugzilla.novell.com/page.cgi?id=fields.html#status
Michal Kubeček
Dne Čt 17. května 2012 12:05:28 Michal Kubeček napsal(a):
On Thursday 17 of May 2012 01:33EN, Jim Henderson wrote:
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
Bugzilla documentation says something different, according to it the RESOLVED -> VERIFIED -> CLOSED path is for QA, not for the reporter. See https://bugzilla.novell.com/page.cgi?id=fields.html#status
Unfortunately this documentation is valid only for unreleased products. For a defect that is found after a given version of openSUSE is shipped (the large majority of them), it should be set to CLOSED when QA verifies the fix. But there's no QA for openSUSE! (Correct me if I'm wrong on this last statement.)
Without any QA in place, RESOLVED is the final state for bugs found after release.
Just my two cents, Petr
On Thu, 17 May 2012 12:05:28 +0200, Michal Kubeček wrote:
On Thursday 17 of May 2012 01:33EN, Jim Henderson wrote:
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
Bugzilla documentation says something different, according to it the RESOLVED -> VERIFIED -> CLOSED path is for QA, not for the reporter. See https://bugzilla.novell.com/page.cgi?id=fields.html#status
Michal Kubeček
I've seen it done both ways, but yes, this way is the way that is probably most common.
Jim
Le jeudi 17 mai 2012, à 01:33 +0000, Jim Henderson a écrit :
On Thu, 17 May 2012 03:24:24 +0200, Carlos E. R. wrote:
On 2012-05-16 14:11, Vincent Untz wrote:
c) reporters tend to not go back to their bugs to close them when they're fixed/obsolete.
No, I don't close my bugs because I expect an answer.
Even after they've been fixed and confirmed to be fixed?
That's what Vincent's talking about - reporters don't tend to flag a bug as closed after it's fixed - the bug is marked fixed and that's the end of it.
But Bugzilla workflow includes a bug being flagged as fixed, the reporter verifies that it's fixed, and the bug is then closed.
Sorry, I guess I added some confusion with my mail :/ As far as I know, we don't use the "FIXED -> VERIFIED -> CLOSED" workflow. I was really just talking about old bugs that the reporters can test again in more recent versions of openSUSE.
Vincent
On Mon, 21 May 2012 10:08:19 +0200, Vincent Untz wrote:
Sorry, I guess I added some confusion with my mail :/ As far as I know, we don't use the "FIXED -> VERIFIED -> CLOSED" workflow. I was really just talking about old bugs that the reporters can test again in more recent versions of openSUSE.
Ah, I see - thanks for the clarification, Vincent.
Jim
Le jeudi 17 mai 2012, à 03:24 +0200, Carlos E. R. a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2012-05-16 14:11, Vincent Untz wrote:
c) reporters tend to not go back to their bugs to close them when they're fixed/obsolete.
No, I don't close my bugs because I expect an answer.
Do you still expect answers for openSUSE 10.3? My point is that developers/packagers/triagers/etc. all try to close bugs when they're fixed, but it happens that some bugs are just forgotten. And in that case, the reporter can help too and say "oh, this 10.3 bug was fixed a long time ago, let's close it" after checking things.
Cheers,
Vincent
On 16.05.2012 05:04, Jim Henderson wrote:
- There should be some sort of "screening" that takes place before a
bug is accepted - to ensure that the devs aren't seeing multiple copies of the same bugs. I think it's fair to say that those who write code would far rather *write code* than try to sort through duplicate bug entries. While it's likely there still would be dupes with a screening process, the incidence should be reduced.
By default get assigned to the bnc-screening team who then assign to the package maintainer(s), however without any evaluation. Who are these people actually? It looks to me like Chinese SUSE/Novell staff. That would be a place to expand upon if we could find voluteers subscribing to the screening list and trying to do some further work such as classifying, possibly triaging and identifying duplicates before assigning to the actual maintainers. That doesn't require much technical skills the hard part is luring new people into contributing first.
- There needs to be more of a "human" reply, especially on older bugs,
that are being addressed. Looking at the current Bugzilla stats, I see that (for example) 12.1 has 183 bugs in a "NEEDINFO" state - there needs to be some sort of consistent follow-up to ensure that the info needed to fix the issues is provided, and if it isn't and the information can't be obtained (because the issue cannot be duped, those looking at it don't have the necessary hardware, etc), then the bug needs to be closed.
- For older bugs, (such as on 10.3-11.3, which there are still open
bugs on), a determination needs to be made as to whether the current supported releases have those issues, and if they do, the bugs need to be updated. If they don't, then the bugs need to be flagged in an appropriate way.
I think it would be appropriate to automaitically send a reminder to those bugs against unsupported releases in NEEDINFO state asking people to triage against a supported version of oS and then to close those not receiving any response in a couple of weeks. That's better than simply WONTFIX'ing in case there are still people that care about them. In fact that is what I already do with bugs against my packages.
On Wed, 16 May 2012 14:31:50 +0200, Guido Berhoerster wrote:
On 16.05.2012 05:04, Jim Henderson wrote:
- There should be some sort of "screening" that takes place before a
bug is accepted - to ensure that the devs aren't seeing multiple copies of the same bugs. I think it's fair to say that those who write code would far rather *write code* than try to sort through duplicate bug entries. While it's likely there still would be dupes with a screening process, the incidence should be reduced.
By default get assigned to the bnc-screening team who then assign to the package maintainer(s), however without any evaluation. Who are these people actually? It looks to me like Chinese SUSE/Novell staff. That would be a place to expand upon if we could find voluteers subscribing to the screening list and trying to do some further work such as classifying, possibly triaging and identifying duplicates before assigning to the actual maintainers. That doesn't require much technical skills the hard part is luring new people into contributing first.
I think this probably comes back to providing some entry-level training those who want to help but aren't sure what's involved. Bugzilla - for all its good points - does present a pretty intimidating interface to the uninitiated.
Maybe we could organize a live online-session every couple of months (just throwing ideas out here) for those who want to help with bug squashing but aren't sure how they can. A live Q&A often times will have a less intimidating feel, because if discussion is encouraged by the moderator/coordinator, a good discussion can occur that gets people excited about helping out.
I think if we can get users to feel some sense of ownership - that openSUSE is /their/ distribution, that will help feed the process. / Especially/ if they can see the results of their contributions in a real and tangible way.
- There needs to be more of a "human" reply, especially on older
bugs, that are being addressed. Looking at the current Bugzilla stats, I see that (for example) 12.1 has 183 bugs in a "NEEDINFO" state - there needs to be some sort of consistent follow-up to ensure that the info needed to fix the issues is provided, and if it isn't and the information can't be obtained (because the issue cannot be duped, those looking at it don't have the necessary hardware, etc), then the bug needs to be closed.
- For older bugs, (such as on 10.3-11.3, which there are still open
bugs on), a determination needs to be made as to whether the current supported releases have those issues, and if they do, the bugs need to be updated. If they don't, then the bugs need to be flagged in an appropriate way.
I think it would be appropriate to automaitically send a reminder to those bugs against unsupported releases in NEEDINFO state asking people to triage against a supported version of oS and then to close those not receiving any response in a couple of weeks. That's better than simply WONTFIX'ing in case there are still people that care about them. In fact that is what I already do with bugs against my packages.
That's a very good idea (I assume you mean "asking people to test" rather than "triage"). :)
Jim
On Wed, 2012-05-16 at 16:18 +0000, Jim Henderson wrote:
Maybe we could organize a live online-session every couple of months (just throwing ideas out here) for those who want to help with bug squashing but aren't sure how they can. A live Q&A often times will have a less intimidating feel, because if discussion is encouraged by the moderator/coordinator, a good discussion can occur that gets people excited about helping out.
Good documentation and training is always helpful. But we can't guarantee that people go to these places. We tell someone "File a bug!" and point them to bugzilla.novell.com. They're faced with a daunting interface right from the get-go.
And let's be honest. People don't want to read, read, read. They encounter a problem and just want to report it and be done with it. So our first step should be making bugzilla less daunting (if that's even possible.) And again, some people have suggested in the past to do away with bugzilla and start anew with either a dedicated openSUSE bugzilla or some other bug tracking software.
I have a feeling that while we're addressing resolving existing reported bugs, we probably have a wealth of unreported bugs out there simply because people feel uncomfortable reporting them. :-)
Bryen
Le 16/05/2012 18:41, Bryen M Yunashko a écrit :
And let's be honest. People don't want to read, read, read. They encounter a problem and just want to report it and be done with it.
I don't think so.
people want they bug *fixed*. It's of no use to have more bugs reported if we can't fix them...
now bugzilla is frightening, so only solid people report, and then this is not enough to have good reports and fixed bugs.
So, IMHO, we should first have *less* bugs, but *more* solved
Most bug report have to be managed extremely fast, because it's often difficult to keep a buggy system only waiting a needinfo
The best ever debugging system I experienced was an online chat system. Not IRC, pear to pear (one to one) system.
I don't imagine we can manage this. The main advantage is that it works with a simple browser, no special IRC config
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
jdd
On Wed, 2012-05-16 at 19:01 +0200, jdd wrote:
The best ever debugging system I experienced was an online chat system. Not IRC, pear to pear (one to one) system.
I don't imagine we can manage this. The main advantage is that it works with a simple browser, no special IRC config
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
jdd
Conversing with people is indeed a useful tool. It helps to identify whether the bug is already known or whether it really is a bug and not a user-caused issue. I frequently utilize my presence on IRC to vet out my issues before deciding whether to file a bug somewhere.
But that's because IRC is second-nature to me and part of my integrated computing usage on a daily basis. That's not the case for many people, and it would be next to impossible to come up with a system to require IRC discussion before filing a bug. Not to mention bug filing isn't just for users, developers also file bugs on their own projects in order to keep track of their own work.
Moreover, requiring human interface before filing a bug... well, that sounds similarly close to phone-support, and we don't really offer that kind of support. That's usually something for paid software distributors.
Bryen
On Wed, 16 May 2012 19:01:20 +0200, jdd wrote:
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
My inclination would be not to require a specific interaction (IRC, MLs, forums, etc), but to strongly encourage that before a bug be submitted that it be discussed in one of our communications venues to make sure it isn't, say, a configuration error.
Statistically speaking, support organizations tend to spend much less time on software defects when working with customers, but find that most support incidents are related to either usability or configuration (or some combination of the two).
So there is clearly value in that interaction with others before logging a bug.
I wonder if there's any way we can get a sense out of the resolved bugs how many of them were configuration or user education issues rather than bona fide bugs?
Jim
Jim Henderson wrote:
On Wed, 16 May 2012 19:01:20 +0200, jdd wrote:
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
My inclination would be not to require a specific interaction (IRC, MLs, forums, etc), but to strongly encourage that before a bug be submitted that it be discussed in one of our communications venues to make sure it isn't, say, a configuration error.
Statistically speaking, support organizations tend to spend much less time on software defects when working with customers, but find that most support incidents are related to either usability or configuration (or some combination of the two).
So there is clearly value in that interaction with others before logging a bug.
Most definitely - and we already do that in the fora and on the mailing lists. Quite well too, I think.
I wonder if there's any way we can get a sense out of the resolved bugs how many of them were configuration or user education issues rather than bona fide bugs?
In theory bona fide bugs should lead to updates, which would mean the report having been assigned with NEEDINFO to maintenance@opensuse.org.
On Wed, 16 May 2012 21:14:51 +0200, Per Jessen wrote:
So there is clearly value in that interaction with others before logging a bug.
Most definitely - and we already do that in the fora and on the mailing lists. Quite well too, I think.
Overall, yes - but I think we might do better in promoting that idea to new users - again I'm thinking about the landing page.
I wonder if there's any way we can get a sense out of the resolved bugs how many of them were configuration or user education issues rather than bona fide bugs?
In theory bona fide bugs should lead to updates, which would mean the report having been assigned with NEEDINFO to maintenance@opensuse.org.
Cool.
Seems like in addition to the existing discussion, we need to start documenting what we do today. We can then identify the gap between what we do and what we should be doing, and plan how to get from one place to the other.
Jim
Hello,
Am Mittwoch, 16. Mai 2012 schrieb Jim Henderson:
I wonder if there's any way we can get a sense out of the resolved bugs how many of them were configuration or user education issues rather than bona fide bugs?
The number of FIXED bugs in relation to the number of bugs with other resolutions is a good measurement IMHO.
I used bugzilla reporting to generate an overview of bugs by resolution for various openSUSE versions: http://bit.ly/KzuDVL
Those numbers are hard to read, therefore I exported them as CSV and made some nice graphs out of them which - show the percentage of bugs per resolution - for all openSUSE releases since 10.0
Actually I made several graphs:
a) all bugs (including open bugs) www.cboltz.de/tmp/opensuse-bug-statistics-1.png
a) closed bugs www.cboltz.de/tmp/opensuse-bug-statistics-2.png
b) closed bugs, filtered view without DUPLICATE (they don't count) and NORESPONSE (which is the bugreporter's fault) - in other words: bugs in this graph were closed purely by the developer's decision. www.cboltz.de/tmp/opensuse-bug-statistics-3.png
d) total number of bugreports (by numbers, not by percentage) www.cboltz.de/tmp/opensuse-bug-statistics.png
Some interesting details: - of course 12.1 and 12.2 have the biggest number of open bugreports, but OTOH still 12% of the 11.3 bugs and 28% of the 11.4 bugs are open. That's a huge number of possibly obsolete bugs. - about 15% of all (closed) bugreports are duplicates - and this number didn't change much over the openSUSE releases. - about 5-9% of the (closed) bugreports are closed because of missing feedback (NORESPONSE) - the number of bugreports is nearly stable since 11.3 (about 4000 bugreports per release). Earlier releases had more bugreports. At least for 11.1, I'd bet this was caused by ZENworks *g,d&r*
To get back to the initial question how many bugreports are PEBKAC: If you sum up INVALID, WORKSFORME and WONTFIX, you'll get a total of 20-30%. That's already a pessimistic calculation because WONTFIX doesn't necessarily mean PEBKAC, it can also mean a developer decided not to fix something due to lack of time etc., and WORKSFORME also includes bugs that are not reproducable, but might still be valid.
Let me finish this mail with good news: More than 50% of the closed bugreports are marked as FIXED :-) If you ignore DUPLICATE and NORESPONSE (which are not the developer's fault), we even have 60-70% of all bugs fixed.
Regards,
Christian Boltz
PS: Those graphs won't be online forever - there's a reason I have /tmp/ in the path. Feel free to save them or even upload them whereever you want if you want a permanent copy.
On Thu, 17 May 2012 01:40:05 +0200, Christian Boltz wrote:
Am Mittwoch, 16. Mai 2012 schrieb Jim Henderson:
I wonder if there's any way we can get a sense out of the resolved bugs how many of them were configuration or user education issues rather than bona fide bugs?
The number of FIXED bugs in relation to the number of bugs with other resolutions is a good measurement IMHO.
Good thought - I've been interspersing this discussion with "day job" work, so hadn't gotten into the resolution states yet. NORESPONSE is probably one that doesn't make sense to classify as config/user ed issue, but most of the others others probably are.
I used bugzilla reporting to generate an overview of bugs by resolution for various openSUSE versions: http://bit.ly/KzuDVL [...]
Outstanding data analysis - that's extremely helpful.
Thanks,
Jim
jdd wrote:
Le 16/05/2012 18:41, Bryen M Yunashko a écrit :
And let's be honest. People don't want to read, read, read. They encounter a problem and just want to report it and be done with it.
I don't think so.
people want they bug *fixed*. It's of no use to have more bugs reported if we can't fix them...
Actually fixing a bug and getting the fix installed on a user system is quite a long process, especially for inexperienced users.
now bugzilla is frightening, so only solid people report, and then this is not enough to have good reports and fixed bugs.
So, IMHO, we should first have *less* bugs, but *more* solved
Less bugs reported cannot be good for us unless it actually means less bugs in openSUSE. The number of bugs reported, regardless of what happens to them, is a strong indicator of interest and quality.
Most bug report have to be managed extremely fast, because it's often difficult to keep a buggy system only waiting a needinfo
That is an individual matter, but I agree, most (potential) reporters probably have just one system available, which does make our response-time critical. Factory is a different matter altogether.
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
No doubt very useful, but hardly possible for a non-commercial operation.
On Thu, May 17, 2012 at 5:05 AM, Per Jessen per@computer.org wrote:
jdd wrote:
Le 16/05/2012 18:41, Bryen M Yunashko a écrit :
And let's be honest. People don't want to read, read, read. They encounter a problem and just want to report it and be done with it.
I don't think so.
I disagree, Bryen is right - people don't want loads of reading. Everyone is very busy. I NEVER watch videos, I just won't spend ten minutes watching some poor quality explanation of something I can read in thirty seconds. But we need to find some middle road: we are constantly pushing the fact that we are user-friendly enough for less expert users, so we're going to have to deal with bugs from those users.
people want they bug *fixed*. It's of no use to have more bugs reported if we can't fix them...
Actually fixing a bug and getting the fix installed on a user system is quite a long process, especially for inexperienced users.
This is important. We can include some documentation at some point in the process - perhaps a thank-you note post-submission (once someone has invested the time to submit, they are more likely to read it) - that explains this: That the bug fix may take a long time to implement, and they may need to do a fresh install.
now bugzilla is frightening, so only solid people report, and then this is not enough to have good reports and fixed bugs.
Agreed, it is daunting. What if the the landing-page had two simple options: one, "I'm a technical user and have a Bugzilla login, send me there!" and two "I want to report a bug and haven't used bugzilla. Take me to the easy forum OR show me how to use bugzilla" and have a dedicated forum thread for bug reporting, where we can more easily discuss a bug and a third party can put it on bugzilla. The issue there would be (a) personnel to do the escalating and (b)possible loss of information.
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
No doubt very useful, but hardly possible for a non-commercial operation.
That'd put off a lot of users. Many don't use IRC. Could say the same for forums, but I think people are more comfortable with forums generally.
Another idea is that we have some non-technical flags on a bug: stops install process/affects configuration/security/ random UI problems/obstacle to productivity/printing/ specific software problems/no sound/visual glitches/ other small things that are annoying /trivial issue that looks unprofessional - that might allow a keen but less technical user to assist with triage.
On Thu, 17 May 2012 08:33:19 +1000, Helen South wrote:
On Thu, May 17, 2012 at 5:05 AM, Per Jessen per@computer.org wrote:
jdd wrote:
Le 16/05/2012 18:41, Bryen M Yunashko a écrit :
And let's be honest. People don't want to read, read, read. They encounter a problem and just want to report it and be done with it.
I don't think so.
I disagree, Bryen is right - people don't want loads of reading. Everyone is very busy. I NEVER watch videos, I just won't spend ten minutes watching some poor quality explanation of something I can read in thirty seconds. But we need to find some middle road: we are constantly pushing the fact that we are user-friendly enough for less expert users, so we're going to have to deal with bugs from those users.
Well, yes, but in terms of overall users (rather than individual experiences), I think there's benefit to (a) having things documented, and (b) providing those short snippets for those who learn that way.
People learn in different ways - myself, I'm far more kinesthetic (hands- on) in learning that those who can read something and get it. But I also have no problem just diving in in most cases and starting to do rather than wanting to be told what/how to do something so I don't break it.
From a training perspective, if we can come up with some content that can
be re-purposed for different learning methods, we can cover a lot of ground fairly easily. But the learning "industry" has been moving away from more monolithic methods of learning (read: instructor-led classroom- based training) and more towards more agile methods, such as podcasts/ video/short online sessions/etc.
To successfully get our community members (especially the non-technical ones) involved, we need to engage them in a way that they are able/ willing to consume. That's why we have diversity in our communications mediums with forums, mailing lists, IRC, and so on.
As my former boss (when I was in Novell's training department) said to me once, if he wants to repair a hole in his roof, he's going to use Google to find a video of how to fix a hole in his roof. He's not going to take a class on how to put a roof on a house.
The same principle applies in this instance. We need to understand how our community *in general* consumes information in order to get involved, and then provide it in a format that they are comfortable with.
It's also axiomatic, I think, to say that the way you get people involved in something they're not comfortable with is start with something they are comfortable with. So the delivery mechanism matters, and it's not going to be just one method - because that won't reach as much of the audience.
people want they bug *fixed*. It's of no use to have more bugs reported if we can't fix them...
Actually fixing a bug and getting the fix installed on a user system is quite a long process, especially for inexperienced users.
This is important. We can include some documentation at some point in the process - perhaps a thank-you note post-submission (once someone has invested the time to submit, they are more likely to read it) - that explains this: That the bug fix may take a long time to implement, and they may need to do a fresh install.
Agreed. Follow-through is important, as is setting the expectations (and then meeting the expectations that are set - otherwise setting the expectation is meaningless).
Agreed, it is daunting. What if the the landing-page had two simple options: one, "I'm a technical user and have a Bugzilla login, send me there!" and two "I want to report a bug and haven't used bugzilla. Take me to the easy forum OR show me how to use bugzilla" and have a dedicated forum thread for bug reporting, where we can more easily discuss a bug and a third party can put it on bugzilla. The issue there would be (a) personnel to do the escalating and (b)possible loss of information.
I like this. Another idea would be to have some mentors available to help guide someone through the process of submitting a bug. Using something like Teamviewer (would love to hear about a good OSS alternative to this, too) to do a live demonstration and to gather information could well work. Never underestimate the power of a personal interaction.
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
No doubt very useful, but hardly possible for a non-commercial operation.
That'd put off a lot of users. Many don't use IRC. Could say the same for forums, but I think people are more comfortable with forums generally.
Another idea is that we have some non-technical flags on a bug: stops install process/affects configuration/security/ random UI problems/obstacle to productivity/printing/ specific software problems/no sound/visual glitches/ other small things that are annoying /trivial issue that looks unprofessional - that might allow a keen but less technical user to assist with triage.
Sort of a self-classification system that's in lay terms? That could be good. We'd want to get some of the target audience involved in crafting the wording. Doing that would help ensure that we used wording that made sense to non-tech people, and of course would help increase buy-in.
Jim
On Thu, 2012-05-17 at 00:14 +0000, Jim Henderson wrote:
Well, yes, but in terms of overall users (rather than individual experiences), I think there's benefit to (a) having things documented, and (b) providing those short snippets for those who learn that way.
People learn in different ways - myself, I'm far more kinesthetic (hands- on) in learning that those who can read something and get it. But I also have no problem just diving in in most cases and starting to do rather than wanting to be told what/how to do something so I don't break it.
The point isn't whether we should or shouldn't include such materials (however that training material is going to be delivered.) The point is to ensure we realize that there has to be a balance between having such materials and making interfaces reasonably intuitive enough.
The interface really should be somthing that is intuitive enough that people can get a reasonable start and if they want to know more, can be pointed to detailed documentation and training materials.
Bryen
On Wed, 16 May 2012 19:20:53 -0500, Bryen M Yunashko wrote:
The point isn't whether we should or shouldn't include such materials (however that training material is going to be delivered.) The point is to ensure we realize that there has to be a balance between having such materials and making interfaces reasonably intuitive enough.
Ah, I see - somehow I missed that point. Thanks for setting me straight. :)
The interface really should be somthing that is intuitive enough that people can get a reasonable start and if they want to know more, can be pointed to detailed documentation and training materials.
Agreed. Training that's inaccessible is not terribly useful, we're agreed on that. :)
Jim
On Thu, 2012-05-17 at 00:14 +0000, Jim Henderson wrote:
People learn in different ways - myself, I'm far more kinesthetic (hands- on) in learning that those who can read something and get it. But I also have no problem just diving in in most cases and starting to do rather than wanting to be told what/how to do something so I don't break it.
From a training perspective, if we can come up with some content that can be re-purposed for different learning methods, we can cover a lot of ground fairly easily. But the learning "industry" has been moving away from more monolithic methods of learning (read: instructor-led classroom- based training) and more towards more agile methods, such as podcasts/ video/short online sessions/etc.
We could adapt something akin to codeacademy.com. That is a brilliant way to teach programming, and I think it could be adaptable to other areas.
On Thu, 17 May 2012 11:32:53 -0700, Roger Luedecke wrote:
From a training perspective, if we can come up with some content that can be re-purposed for different learning methods, we can cover a lot of ground fairly easily. But the learning "industry" has been moving away from more monolithic methods of learning (read: instructor-led classroom- based training) and more towards more agile methods, such as podcasts/ video/short online sessions/etc.
We could adapt something akin to codeacademy.com. That is a brilliant way to teach programming, and I think it could be adaptable to other areas.
That's a good thought, Roger. Definitely an option worth looking at.
Jim
On Thu, 2012-05-17 at 08:33 +1000, Helen South wrote:
On Thu, May 17, 2012 at 5:05 AM, Per Jessen per@computer.org wrote:
jdd wrote:
Le 16/05/2012 18:41, Bryen M Yunashko a écrit :
And let's be honest. People don't want to read, read, read. They encounter a problem and just want to report it and be done with it.
I don't think so.
I disagree, Bryen is right - people don't want loads of reading. Everyone is very busy. I NEVER watch videos, I just won't spend ten minutes watching some poor quality explanation of something I can read in thirty seconds. But we need to find some middle road: we are constantly pushing the fact that we are user-friendly enough for less expert users, so we're going to have to deal with bugs from those users.
Agreed. Concise written explanations with an eye to the n00b is ideal. Videos aren't a bad OPTION to have linked in the articles.
people want they bug *fixed*. It's of no use to have more bugs reported if we can't fix them...
Actually fixing a bug and getting the fix installed on a user system is quite a long process, especially for inexperienced users.
This is important. We can include some documentation at some point in the process - perhaps a thank-you note post-submission (once someone has invested the time to submit, they are more likely to read it) - that explains this: That the bug fix may take a long time to implement, and they may need to do a fresh install.
A little encouragement and personal interaction go a long way. There are times one will file bugs, and they never hear a thing.
now bugzilla is frightening, so only solid people report, and then this is not enough to have good reports and fixed bugs.
Agreed, it is daunting. What if the the landing-page had two simple options: one, "I'm a technical user and have a Bugzilla login, send me there!" and two "I want to report a bug and haven't used bugzilla. Take me to the easy forum OR show me how to use bugzilla" and have a dedicated forum thread for bug reporting, where we can more easily discuss a bug and a third party can put it on bugzilla. The issue there would be (a) personnel to do the escalating and (b)possible loss of information.
Brilliant! Though, we could possibly have a simple (and sloppy?) front end available. Maybe even as a widget. Call it Stomper.
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
No doubt very useful, but hardly possible for a non-commercial operation.
That'd put off a lot of users. Many don't use IRC. Could say the same for forums, but I think people are more comfortable with forums generally.
Good point. I take it for granted that I know IRC now, but when I was first learning it, there was a nasty learning curve. This may be feasible if we had a web-based front-end that looks like a regular 'ol chat room plugin. Then it is presented as a paradigm familiar to more users than raw IRC.
Another idea is that we have some non-technical flags on a bug: stops install process/affects configuration/security/ random UI problems/obstacle to productivity/printing/ specific software problems/no sound/visual glitches/ other small things that are annoying /trivial issue that looks unprofessional - that might allow a keen but less technical user to assist with triage.
VERY good idea. I may not be useful in a kernel panic, but I can be useful in many other areas.
On Wed, 2012-05-16 at 21:05 +0200, Per Jessen wrote:
could it be possible / usefull to *require* an IRC chat *before* reporting a bug????
No doubt very useful, but hardly possible for a non-commercial operation.
IRC tends to be far less effective than forums anyway. The forums are really the place to get excellent and timely assistance.
On Wed, 16 May 2012 11:41:05 -0500, Bryen M Yunashko wrote:
Good documentation and training is always helpful. But we can't guarantee that people go to these places. We tell someone "File a bug!" and point them to bugzilla.novell.com. They're faced with a daunting interface right from the get-go.
True. But what we can do is encourage people to go there. There are bound to be people who want to report bugs but are intimidated by the process because they're unfamiliar with it.
So the goal of such material would be to lower that barrier to use.
And let's be honest. People don't want to read, read, read. They encounter a problem and just want to report it and be done with it. So our first step should be making bugzilla less daunting (if that's even possible.) And again, some people have suggested in the past to do away with bugzilla and start anew with either a dedicated openSUSE bugzilla or some other bug tracking software.
Sure. I'm thinking more like online demonstrations, no more than say 5 minutes or so, that cover essential operations.
The way people learn in the modern digital age is in small snippets. They need to be able to (a) find the relevant information, and (b) quickly consume it so they can apply it.
Khan Academy's model would, I think, be very appropriate to what we're doing here. Audio/video demo that covers an atomic operation in a short period of time, rather than a top-down overview of the entire system that takes a couple hours to consume and digest.
I have a feeling that while we're addressing resolving existing reported bugs, we probably have a wealth of unreported bugs out there simply because people feel uncomfortable reporting them. :-)
One of the outcomes I'm expecting is that when we make it easier to report bugs, more bugs will be reported. That's why it's important to have a scalable process for evaluating and prioritizing bugs before we start opening the floodgates - otherwise we'll end up with something that's even less manageable.
Jim
On 05/16/2012 07:44 PM, Jim Henderson wrote:
On Wed, 16 May 2012 11:41:05 -0500, Bryen M Yunashko wrote:
Good documentation and training is always helpful. But we can't guarantee that people go to these places. We tell someone "File a bug!" and point them to bugzilla.novell.com. They're faced with a daunting interface right from the get-go.
True. But what we can do is encourage people to go there. There are bound to be people who want to report bugs but are intimidated by the process because they're unfamiliar with it.
So the goal of such material would be to lower that barrier to use.
Not only: Lower the barrier to and also help to create better bug reports.
The better the bug report is - the less time the developer needs to ask questions back - the sooner it gets fixed.
an example: A developer just told me that function vsnprintf is broken. I wrote a small test program that works for me and had to ask back what is different. If I would have received a test program that fails and that I could execute on my own, the bug would be fixed already - but now I'm waiting...
Andreas
On Fri, 18 May 2012 16:25:05 +0200, Andreas Jaeger wrote:
On 05/16/2012 07:44 PM, Jim Henderson wrote:
On Wed, 16 May 2012 11:41:05 -0500, Bryen M Yunashko wrote:
Good documentation and training is always helpful. But we can't guarantee that people go to these places. We tell someone "File a bug!" and point them to bugzilla.novell.com. They're faced with a daunting interface right from the get-go.
True. But what we can do is encourage people to go there. There are bound to be people who want to report bugs but are intimidated by the process because they're unfamiliar with it.
So the goal of such material would be to lower that barrier to use.
Not only: Lower the barrier to and also help to create better bug reports.
Absolutely. Bug reports that include the proper information are important.
But that also plays into the general issue of getting the right information when someone's asking for help. We see this on the forums / all the time/.
When the question boils down to "my thing's broke", it's very difficult to help someone. Takes a little guidance, especially for the non- technical, because they don't know the terminology (generally), so even if we ask for the boot log (for example), they may not know how or where to get it.
Similarly, the short description (in a bugzilla report) or subject (in the forum or on the ML) is important. "GNOME Gripe #1" isn't descriptive or useful.
I encourage people to read Eric S. Raymond's essay on "how to ask questions the intelligent way", and am working on a blog entry that helps define what you should include in your first post in the forum on an issue.
But that is a perennial problem, as long as I've been doing online support, people have neglected to provide full or complete information when starting to ask for help. (And lest you think I was joking earlier, I actually did have someone post a message on CompuServe years ago that had a subject line of "Help" and the entire text of the message was, I kid you not, "My thing's broke.").
Jim
But that is a perennial problem, as long as I've been doing online support, people have neglected to provide full or complete information when starting to ask for help. (And lest you think I was joking earlier, I actually did have someone post a message on CompuServe years ago that had a subject line of "Help" and the entire text of the message was, I kid you not, "My thing's broke.").
Jim
I can verify that you're not joking. I've seen some shockers on the University forums. While most openSUSE users will have at least a bit of knowledge, our increasingly broad userbase means some screening is necessary. In some cases, no amount of documentation will make a difference, but for keen beginners, there's definite value in guiding them through the process.
At this point, before we start making decisions, I feel it would be helpful to lay out a clearer picture of what is happening where. There's no point in worrying about screening if these are only a tiny minority of trivial bugs - that a dev can within moments decide aren't worth attention and mark with "Thank you for submitting this bug, however information is incomplete therefore we're marking it 'unactionable'" or something. More important is if genuine bugs are being delayed or problematized by purely because the reporter made mistakes with Bugzilla, or there's real timewasting going on trying to obtain information and so forth.
Need data. Qualitative and quantitative. :)
On Sat, 19 May 2012 12:31:43 +1000, Helen South wrote:
I can verify that you're not joking. I've seen some shockers on the University forums. While most openSUSE users will have at least a bit of knowledge, our increasingly broad userbase means some screening is necessary. In some cases, no amount of documentation will make a difference, but for keen beginners, there's definite value in guiding them through the process.
Yup. :)
At this point, before we start making decisions, I feel it would be helpful to lay out a clearer picture of what is happening where. There's no point in worrying about screening if these are only a tiny minority of trivial bugs - that a dev can within moments decide aren't worth attention and mark with "Thank you for submitting this bug, however information is incomplete therefore we're marking it 'unactionable'" or something. More important is if genuine bugs are being delayed or problematized by purely because the reporter made mistakes with Bugzilla, or there's real timewasting going on trying to obtain information and so forth.
Need data. Qualitative and quantitative. :)
Absolutely. I'm a huge fan of data-driven decision making. :)
Jim
Hey,
On 16.05.2012 18:41, Bryen M Yunashko wrote:
Good documentation and training is always helpful. But we can't guarantee that people go to these places. We tell someone "File a bug!" and point them to bugzilla.novell.com. They're faced with a daunting interface right from the get-go.
What do you find so daunting about the guided interface that we link to everywhere? Maybe we can improve that one.
https://bugzilla.novell.com/enter_bug.cgi?product=openSUSE+12.2&format=g...
But I fear you can't make bugreporting _much_ simpler without placing more burden on a first-level screening...
Henne
On Mon, 21 May 2012 15:15:40 +0200, Henne Vogelsang wrote:
What do you find so daunting about the guided interface that we link to everywhere?
Hmmm, I've never seen that one. I'll make sure the forums staff knows about it so we point people using that URL rather than the URL to the standard bug reporting interface.
Jim
Jim Henderson wrote:
On Mon, 21 May 2012 15:15:40 +0200, Henne Vogelsang wrote:
What do you find so daunting about the guided interface that we link to everywhere?
Hmmm, I've never seen that one. I'll make sure the forums staff knows about it so we point people using that URL rather than the URL to the standard bug reporting interface.
Isn't the guided form the default? It seems to be when I log in.
Isn't the guided form the default? It seems to be when I log in.
-- Per Jessen, Zürich (13.2°C)
Going via googling for 'opensuse file a bug' -> first result looks suitable - http://en.opensuse.org/openSUSE:Submitting_bug_reports
--> developers and experienced users go here: https://bugzilla.novell.com/index.cgi and not being logged in, then I pick 'enter a bug' (not knowing to search first... though being a hypothetical 'experienced' user I probably should - it takes me to Login page. and then I do see the guided page.
I did end up at the non-guided page previously, but not sure how.
On the first opensuse.org page it does recommend beginners to go to the forums first. So what's happening there when users say they've found a bug?
It seems that at least so far, the filtering process isn't too bad; possibly it could be a little bit clearer and bolder for those with short attention spans, but it doesn't look as broken as I was expecting.
This is part of why we need to gather information before making decisions: what does the whole process really look like, and what are the real issues. No point in "barking up the wrong tree"!
Hey,
On 05/21/2012 07:51 PM, Jim Henderson wrote:
On Mon, 21 May 2012 15:15:40 +0200, Henne Vogelsang wrote:
What do you find so daunting about the guided interface that we link to everywhere?
Hmmm, I've never seen that one. I'll make sure the forums staff knows about it so we point people using that URL rather than the URL to the standard bug reporting interface.
It is the default for every new user since a couple of years AFAIR.
Henne
On 05/22/2012 06:49 AM, Henne Vogelsang wrote:
Hey,
On 05/21/2012 07:51 PM, Jim Henderson wrote:
On Mon, 21 May 2012 15:15:40 +0200, Henne Vogelsang wrote:
What do you find so daunting about the guided interface that we link to everywhere?
Hmmm, I've never seen that one. I'll make sure the forums staff knows about it so we point people using that URL rather than the URL to the standard bug reporting interface.
It is the default for every new user since a couple of years AFAIR.
I think the reason I'm not seeing that as the default is because I tend to just go to http://bugzilla.novell.com - but doing that now I see a version of the guided interface. I remember back in January when I was entering bugs against a different product, though, I hadn't remembered seeing it walk me through the guided interface.
But it's good to know that that's changed now consistently.
Jim
On 2012/05/16 16:18 (GMT) Jim Henderson composed:
On Wed, 16 May 2012 14:31:50 +0200, Guido Berhoerster wrote:
I think it would be appropriate to automaitically send a reminder to those bugs against unsupported releases in NEEDINFO state asking people to triage against a supported version of oS and then to close those not receiving any response in a couple of weeks. That's better than simply WONTFIX'ing in case there are still people that care about them. In fact that is what I already do with bugs against my packages.
That's a very good idea (I assume you mean "asking people to test" rather than "triage"). :)
"Couple of weeks" to respond before resolving is nuts. If it's already more than year old, give it at least 6-8 weeks so that the reminders don't get lost at the start of a holiday, maternity leave, busy final weeks right before a GM cut, or other personal time or hardware constraint.
On Wed, 16 May 2012 12:53:31 -0400, Felix Miata wrote:
"Couple of weeks" to respond before resolving is nuts. If it's already more than year old, give it at least 6-8 weeks so that the reminders don't get lost at the start of a holiday, maternity leave, busy final weeks right before a GM cut, or other personal time or hardware constraint.
Well, again, getting into the specific details at this stage is perhaps premature. What we'll want to do is identify what'll handle, say, 80% of the open issues that are affected, and use that as a guideline. Let's keep it at a higher level for now and we can quibble over whether 2-3 weeks, 6-8 weeks, or another 12 months is appropriate. :)
There's no rule that says if we close a bug it can never be opened again. The idea is to get things moving forward.
Jim
On Wed, 16 May 2012 03:04:39 +0000, Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
Just wanted to update the group here, I've not forgotten about this - just got pulled into a new contract and had to dive in deep on that a couple weeks ago, and had my "community" attention diverted by the forum issues.
I've not stopped thinking about how to approach this, just had other priorities end up taking my time.
We do have a project set up in retro.opensuse.org now (thanks, Henne), but I haven't started filling in the details yet. I'm hoping things will slow down a bit this week and I can start breaking the tasks down based on what we have so far.
Jim
Jim Henderson wrote:
On Wed, 16 May 2012 03:04:39 +0000, Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
Just wanted to update the group here, I've not forgotten about this - just got pulled into a new contract and had to dive in deep on that a couple weeks ago, and had my "community" attention diverted by the forum issues.
I've not stopped thinking about how to approach this, just had other priorities end up taking my time.
Yeah, life always gets in the way of the important stuff :-)
I had hoped this would get going soon, but it's been just over a month now: https://bugzilla.novell.com/show_bug.cgi?id=762517
On Tue, 19 Jun 2012 20:54:35 +0200 Per Jessen per@computer.org wrote:
Jim Henderson wrote:
On Wed, 16 May 2012 03:04:39 +0000, Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
Just wanted to update the group here, I've not forgotten about this
- just got pulled into a new contract and had to dive in deep on
that a couple weeks ago, and had my "community" attention diverted by the forum issues.
I've not stopped thinking about how to approach this, just had other priorities end up taking my time.
Yeah, life always gets in the way of the important stuff :-)
I had hoped this would get going soon, but it's been just over a month now: https://bugzilla.novell.com/show_bug.cgi?id=762517
Hi Can you get here? https://bugzilla.novell.com/editwhines.cgi
Malcolm wrote:
On Tue, 19 Jun 2012 20:54:35 +0200 Per Jessen per@computer.org wrote:
Jim Henderson wrote:
On Wed, 16 May 2012 03:04:39 +0000, Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
Just wanted to update the group here, I've not forgotten about this
- just got pulled into a new contract and had to dive in deep on
that a couple weeks ago, and had my "community" attention diverted by the forum issues.
I've not stopped thinking about how to approach this, just had other priorities end up taking my time.
Yeah, life always gets in the way of the important stuff :-)
I had hoped this would get going soon, but it's been just over a month now: https://bugzilla.novell.com/show_bug.cgi?id=762517
Hi Can you get here? https://bugzilla.novell.com/editwhines.cgi
Hi Malcolm,
uh, nope:
Authorization Required Sorry, you aren't a member of the 'bz_canusewhines' group, and so you are not authorized to schedule whine reports.
On Tue, 19 Jun 2012 21:39:31 +0200 Per Jessen per@computer.org wrote:
Malcolm wrote:
On Tue, 19 Jun 2012 20:54:35 +0200 Per Jessen per@computer.org wrote:
Jim Henderson wrote:
On Wed, 16 May 2012 03:04:39 +0000, Jim Henderson wrote:
OK, so this is something I'd like to start trying to get my arms around so we can have a more comprehensive approach to dealing with bug reports and increase the rate of resolution as well as the follow-up rate when additional information is needed.
Just wanted to update the group here, I've not forgotten about this
- just got pulled into a new contract and had to dive in deep on
that a couple weeks ago, and had my "community" attention diverted by the forum issues.
I've not stopped thinking about how to approach this, just had other priorities end up taking my time.
Yeah, life always gets in the way of the important stuff :-)
I had hoped this would get going soon, but it's been just over a month now: https://bugzilla.novell.com/show_bug.cgi?id=762517
Hi Can you get here? https://bugzilla.novell.com/editwhines.cgi
Hi Malcolm,
uh, nope:
Authorization Required Sorry, you aren't a member of the 'bz_canusewhines' group, and so you are not authorized to schedule whine reports.
Hi Well I guess that's what you need access to "Whining" is when Bugzilla executes a saved query at a regular interval and sends the resulting list of bugs via email.
Malcolm wrote:
On Tue, 19 Jun 2012 21:39:31 +0200 Per Jessen per@computer.org wrote:
Malcolm wrote:
Hi Can you get here? https://bugzilla.novell.com/editwhines.cgi
Hi Malcolm,
uh, nope:
Authorization Required Sorry, you aren't a member of the 'bz_canusewhines' group, and so you are not authorized to schedule whine reports.
Hi Well I guess that's what you need access to "Whining" is when Bugzilla executes a saved query at a regular interval and sends the resulting list of bugs via email.
Yes, that sounds like the sort of thing - would you know who to contact to get the access?
-
Andreas Jaeger -
Bryen M Yunashko -
Carlos E. R. -
Christian Boltz -
Felix Miata -
Guido Berhoerster -
Helen South -
Henne Vogelsang -
jdd -
Jim Henderson -
Malcolm -
Marcus Meissner -
Michal Kubeček -
Per Jessen -
Petr Tesarik -
Philipp Thomas -
Rajko -
Roger Luedecke -
Vincent Untz