On 10/14/2013 08:47 PM, Richard Brown wrote:
On 14 Oct 2013 18:11, "Ancor Gonzalez Sosa"
wrote: The openSUSE Team is reporting bugs in bugzilla after diagnosing the problem behind every "red" test in that instance of openQA. Which is probably way more useful for most contributors than having access to the raw openQA results. I know I've mentioned it already, but this comment made me feel like I should clarify.
I will also clarify my words, since I obviously explained wrongly, according to some reactions.
One of the best things openQA brings to the table, for me, is a 'reference' to compare my own testing to. This means I don't see openQA as just an automated, closed loop, "test->analyse->flag" system to provide flagged test failures ("red" or otherwise) for vetting and bug filing.
I'll regularly use the raw openQA results to compare against my own results - not only does this save me having to ask in #opensuse-gnome IRC "does anyone else get xyz with abc?" but seeing those raw results gives me a solid picture of what is largely working in openSUSE - meaning I can focus my time testing on the bits which openQA doesn't cover or has flagged as potentially needing attention - it's quite often the 'yellows' that openQA finds that really need the work, and right now, we can't see what openQA v2 is finding.
I want to avoid adding to the downward spiral I fear this thread is taking, but I feel obligated to object to the suggestion that 'most contributors' don't need to see the raw openQA results.
I never said that "most contributors don't need to see the raw openQA results". Please, re-read. I said that, CURRENTLY, we are NOT ABLE to provide access to raw results, but we are, at least, providing bug reports based on it. I stated that, in my opinion, bug reports are more useful to most contributors than direct access to raw result, just as a side comment. So, once again, we are not providing direct access to the raw results because there are unresolved security issues on it. And that is the only reason. Is NOT a conscious decision in order to keep people away from the results. Is NOT because we have decided that bug reports are more useful (it was just an unfortunate side note). Is NOT because we think that contributors don't need access to raw results or they are not able to understand them. And is NOT because we want to filter the results in any way.
I'd like to think of myself as a rather typical contributor who likes testing, and I'd like..no..I need to see as much information as possible about the automated test success/failures of openSUSE, so I know where I can best invest my time as a contributor, with the time consuming contributions of testing and bug resolution. Without that information, I risk duplicating work which openQA, the openSUSE team, or others, may have already completed, and that's not good for myself or the distribution.
As Guido pointed out, this is most important at during the 'fluid' period before a release - You don't want contributors testing things which are already known, or trying to resolve issues that are already fixed. That's the #1 selling point of openQA - it's a testing platform hooked into OBS and so is in a position to provide that information to the community so we have a collective baseline to work from. Every day we don't have raw results from openQA is a day it's a little harder for community testers to efficiently test.
I understand the explanations given regarding openQA v2, but that doesn't change the importance of as much information as possible so the community know what is broken & what to test.
I see the last test result on openQA v1 was on 10/10/2013 - has there been no checkins to 13.1 since then or is something broken?
I don't know. The people behind openqa.opensuse.org (who are not and have never been the openSUSE Team itself) is the right people to ask. But the lack of results in openqa.o.o has nothing to do with the existance of another instance running V2 (in a completely different hardware and managed by absolutely different people). Cheers. -- Ancor González Sosa openSUSE Team at Suse Linux GmbH -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org