On Wednesday 10 February 2016 18:29:00 Christopher Hofmann wrote:

since we as the Yast team decided to increase our test coverage, we were creating and submitting several tests to openQA in the past and we would like to continue doing that. […]

So I suggest to set up a meeting to discuss and write down a first version of an openQA contribution manifest.

I think the right people wo that should be from QA (rbrown?), openSUSE (coolo?), SLEPOS (nadvornik?) and YaST (myself) - anyone else interested?

I am interested. Let me introduce myself to the ones not yet knowing me. I am working with SUSE since 2015-11-01 as a QA Engineer and my main task at the moment is openQA backend development. So I am sure I could contribute to the proposed meeting but depending on the number of participants it might make more sense if others participate, e.g. rbrown.

[…] In some cases it felt quite complicated to get those tests merged and running in the official instances.

To make contribution of tests easier and more straightforward I think it would be helpful to define some rules and methods about - how to handle differences between openSUSE and SLE in regards of testing. - how to keep the balance between test coverage and testing speed means: which tests should go into which test suites or maybe even a better definition of what all the test suites are actually supposed to be good for. - define a coding style in regards of how things should be done, e.g defining/checking for ENVs vs. check_screen, when to soft fail vs. die() etc.

I can understand the issues you mention and I think a meeting can help. However I am also convinced that everything that you described can evolve in a continuous process. Multiple tickets in our issue tracker on https://progress.opensuse.org/projects/openqav3/issues are already mentioning ideas and challenges to improve it and you can also add your suggestions there or contribute to existing questions and tasks. For example there are: - "Staging openQA instances": https://progress.opensuse.org/issues/8542 - "Review support for needle changes": https://progress.opensuse.org/issues/10144 - "Write deploy test script": https://progress.opensuse.org/issues/8104 - "Check supported version of os-autoinst": https://progress.opensuse.org/issues/10474 - "Improve documentation on assert_screen ...": https://progress.opensuse.org/issues/10380 Besides, there are still some tickets open where you and others from "the Yast team" contributed a lot more than I could :-)

Clearly defined rules make it easier to avoid discussions in PRs since people involved might currently have different opinions how a proper test should look like.

Speaking for myself if I see a good "general guideline" or "common question" popping up in a PR discussion I note this down in a ticket or the wiki or just improve directly what the issue was about so it is actually worth the effort to discuss. If you feel that the discussion is getting nowhere at least when talking to me feel free to be blunt and tell me to stop moaning :-) Looking forward to feedback from others on the meeting idea and the way to go forward and I am all open to support this where I can. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org