[Bug 677789] New: kiwi: integrate automation testsuite written by Bernhard Wiedemann
https://bugzilla.novell.com/show_bug.cgi?id=677789 https://bugzilla.novell.com/show_bug.cgi?id=677789#c0 Summary: kiwi: integrate automation testsuite written by Bernhard Wiedemann Classification: openSUSE Product: openSUSE.org Version: unspecified Platform: Other OS/Version: Other Status: NEW Severity: Enhancement Priority: P5 - None Component: System Imaging AssignedTo: ms@novell.com ReportedBy: ms@novell.com QAContact: adrian@novell.com Found By: --- Blocker: --- I uploaded a first alpha of my os-autoinst-based kiwi testsuite into http://gitorious.org/~bmwiedemann/kiwi/bmwiedemanns-kiwi this includes http://gitorious.org/~bmwiedemann/kiwi/bmwiedemanns-kiwi/blobs/master/tests/... utoinst.kiwi/README The tests need kvm-0.13+ installed (only for the zypper test part, 0.12.x is good otherwise). One important missing thing is the checklog tool, where I need to do some improvement on the os-autoinst side to allow it to delegate the checks to distri-specific code.
Sounds great. Is the branched version of kiwi with your test framework ready to become included or do we need more time. I think we need more time because some doc hint seems to be required as well.
you can merge it into kiwi, as it does not break anything. Of course, it still needs more work to be done, to be more useful.
Is the os-autoinst framework part of the testing framework or is this something different ?
http://www.os-autoinst.org/ - it's git repo is cloned by the setup-script, and used by the run-script so it is sort of an external dependency for the kiwi tests. --- 08.03.2011 Today I have pushed updates to http://gitorious.org/~bmwiedemann/kiwi/bmwiedemanns-kiwi Together with updated os-autoinst that allows to do a fully automated testrun on a kiwi-generated 11.3 jeos iso. This creates a computer-readable test result summary. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c
Marcus Schaefer
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c1
Marcus Schaefer
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c2
Marcus Schaefer
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c3
Bernhard Wiedemann
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c4
Marcus Schaefer
It currently does not include the kiwi testing part but that can be symlinked as the kiwi-test's setup.sh does.
I plan to hold , write, update the tests itself in the kiwi repo and provide the test description together with the kiwi image description. It would be great if your test-suite can provide a tool which allows me to call something like: os-autotest /path/to/os-image-file /path/to/test-case and then the test runs. Does something like this exist ? if not could you add that ? and last but not least; Do you plan to write some documentation for your test-suite, maybe available in the wiki or another place I can point people to if they want to know more about writing the tests ? Hope I did not bother you too much Thanks -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c5
Bernhard Wiedemann
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c6
Marcus Schaefer
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c
Marcus Schaefer
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c7
Bernhard Wiedemann
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c8
--- Comment #8 from Marcus Schaefer
wenn ich das Ergebnis des testlaufs anschaue bekomme ich in meinem Fall folgendes:
==== bootloader: not-autochecked wait_booted: not-autochecked change_tty2: OK-11.4 login: unknown login1: unknown shutdown: not-autochecked isosize: OK overall: fail
* was bedeutet "not-autochecked" ? * wie kann ich herausfinden warum zB bei login der status 'unknown' ist den login hat er gemacht wie im vncviewer gesehen habe * warum ist overall 'fail' ?
1. not-autochecked is normal. Das heisst nur, dass der Test nix weiter prueft. 2. unknown ist das Ergebnis, was rauskommt, wenn keiner der bekannt guten(OK) oder bekannt schlechten(fail) MD5 Werte oder screenshots(testimgs/) gesehen wurde. Wenn du ein screenshot hast, der ein definitiv richtiges Ergebnis zeigt, dann fuegst du entweder die md5sum des ppm files zu tests/os-autoinst.kiwi/iso/test.d/300_login.pm oder aber eine MD5sum von einem vordefinierten Ausschnitt (die Werte, die tools/inststagedetect2.pl $ppm liefert). 3. Das overall wird in distri/kiwi.oem/check.pm bestimmt und ist nur OK, wenn keine schlechten Ergebnisse vorkommen, wozu die "unknown"s aber zaehlen. Kann man natuerlich auch anders definieren (z.B. bei Factory tests reicht es, wenn zypper, yast und (xterm oder firefox) gingen). fuer das ogg/theora encoden nehm ich auf SLES11 http://openqa.opensuse.org/opensuse/tools/umlffmpeg http://openqa.opensuse.org/opensuse/tools/dummypiper http://openqa.opensuse.org/opensuse/tools/linux-2.6.36 nach /usr/local/bin/ , chmod a+x und http://openqa.opensuse.org/opensuse/img/lenny32.img.gz auspacken die Pfade in umlffmpeg anpassen und geht -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=677789
https://bugzilla.novell.com/show_bug.cgi?id=677789#c9
Marcus Schaefer
participants (1)
-
bugzilla_noreply@novell.com