https://bugzilla.novell.com/show_bug.cgi?id=430068 Summary: Test harness for the build service for automated testing Product: openSUSE.org Version: unspecified Platform: Other URL: http://idea.opensuse.org/content/ideas/test-harness-for- the-build-service-for-automated-testing OS/Version: Linux Status: NEW Severity: Enhancement Priority: P5 - None Component: BuildService AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: nagappan@gmail.com QAContact: adrian@novell.com Found By: Other Created an attachment (id=241808) --> (https://bugzilla.novell.com/attachment.cgi?id=241808) test harness patch Features added as part of HackWeek * Test automation will be run in same enviroment as build environment. * A shell environment for the user, with which user can invoke any scripting language. Advantages * User can add their testing scripts, written in any scripting language and they can invoke the scripts inside test-harness.sh. * Environment is recreated everytime, when the test-harness is invoked. Thus giving a reliable environment and test results. * Its not restricted to just test one application, you can add how many ever package you want to test, just add all of them in BuildRequires (in future TestRequires) and invoke all the tests !!! Pre-Requisite to use the test environment * Need to have the test initiater code in test-harness.sh, uploaded to the current project where you want to test the application. * Log files should be generated in XML or HTML format / extension only. * Once the test is done, the result XML and / or HTML file(s) has to be moved to /tmp/RESULT directory, which will be uploaded to repository directory. Known issues * Test results are uploaded to repository directory. * Tests are done only in RPM based environment. * If test-harness.sh is not available in the current project, still it goes till the invocation of test-harness.sh file. * All required projects for testing should be either linked or should be built in the same project. * Need to give all the packages required for testing in BuildRequires. * Tested in OpenSuSE-10.2 and that too in chroot environment. Need to check them with other distributions and in Xen environment. Future ideas * Instead of BuildRequires, It will be good, if we have TestRequires field in spec file, with which we can just install packages relate to testing and that too at the time of testing. * Tests should be done for DEB based environment. * Need to find appropriate directory and upload them. * Since we are giving a shell environment through test-harness.sh, we need to restrict the commands !?! * When test results are uploaded, it should not delete the rpm's available in repository. Acknowledgment Thanks to Michael Schroeder, darix and bauersman for all their valuable help and suggestion. NOTE: Before starting this project, we don't have any idea how build system works, no perl / ruby knowledge, which are required knowledge for this project ! Example test-harness.sh script content ##################### Script starts here ################### echo "gnome-session" > ~/.xinitrc export DISPLAY=localhost:1 /usr/bin/Xvfb -reset -terminate -once :1 -screen 0 1024x768x24& pid=`pidof Xvfb` sleep 5 mkdir -p /tmp/test tar zxv -C /tmp/test -f gedit.tar.gz lwd=`pwd` cd /tmp/test/gedit ldtprunner gedit-execution.xml cd $lwd mkdir -p /tmp/RESULT cp -a /tmp/test/gedit/gedit-log.xml /tmp/RESULT/ kill -s TERM `echo $pid | cut -d " " -f 1` # Kill Xvfb session at end ##################### Script ends here ################### General Links related to testing * Linux Desktop Testing Project - http://ldtp.freedesktop.org * Integrate LDTP tests into jhbuild - http://bugzilla.gnome.org/show_bug.cgi?id=349114 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.