Capturing JUnit Test Results

COMP285 Lectures 2022-03-02

Generating Reports

Ant Formatters

Ant has three types of formatter for the <junit> task:

• brief
• plain
• xml

If we set usefile=true then each test case will generate it’s own file.

We can also set printsummary=false if we don’t want this output on the command line.

You can also use multiple formatters at the same time:

<target name="test-xml" depends="test-compile">
	<junit haltonfailure="true" printsummary="false">
		<classpath refid="test.classpath"/>
		<formatter type="brief" usefile="false"/>
		<formatter type="xml"/>
		<test todir="${test.data.dir}" name="org.eclipseguide.persistence.FilePersistenceServicesTest"/>
	</junit>
</target>

Generating HTML Test Reports

From the XML report we can generate HTML reports using XSLT. We can do this using the <junitreport> task:

<junitreport todir="${test.data.dir}">
	<fileset dir="${test.data.dir}">
		<include name="TEST-*.xml"/>
	</fileset>
	<report format="frames" todir="${test.reports.dir}"/>
</junitreport>

This should be run after the <junit> task.

This aggregates all TEST-*.xml files into a HTML report that is human-readable.

Generating All Reports on Failure

By using haltonfailure="yes" the tests will stop and not print reports if there is a test failure. We can use the following methods to generate our reports anyway:

<target name="test" depends="test-compile">
	<junit printsummary="no" haltonfailure="no" errorProperty="test.failed" failureProperty="test.failed">
		<classpath refid="test.classpath"/>
		<formatter type="brief" usefile="false"/>
		<formatter type="xml"/>
		<batchtest todir="${test.data.dir}">
			<fileset dir="${build.test.dir}" includes="**/*Test.class"/>
		</batchtest>
	</junit>
	<junitreport todir="${test.data.dir}">
		<fileset dir="${test.data.dir}">
			<include name="TEST-*.xml"/>
		</fileset>
		<report format="frames" todir="${test.reports.dir}"/>
	</junitreport>
	<fail message="Tests failed. Check log and/or reports." if="test.failed"/>
</target>

This uses the failureProperty and errorProperty to change the default behaviours.

<batchtest>

We can run a set of test files instead of individually declaring them with <test>:

<target name="test-batch" depends="test-compile">
	<junit printsummary= "no" haltonfailure="no">
		<classpath refid="test.classpath"/>
		<formatter type="brief" usefile="false"/>
		<formatter type="xml"/>
		<batchtest todir="${test.data.dir}">
			<fileset dir="${build.test.dir}" includes="**/*Test.class"/>
		</batchtest>
	</junit>
</target>

By naming all our test classes with the ending Test, we can find them easily and put them in a fileset.