UoL CS Notes
Menu

Testing Principles - Types of Testing & Bug Reports

COMP285 Lectures

Orthogonal Testing

This is where each range of data (not every combination of data) is tested. We are trying to test only a single attribute at a time.

Modes of Error

The mode of an error is how many parts of a function have errors in them. Consider that we have the following function that counts the days until a certain date:

countDaysTill(int day, int month, int year)

We could have bugs appear when we check different variables of the input:

  • Bug in year - Single Mode Fault
  • Buy in day and year - Double Mode Fault

Exhaustive testing of some single mode faults is sometimes possible.

Visualising Orthogonal Testing

When there is a large amount of data to test we can visualise this as a 3D graph (for 3 variables). We can then use this to see where the errors occur on the input.

Statistical Testing

This is where we give a random input to the program and see if the results are correct.

This can be a good method when there are so many inputs that you can’t do other methods of testing.

Statistical Testing Example

Consider that a bug effects 0.01% of positions, evenly distributed, and we test 10,000 positions randomly. What is the chance that we miss the bug?

  1. The chance of passing the bug for each test is:

    \[1-(0.0001)=0.9999\]

  2. Therefore the change of not finding the bug is:

    \[0.9999^{10000}=0.36=36\%\]

We can also work this backwards, with logs, to find the defect density.

Testing with Identities

To ensure that we don’t have buggy tests we can test with identities like so:

\[\sqrt x \times \sqrt x = x\]

We can do this to ensure that our tests are as simple as possible.

Coding by Contract

This is the practice of constraining the input of a method:

  • You can do this by throwing an exception if the input is outside the range.
  • You can use this to reduce the amount of tests that you do.

We may want to use validation testing to ensure that exceptions are produced when they are supposed to.

Generating Useful Bug Reports

We should always include the following information so that our bug reports are useful:

  • Date, Product Name, Platform
  • Description
  • Logs
  • Version of Software
  • How to recreate the bug.