Testing using Unit Tests

As those test run automatically it is a very convenient way to check if something that was changed did corrupt the system (at least the test). Then the change done should be observed very carefully, and if necessary undone and repeated in smaller steps. Although this is not a way to catch any error hidden inside the software but to find a lot of them. Automatic testing has the advantage of not requiring user interaction. Those tests can be run after each change made to the system. Bugs that were produced when implementing the change can be spotted immediately.

Much of help for automatic testing is the JUnit [Gamma, JUnit] testing framework that was developed by KENT BECK and ERICH GAMMA in 1997. It is based on white-box, regression testing strategies which are implemented by the developer of the concerned code within their own test classes which are called by the JUnit testing environment.

The importance of testing is stressed in the following quote:

The fewer tests you write, the less productive you are and the less stable you code becomes. The less productive and accurate you are, the more pressure you feel. ... You would see the value of the immediate feedback you get from writing and saving and rerunning your own unit tests. [Gamma, JUnit]

The following excerpts were taken from an article at JavaWorld [Nygard] where two Sun developers expose their enthusiasm for unit testing and show possibilities for extending unit testing to check distributed components such as Enterprise Java Beans.

We can never over test software, but we seldom test it enough. ... Unit testing is a critical, but often misunderstood, part of the software development process. Unit testing involves individually testing each small unit of code to ensure that it works on its own, independent of the other units. In object-oriented languages, a unit often, but not always, equivocates to a class. If developers knew for certain that each piece of the application works as it was designed to do, they would realize that problems with the assembled application must result from the way the components were put together. Unit testing tells developers that an application's pieces are working as designed.

Process of testing:

1. Decide what the component should do.
2. Design the component. This can be done formally or informally, depending on the complexity of the component.
3. Write unit tests to confirm that behavior. At first, the tests will not compile, since the code they test is not yet written. Your focus here is on capturing the intent of the component, not on the implementation.
4. Start coding the component to the design. Refactor as needed.
5. When the tests pass, stop coding.
6. Consider other ways the component can break; write tests to confirm and then fix the code.
7. Each time a defect is reported, write a test to confirm. Then fix the code.
8. Each time you change the code, rerun all tests to make sure you haven't broken anything.

Important attributes of unit tests are:

• they cover each small unit of code (e.g. a class)
• they are organized into groups or suites of tests
• they have to execute 100
• it should be run often (e.g. each 10-15 minutes) covering only that amount of changes done
• they are regressive tests, i.e. the changes since the last test are responsible for breaking the test
• they raise confidence in the system as a whole, because the components are thoroughly tested
• they don't focus on code coverage but on vulnerable parts which are most likely to break
• they allow developers to write tests first and code tested by them afterwards
• it is like welding requirements in code, thereby gaining clarity in purpose, and preserving them (the requirements) as long as the code lives
• they beware of over-design
• they allow changes without second thoughts, as the tests will point on any failure made
• they help when maintaining as the tests more clearly display the intended use of the class than the class itself

Problems of testing distributed components, or those heavily relying on their (complicated) context:

• it would be necessary to build a test stub to test the components directly
• this test stub would almost be as complicated as the real context (e.g. Application Server, Database)
• therefore the testing environment has to run within the context to test the individual objects, (e.g. as a servlet on a application server)

WILLIAM WAKE has written some essays regarding unit testing, and in one of them [Wake, JUnit] statet the following point:

Unit tests can be tedious to write, but they save you time in the future (by catching bugs after changes). Less obviously, but just as important, is that they can save you time now: tests focus your design and implementation on simplicity, they support refactoring, and they validate features as you develop.

MARTIN FOWLER [Fowler, p.91,] tells about testing practice:

By writing the test you are asking yourself what needs to be done to add the function. Writing the test also concentrates on the interface rather than the implementation (always a good thing). It also means you have a clear point at which you are done coding - when the test works. ... Whenever you are tempted to type something into a print statement or a debugger expression, write it as a test instead.

The authors of JUnit say what kind of test should be written [Gamma, JUnit]:

You can always write more tests. However, you will quickly find that only a fraction of the tests you can imagine are actually useful. What you want is to write tests that fail even though you think they should work, or tests that succeed even though you think they should fail. Another way to think of it is in cost/benefit terms. You want to write tests that will pay you back with information.