Software of these days are much more complex and large then those of 10 years before. Two-third of their related costs is maintenance, out of which the biggest part is testing and bug fixing.

A serious problem during the development process is to manage changes and their consequences. Bug fixing means frequent changes and it often happens, that due to the complexity of the system, during bug fixing new problems occur. The later a problem is fixed the more the cost and time demand required.

Nowadays the most effective agile methodology requires the daily run of tests, which happens mostly at night. However, the testing of a very complex software requires even more time, that's why it is sometimes impossible to complete testing. Much of the test tools are unable to select the necessary (right) tests. Therefore they either run all or there is an ad hoc selection. Neither is effective enough, that's why we discover bugs in most softwares day after day.

At the same time a single daily testing is not enough, because the team modifies the code many times during the day and the next day no one knows whom the bug originally belonged to. At best bugs should be discovered immediately (just in time). In that case its reparation is almost free of charge. In the unit testing phase the average fixing time is about 3.5 hours, while an immediate repairing takes only minutes.

Continuous testing – an agile methodology – is the answer to these challenges. The very first test tool has been evolved at MIT (Massachusetts Institute of Technology). This tool runs all tests after the modification of the code. Unfortunately it could be very time consuming, e.g. tests wouldn't run between two modifications. This method can be effectively used in case of small test sets, or very speedy tests.

The next generation of continuous testing tools doesn't run all test cases. Instead it selects out of the complete test set only those which should be re-run due to program modification. The key to continuous testing is test selection. The currently downloadable tools select a lot more test then needed, therefore in case of larger and more complex systems tests cannot be run between two saves.

In optimal case after all code modifications only a few tests would be run, which means bugs will be discovered just in time – before the next save. We call it optimal if the test-selection is program branch-level.

if
    statement_1
else
    statement_2

The code snippet above when statement_2 has been modified, then only test covering the else branch will be selected and executed, while the tests covered by the then branch will be omitted.
The continuous test tool does not interfere with software developers' work, because in case of bug-free modifications, there is nothing to appear on the screen, everything happens in the background. Should a bug be discovered its place appears immediately, making its fixing easy. The more the developer saves his work, the lesser tests should be selected, because even though there is lesser time for running the tests, they can be run until the next save due to the lesser number of test cases.

We just finished the beta version of 4D JATEST, a continuous test tool, corresponding to the above. The tool is able to select JUnit tests for Eclipse. Jatest selects only those tests, which should be re-run because program modifications while JUnit Runner runs these tests.