smoke tests

It's funny that I first learned what smoke tests are just a few minutes ago. autrijus is mentioning them in his pugs reports all the time, but I just thought it's some slang for tests ;-/

Anyway, the scenario is: daily builds are good, so is frequent testing.

However, sometime testing takes time. Testbases grow big (which is good) and sometimes take a long time to run - up to days for large projects that use tests to exercise boundary cases. This is a problem - how do you build daily with such long tests ?

Solution: smoke tests - a small and quick set of tests that take a hour or two to run. These assure that the system "doesn't smoke" after a build - i.e. doesn't crash the machine, throw a segfault, declare that 2+2=5. Just a few trivial tests can go a long way cleaning up major and basic bugs.

So, you build daily + run a smoke-test suite. If it fails - you found a bug and you know it's new - was entered today -> easy to fix. If it passes, fine. Comprehensive tests run all the time and are ready the test the program more thoroughly.

Note: smoke tests are not too applicable to small programs that don't have huge testbases.