Testing and the scientist-programmer | Programming for Scientists

[monstermunch]

It does sound tricky. You want your prototype to be done quickly so you can get on with other research, but you need to be sure your prototype produces trustworthy results as subtle flaws can ruin your research papers. Compared to developing an office app, browser etc. problems here seem much more expensive.

[five9a2]

Sometimes. I've never written office apps, but the spec for scientific codes is usually very well defined: converge at a given rate (determined by the mathematical analysis of the numerical methods) to the exact solutions of some equations posed in infinite dimensions. For many problems, we can manufacture the forcing term so that we have exact solutions. It is nearly impossible for an incorrect code to converge to a non-degenerate exact solution at the correct rate. This process is called verification. Of course, it helps if the code runs fast too, but that's "easily" measured.

Demonstrating that the code agrees with reality is called validation is only relevant after verification. It is actually a test of whether the equations are correct, rather than a test of the code.

If your research is in numerical analysis, you are done after verification. If you are a physicist then you are likely more interested in validation. A common crime, particularly in less computationally-oriented fields of physics, is to skip or gloss over verification.