Motivations for Unit Testing

Other than the fact that unit testing is about as much fun as you can have with your clothes on, why should you or your team or you organization unit test?  I can sum up the motivation in a single phrase: Demonstrate in a useful way that the code units work in a cost effective way.  On the surface this looks pretty good, but the folks with the money and power (obviously not you and I) have a habit of drilling down into such phrases to understand them better.  Let's try to get out in front of these questions by breaking the phrase down.

Demonstrate.  What do I mean by "demonstrate"?  What I am looking for here is something that shows that the "units" of the software do what they are supposed to do (more on that later).  I want to demonstrate this initially as the team is constructing each "unit" of the software to show that each unit is finished and subsequently as the other units are added to the mix to show that the team has not broken any previous units as it added the new units.  When all of the units are in place and all of the unit tests work, it is time to consider releasing the software to the next, broader form of testing (e.g., integration, performance, usability, beta, and production).  The audience of the demonstration includes the developer of the software unit, the other members of the software development team, and assorted management.  The inclusion of management should mean that the unit tests have to be comprehensive in some sense.

Useful Way.  There is a classic Monty Python skit called the "Silly Job Interview" in which an applicant is subjected to a sequence of silly tests.  The applicant has no way of knowing what the purpose of the tests is or what the proper response should be.  This makes for great comedy but makes a poor basis for testing.  Obviously, I could write a bunch of unit tests that simply returned a "RIGHT" or "WRONG" answer without specifying what was right or wrong, but I would never do that and I am sure that you would not either.  A useful test will say something like, the lower left rear widget should have returned "blah" when fed a green and orange parameter but returned "huh".  Ideally, when this test fails I would only have to look source code for this test and the source code for the lower left rear widget to discover and fix the problem.  The least useful results would be to have all of the unit tests fail because of a single problem in one class.

Code Unit.  The smaller the unit of code that is under test the better.  The finer the granularity of the test, the more useful that particular test will be.  A unit test could exercise an entire subsystem but such a unit test is not particularly useful.  A failure points at the entire subsystem, leaving a lot of effort to search out and fix the problem.  A unit test of a single method of a single class is more useful.  A failure points at the method and with a little luck the method does not go on for page after page.  A unit test of a single path through a single "short" method of a single class is very useful.  The ideal unit test isolates the problem to a single line of the source code.

Works.  There is a saying that something like this: "If you do not know where you are going, any road will do."  Every unit test encompasses five elements:

• The code unit under test (see above).
• The context of the test.  This is the combination of the input values and the state of the code unit prior to the test invocation.
• The actual results of the test.  This is the combination of the output values and the changed state of the code unit just after the test invocation.
• The expected results of the test.  This is the same type of things as the results with the additional value that the expected results have been "blessed" in some way.
• A means of comparison of the expected to the actual that says OK or NOT OK.

Cost Effective.  Resources such as time and money are finite.  The folks with the money and power (remember them?) want to get something back for the investment of these resources.  Their thinking goes something like this:

• A defect has some costs associated with it.  This might be the cost of lost business, the cost of compensating the customers for their difficulties, and the cost of lawsuits. 
• A defect has a probability of occurrence.  It might be hard to compute this probability but generally a rough estimate can be concocted.
• From these two values, it is possible to compute the expected cost.  If a defect has a potential cost of $1,000,000 and a probability of 1 in a million, we can compute the expected cost to be $1.00.  [It is way more complicated than this but what better way to demonstrate that resources are finite than to ignore the complexities.]
• Each unit test has an expected cost to create and to run repeatedly. 
• Each unit test has a probability that it will reliably detect a new defect that is different from all of the defects detected by other unit tests.  At a very fine granularity, this is very close to one, but it is worth thinking about the fact that the unit test might not add any particular value.  [And again, I am glossing over substantial complexities in the interest of time.]
• If the cost to produce and run a unit test to detect a given defect is greater than the expected cost of the defect, it is hard to justify the time and effort to build the unit test.

The chances are that there is a fixed amount of resources that can be used for coding and unit testing.  It is going to be harder to eliminate code than it is to eliminate unit testing.  Understanding this would suggest that we need to figure out is how to create unit tests with the minimum of effort and the maximum of effectiveness.  I do not know about you, but I find it much easier to to be motivated to do easy things than to do hard things. 

tags: unit+testing