Saturday, September 26, 2009

The Unit Test shoe does not fit all feet

In the last year, my company has written software of four very different types.

One type is intended to be used internally, rarely, by gurus. It's fine for it to require human intervention, to break, to require editing of Python source files to change its behavior. This stuff is developed on our own time (and dime) to solve short-term problems and to prototype concepts.

The second type is demoware. It has to work along exactly one path while we take screenshots or record a video. This is developed on a small budget (internal or external) and a tight schedule. But the target audience isn't expecting anything more than a demo reel and a powerpoint presentation.

A third kind is a customer-deliverable application. This has to work pretty reliably; the customer will use it frequently, time is valuable, all of that. This development is fully funded.

And then there's the fourth kind. The fourth kind is life-safety critical software. We develop software for mobile robots, some of which are big and scary. This development is fully funded too.

Anyone who says that we shouldn't use unit testing for the fourth type of software is an idiot.

Anyone who says that we should use unit testing for the first type is an idiot too.

Anyone who says that we should use unit testing for the second type of software is oblivious to the constraints of time and budget --- the constraints that make sure that, in a business, everyone gets paid. This testhead is not quite an idiot, but is limited by perfectionist blinders that will hurt project, team, and company.

The third type of software -- 'pretty reliable' end-user software -- is the only place where the unit-test-or-not discussion is worth having.