Just because the food has gone off it doesn’t mean you have a bum taster. On the contrary, your tastebuds are doing what they should. So what does taste testing have to do with product development?

When tests fail, the test is doing its job. Finding “fails” fast is why we create tests in the first place. This is why we validate.

I recently had a conversation with a client about why validation exposing “fails” should not be seen as application failure per se. It exposed a couple of misconceptions non-developers often have of the development process. It took a while to get my point across. I guess you can say this logic is an acquired taste. One that comes from years of developing products.

Some of those misconceptions are described below.

Code should be perfect.
Not all code can be perfect. In fact, often code is imperfect. Even if it’s flawlessly executed, it may in fact solve a different problem than the one we expected it to solve. Such code cannot be classified as “buggy.” It can, however, be classified as producing unintended results.

When tests fail, the code being tested is “buggy.”
Not necessarily. When tests fail, the only proof is that the test is producing the desired result – not necessarily that the code is buggy. The code could be doing exactly what it was designed to do. But the result, being unexpected, helped us uncover how the application should be coded differently.

Once we realize the flaw in the application design, it’s always worth it to “fix” the problem.
Not so. It wouldn’t make sense to fix things that only affect a small percentage of users. These edge cases can be so seldom that certain levels of risk in failure rates become acceptable.

All technology questions have technology answers.
Again, not so. In order to choose to fix a problem, we must first assess it. A management question arises: how much risk are we willing to accept? How much work will this take? What does that cost? What’s the upside once the work is complete?

“Fixing” the “problem” is the only real solution.
Adaptation is often a suitable alternative to re-writes. Many types of applications are designed to be flexible enough to handle “errors” elegantly and themselves become handy workarounds to “problems.”

What happens very frequently in our business is that products are created and expectations are either met or not met based on what’s created. Perfection for both the entrepreneur and the programmer or goals we expect to attain. However perfection is asymptotic. We can never have the perfect business. By the same token, it’s impossible to build the perfect product. If I buy a bottle of wine, it’s based on a number of factors. Quality is only one of those factors. Price, convenience, prestige, color, varietal, region, all factor into the decision. For the vintner, the challenge is always not to make the perfect wine (though a laudable goal that may be) but to price it correctly given the other factors inherent to it. If its tastes don’t meat the vintner’s expectations but that doesn’t mean the bottle’s bad and can’t be sold.

Bon appetit!