Google Data

By James A. Whittaker

And now for the last plague in this series. I hope you enjoyed them (the posts ...not the plagues!)

Imagine playing a video game blindfolded or even with the heads up display turned off. You cannot monitor your character's health, your targeting system is gone. There is no look ahead radar and no advance warning of any kind. In gaming, the inability to access information about the campaign world is debilitating and a good way to get your character killed.

There are many aspects of testing software that fall into this invisible spectrum. Software itself is invisible. We see it only through the UI with much of what is happening doing so under the covers and out of our line of sight. It’s not like building a car in which you can clearly see missing pieces and many engineers can look at a car and get the exact same view of it. There is no arguing whether the car has a bumper installed, it is in plain sight for everyone involved to see. Not so with software which exists as magnetic fluctuations on storage media. It’s not a helpful visual.

Software testing is much like game playing while blindfolded. We can't see bugs; we can't see coverage; we can't see code changes. This information, so valuable to us as testers, is hidden in useless static reports. If someone outfitted us with an actual blindfold, we might not even notice.

This blindness concerning our product and its behavior creates some very real problems for the software tester. Which parts of the software have enjoyed the most unit testing? Which parts have changed from one build to the next? Which parts have existing bugs posted against them? What part of the software does a specific test case cover? Which parts have been tested thoroughly and which parts have received no attention whatsoever?

Our folk remedy for the blindness plague has always been to measure code coverage, API/method coverage or UI coverage. We pick the things we can see the best and measure them, but do they really tell us anything? We’ve been doing it this way for years not because it is insightful, but simply because it is all our blindness will allow us to do. We’re interacting with our application under test a great deal, but we must rely on other, less concrete senses for any feedback about our effort.

Software testers could learn a lot from the world of gaming. Turn on your heads up display and see the information you've been blind to. There's power in information.