8-year-old exploratory testing
May 2nd, 2007 | Published in Google Testing
Posted by Allen Hutchison, Engineering Manager
Earlier this week a colleague sent me a link to this post over on doodlebyte. In the post, Michael Schidlowsky talks about a recent experience watching an 8-year-old, play with Logo for the first time. The child was able to crash Logo in just a few commands. After that, Michael wanted to try to reproduce the bug:
The other point is that we who use computers every day tend to adopt certain assumptions in our interaction with them. Michael didn't try "rt999..." because he knew that he should always put a space between a command and a parameter. When testing an application, we have to remember that our customers don't always hold the same assumptions that we do, whether they are 8 or 80 years old. Approach your testing with an awareness of your assumptions, and try to find out what happens when you break them. Often the results will surprise you.
Earlier this week a colleague sent me a link to this post over on doodlebyte. In the post, Michael Schidlowsky talks about a recent experience watching an 8-year-old, play with Logo for the first time. The child was able to crash Logo in just a few commands. After that, Michael wanted to try to reproduce the bug:
I had played with UCBLogo for two weeks and hadn’t made it crash once. Brian brought the whole thing down in three commands. The most telling part is that when I tried to reproduce the defect a week later I couldn’t. I issued rt with a ton of 9s and just couldn’t get it to break. As it turns, it only crashes when you omit the space, which of course I didn’t think of doing. It took me more time to reproduce the defect than it took Brian to discover it.This story vividly demonstrates two points about exploratory testing. First is that you have to be very careful to record everything you do during a testing session. In this case, Michael overlooked the fact that the 8-year-old had issued an "rt999..." instead of an "rt 999...". That made a huge difference to his testing and resulted in his taking a lot of time to reproduce the error. Some testers use screen recorders or key loggers to help with this issue.
The other point is that we who use computers every day tend to adopt certain assumptions in our interaction with them. Michael didn't try "rt999..." because he knew that he should always put a space between a command and a parameter. When testing an application, we have to remember that our customers don't always hold the same assumptions that we do, whether they are 8 or 80 years old. Approach your testing with an awareness of your assumptions, and try to find out what happens when you break them. Often the results will surprise you.