Transforming the software testing organization

FutureTest 2008 was unlike any conference I’ve seen for top software test managers — really dug into ways that we can transform software testing. From my perspective, the conference truly exceeded expectations, not only in the number of participants, but also in the quality of the dialog.

I’d like to share just a few of the many insights that I picked up at FutureTest 2008. In many cases, these were things that we already knew, but the keynotes served as a whack on the side of the head. (I was too busy as “master of ceremonies” to take good notes.)

• Test consultant Rex Black presented a comprehensive model for showing the return on investment of different types of testing and for testing at different stages of the application life cycle. Now, analyses of this sort are old hat. They’re not rocket science, and they don’t use any math more complicated than division. But very few test managers take the time to prepare these analyses, based on their own company’s metrics. Rex showed how, in a typical case, the cost of investment is returned nearly 700 percent in savings to the company over the lifespan of a product, factoring in lost business, support costs and remediation costs. Fascinating.

• In an intense panel discussion, voke analyst Theresa Lanowitz said that the most important transformation that test managers can do is change from acting as gatekeepers (or “quality policemen”) to customer advocates. That change might be as simple as altering the name of the test department or rephrasing how issues are reported. It might be as complicated as messing with organizational structure. The perception of test/QA’s role is to be critical in its broad success.

• On the topic of just-in-time testing, McGill lecturer Rob Sabourin (pictured) reiterated that the question isn’t always what to test, but also what not to test. In a hectic, turbulent environment, you can’t test everything. Instead, figure out what you need to test based on the question, “If I find a problem, do I need to fix it?” After all, you don’t need to fix every bug in order to have a successful product. Look at how products will be used and test accordingly.

• Top-shelf blogger Joel Spolsky reminded us that software testing means more than ensuring that the software meets the stakeholders’ formal requirements. Our job is to create quality products, successful products, products that customers want to use. Quality means more than many features and a low bug count. It also extends to the aesthetics and emotional feel of a product. If a product makes the user feel that he or she is safe and in control, it will be more successful than a product that just takes the user along for a ride.

It was an amazing conference. We’ll announce the dates and location for FutureTest 2009 shortly.