One method of test design that I have personally seen work much better than standard manual methods of determining what tests to execute is pairwise and combinatorial test design. In my view, not nearly enough developers or software testers know about it. Here are some good articles about it here: http://www.combinatorialtesting.com/clear-introductions-1 This article, published in IEEE Computer, describes this test design approach and the results of a ten project study that showed this test design selection method found more than twice as many defects per tester hour as traditional, manual test case design methods. http://app.hexawise.com/Combinatorial-Softwar-Testing-Case-Studies-IEEE-Computer-Kuhn-Kacker-Lei-Hunter.pdf - Justin Hunter

1) Track the amount of time it takes to create and execute a set of tests. (A few days' worth, not a few weeks' worth) 2) Measure how many bugs were identified. 3) Do a quick calculation: what was the approximate amount of time spent? What was the approximate value of finding those bugs then vs. in production (or a later stage of testing)? It is fine to use rough estimates here, but make your assumptions explicit. The findings will be useful to a manager making a decision about where to spend money; where to devote limited resources. My advice is to put your faith in hard data, not "HiPPO's" / the "Highest Paid Person's Opinion." Amazon took that approach when it came to their automatic recommendations of products for users; it worked pretty well for them. For more on the HiPPO point, see, e.g., http://hexawise.wordpress.com/2009/08/18/learning-using-controlled-experiments-for-software-solutions/ Also, at the risk of stating the obvious, not all kinds of testing will deliver equal benefits. Good test design (the approach you take to identifying which software tests to execute when), will have a huge impact vs. poor test design approaches. The URL above has information on that point too. - Justin Hunter