Test Strategy

John D. McGregor

Recently a client remarked that the time spent planning their testing was as productive at finding faults as was the actual execution of test cases. Since I was in the midst of helping another client plan their testing strategy this remark got my immediate attention. My second client was trying to determine exactly which of the many permutations of data values to test in an application that has a very complex GUI plus an extensible API. Both of these experiences reminded me that planning any activity is important, but in todays world of Just do it! there is a tendency to act rather than think.

The testing process can be divided into three phases [Hetzel]: planning, acquisition and execution & evaluation. The planning phase provides an opportunity for the tester to determine what to test and how to test it. The acquisition phase is the time during which the required testing software is manufactured, data sets are defined and collected, and detailed test scripts are written. During the execution and evaluation phase the test scripts are executed and the results of that execution are evaluated to determine whether the product passed the test.

In this months column I will focus on the planning phase. I am going to briefly discuss test strategies and then focus on some techniques for determining which features or components within a program to test more intensely than others. I will also discuss a technique for incrementally defining a test suite so that the level of intensity is used to determine the number of test cases to be created.

Test Strategy

The project test plan should describe the overall strategy that the project will follow for testing the final application and the products leading up to the completed application. Strategic decisions that may be influenced by the choice of development paradigms and process models include:

When to test The test plan should show how the stages of the testing process, such as component, integration and acceptance, correspond to stages of the development process. For those of us who have adopted an iterative, incremental development strategy, incremental testing is a natural fit. Testing can begin as soon as some coherent unit is developed and continues on successively larger units until the complete application is tested. This approach provides for earlier detection of faults and feedback into development. For projects that do not schedule periodic deliveries of executable units, the big bang testing strategy, in which the first test is performed on the complete product, is necessary. This strategy often results in costly bottlenecks as small faults prevent the major system functionality from being exercised.

Who will test The test plan should clearly assign responsibilities for the various stages of testing to project personnel. The independent tester brings a fresh perspective to how well the application meets the requirements. Using such a person for the component test requires a long learning curve which may not be practical in a highly iterative environment. The developer brings a knowledge of the details of the program but also a bias concerning his/her own work. I favor involving developers in testing as do many others [Beizer], but this only works if there are clear guidelines about what to test and how. I will discuss one type of guideline below.

What will be tested The test plan should provide clear objectives for each stage in the testing process. The amount of testing at each stage will be determined by various factors. For example, the higher the priority of reuse in the project plan, the higher should be the priority of component testing in the testing strategy. Component testing is a major resource sink, but it can have tremendous impact on quality. I will devote a future column to techniques for minimizing the resources required for component testing and maximizing its benefits. For the remainder of this column I will discuss techniques for system testing such as determining how many cases to build.

Дата добавления: 2015-11-16