Bug prevention – the new definition for software testing

“Bug prevention is the testing’s first goal” – B.Beizer

Beizer stated that “the act of designing tests is one of the most effective bug preventers known,” which extended the definition of testing to error prevention as well as error detection activities. This article gives insight into the power of early testing.

Most of the testers find bugs weeks/months after the designers and developers inject the bug into the product. How many testers proactively engage with their development partners to prevent the bug from ever getting into the product in the first place? If we continue to think of testing as an after-the-development process then our role is nothing more than a bug finder.

To advance our discipline of testing, the testers need to partner with the developers earlier in the lifecycle to move quality upstream and prevent issues. One of the most powerful ways of improving the quality of the product and the team effectiveness is to focus the testing efforts on defect prevention rather than defect detection. This doesn’t negate or eliminate the need for testing. Testing for defect detection alone is like simply treating the symptoms of the problem and ignoring the root cause of the real problem. Defect prevention is about addressing the real problem.

Here are some of the ways for the tester to work in a symbiotic relationship with the developer and move quality upstream:

  1. Participate in code inspection
  2. Engage in design reviews
  3. Prototyping with program manager and designers to prevent sub-optimal designs

Guidelines for writing software test plan

The test plan defines the scope, approach, resources and schedule of the intended testing activities. It identifies the features to be tested, the testing tasks and who will do the tasks.

The three main elements of a test plan are:

  1. Test Coverage: The test coverage measures the proportion of the program exercised by the test suite. It is usually expressed in percentage. It helps in finding areas that are not exercised by the test suite. Having complete test coverage ensures 100% test effectiveness and that the test suite reveals all the defects
  2. Test Methodology: The approach, techniques and tools to be used for testing
  3. Test Responsibilities: Includes the features/functions to be tested and the criteria for success/failure for the tests

The Test Plan contains the following:

  1. The Project Overview
  2. Objective and scope of testing
  3. The test methodology
  4. Resources and responsibilities
  5. The testing efforts and schedule
  6. Entry and exit criteria for testing
  7. The features to be and not to be tested
  8. Success/failure criteria
  9. Test environment
  10. Risk factors and contingency plan
  11. References to the supporting documents which includes FSD, SRS, TDD and the like.

The scheduling of the testing tasks and milestones is done considering the below factors:

  1. Complexity of the feature
  2. Competency of the resource
  3. Stability of the requirements

Each of these factors could be rated as high, medium or low.

For large projects, there will be a single test strategy document. There will be many test plans that draws its content from the test strategy. The Test Strategy is a high level document that defines the testing approach to achieve the testing objectives. It is prepared by the project manager and it will not be updated often. The test plan is derived from Software Requirements Specification, Technical Design Document, and Functional Specification Document. It includes test strategy/test approach as well. It is prepared by the test lead. The test plan needs to be updated often to reflect any deviation from the original plan.

Credit:  Jurvetson (flickr)

If a feature is not to be tested the test plan should state why it is not. For ex: It could be: a low risk feature, tested and found to be stable in the past, not pushed in the current release.

There are several inherent software risk factors in a project like complexity or newness. A well rounded test plan notes all the risks and tasks interdependencies in the test plan and communicates to the stakeholders regarding those dependencies and risks.  Dependencies could be like cross-functional impact of a new feature, delay in getting stable builds, late change in original requirements, late delivery of fix and resource availability.

Once the test plan is documented it requires review and approval by the test manager /release management team in order to move the next phase of testing.