Test coverage is an indirect measure of quality. It measures how well the test suite actually tests the product. Test coverage analysis helps to find out areas in the program not exercised by the test suite so that we can create additional test cases to cover those gaps. It is a structural testing technique that compares the program behavior against the apparent intention of the source code. It helps finding possible pitfalls in the structure and logic of the program.
There are two broad classifications of coverage measures – Path-based and Fault-based:
Path-based coverage: This requires the execution of a particular component of the program. For example, if the scope of the testing is limited to the payments domain alone, we design the test cases in a way that it fully exercises the payments component of the program
Fault-based coverage: This requires that the test suite exercises a program in a way that reveals all the likely faults. It is used when the test strategy is focused on finding all the defects in a feature irrespective of the components involved.
There are 6 fundamental test coverage methods as below:
Statement coverage: This reports whether each executable statement is encountered by the test suite. It is also referred to as line/segment/basic block coverage.
Decision coverage: This reports whether Boolean expressions tested in control structures are evaluated to both true and false. This is also known as branch/path coverage.
Condition coverage: This reports the true or false outcomes of each boolean sub-expressions separated by logical-OR and logical-AND if they occur
Multiple condition coverage: This reports whether every possible combination of Boolean sub-expression occurs
Function coverage: This reports whether the test suite encountered each function/procedure of the program
Call coverage: This reports whether each function call is executed
The test plan defines the scope, approach, resources and schedule of the intended testing activities. It identifies the features to be tested, the testing tasks and who will do the tasks.
The three main elements of a test plan are:
The Test Plan contains the following:
The scheduling of the testing tasks and milestones is done considering the below factors:
Each of these factors could be rated as high, medium or low.
For large projects, there will be a single test strategy document. There will be many test plans that draws its content from the test strategy. The Test Strategy is a high level document that defines the testing approach to achieve the testing objectives. It is prepared by the project manager and it will not be updated often. The test plan is derived from Software Requirements Specification, Technical Design Document, and Functional Specification Document. It includes test strategy/test approach as well. It is prepared by the test lead. The test plan needs to be updated often to reflect any deviation from the original plan.
Credit: Jurvetson (flickr)
If a feature is not to be tested the test plan should state why it is not. For ex: It could be: a low risk feature, tested and found to be stable in the past, not pushed in the current release.
There are several inherent software risk factors in a project like complexity or newness. A well rounded test plan notes all the risks and tasks interdependencies in the test plan and communicates to the stakeholders regarding those dependencies and risks. Dependencies could be like cross-functional impact of a new feature, delay in getting stable builds, late change in original requirements, late delivery of fix and resource availability.
Once the test plan is documented it requires review and approval by the test manager /release management team in order to move the next phase of testing.