Introduction to Test Coverage

 

Test coverage is an indirect measure of quality. It measures how well the test suite actually tests the product. Test coverage analysis helps to find out areas in the program not exercised by the test suite so that we can create additional test cases to cover those gaps. It is a structural testing technique that compares the program behavior against the apparent intention of the source code. It helps finding possible pitfalls in the structure and logic of the program.

There are two broad classifications of coverage measures – Path-based and Fault-based:

Path-based coverage: This requires the execution of a particular component of the program. For example, if the scope of the testing is limited to the payments domain alone, we design the test cases in a way that it fully exercises the payments component of the program

Fault-based coverage: This requires that the test suite exercises a program in a way that reveals all the likely faults. It is used when the test strategy is focused on finding all the defects in a feature irrespective of the components involved.

Test coverage methods:

There are 6 fundamental test coverage methods as below:

Statement coverage: This reports whether each executable statement is encountered by the test suite. It is also referred to as line/segment/basic block coverage.

Decision coverage: This reports whether Boolean expressions tested in control structures are evaluated to both true and false. This is also known as branch/path coverage.

Condition coverage: This reports the true or false outcomes of each boolean sub-expressions separated by logical-OR and logical-AND if they occur

Multiple condition coverage: This reports whether every possible combination of Boolean sub-expression occurs

Function coverage: This reports whether the test suite encountered each function/procedure of the program

Call coverage: This reports whether each function call is executed

Static testing – early finding of defects

Static testing is the process of evaluating the sanity of the code, algorithm and documents without any execution on computer. It is the only available method to find defects in the early stages of the Software Development Life Cycle. It is completed before the implementation of code.

Static testing can be done through:
1. Walkthrough
2. Inspection
3. Informal Review
4. Technical Review

Walkthrough involves presenting a technical document in a meeting. This is a useful technique to ensure that everyone understands the product. The author of the work product calls for a meeting and presents the product to reviewers, inspection team, engineering leads, managers and other key stakeholders of the product.

Inspection is a well-known software engineering tool to improve the entire development process. In this phase we gather data on the weaknesses in the process and number of hours in correcting them using statistical process control techniques. It is a pre-requisite to reviews. It may be wasteful to do reviews unless the document has successfully exited inspection.

Reviews focus on getting consensus or buy-in to a particular document. Review is the most effective and universal test method. It could be a peer group discussion activity. In this phase the developers can find errors like unreachable code or references to variable that are not initialized. Testers can make sure that the documents (screen design, FSD) are built to standards. Incremental reviews throughout the product construction are essential and profitable method of managing quality.

Review Procedure:
1. What are we testing?

a. Requirements review
b. Feasibility review
c. Review screen layout against standards
d. Analyze program’s control and data flow
e. Existential check of the features expected as a part of the product
f. Review the screen designs, FSD

2. What are the criteria for success?

a. Check for correct and required output
b. Test environment

3. Who will do the work ? (roles & responsibilities)