Software testing approaches

“When developing our test strategy, we must minimize the impact caused by changes in the applications we are testing, and changes in the tools we use to test them.”  –Carl J. Nagle

This article evaluates the effectiveness of testing approaches and sheds light on the areas we need to concentrate on while testing. In essence 20% of the test cases should account for 80% of the ROI. For this, the testers need to question the logic behind the test case development by asking: 1) What is the probability of the occurence of bugs in this area? 2) Do I really need to automate this test? 3) What is the impact of GUI changes on the test case? The testers who are short-sighted tend to focus only on the testing milestones like pass rate/deadlines and fail to achieve the long term goals of testing like maintainability, robustness and coverage.

Credit: Flickr by e-strategyblog.com

 

GUI or End-to-End testing:

In this testing approach we try to simulate a customer completing some task involving multiple features and system interactions. This is the least effective test automation approach

Advantages:
1. Exposes Behavioral issues

Disadvantages:
1. Maintenance nightmare: Whenever the GUI changes the tester needs to revise the test case. GUI test cases are tied to the very trivial details of the UI that almost any change in the GUI can cause the test cases to fail. Thus, maintenance becomes tedious. Considering the time to fix it we could rather re-create the test case from scratch
2. Last minute design changes can cause havoc to the automation team
3. Longer test case execution time
4. Does not find much bugs

While developing the test strategy we must minimize the impact of UI changes on the test cases. I would recommend executing GUI test cases later in the lifecycle when the UI is stable and after all the critical fuctional bugs are exposed by the approach “Component based” testing explained below.

 

API or Component based testing:

In this testing approach we directly hit/communicate with the component to test the application. A component is an integrated aggregate of one or more units. We parameterize the properties and pass it as the input data to the API.  The corresponding methods are triggered on the data and it tests the underlying business logic/functionality of the application.

Advantages:
1. Maintainable: As opposed to End-to-End testing this is not dependent on UI changes and hence reduces the automation efforts.
2. Faster test case execution time and hence reduces the Mean Time To Test (MTTT)
3. Finds many bugs early on in the lifecycle. Exposes functional issues

Bug prevention – the new definition for software testing

“Bug prevention is the testing’s first goal” – B.Beizer

Beizer stated that “the act of designing tests is one of the most effective bug preventers known,” which extended the definition of testing to error prevention as well as error detection activities. This article gives insight into the power of early testing.

Most of the testers find bugs weeks/months after the designers and developers inject the bug into the product. How many testers proactively engage with their development partners to prevent the bug from ever getting into the product in the first place? If we continue to think of testing as an after-the-development process then our role is nothing more than a bug finder.

To advance our discipline of testing, the testers need to partner with the developers earlier in the lifecycle to move quality upstream and prevent issues. One of the most powerful ways of improving the quality of the product and the team effectiveness is to focus the testing efforts on defect prevention rather than defect detection. This doesn’t negate or eliminate the need for testing. Testing for defect detection alone is like simply treating the symptoms of the problem and ignoring the root cause of the real problem. Defect prevention is about addressing the real problem.

Here are some of the ways for the tester to work in a symbiotic relationship with the developer and move quality upstream:

  1. Participate in code inspection
  2. Engage in design reviews
  3. Prototyping with program manager and designers to prevent sub-optimal designs