You are here: Home / Archives for Aruna

How can testers influence design choices?

March 1, 2011 By 1 Comment
As a Quality Engineer it’s a great experience being able to plan and facilitate design review sessions with developers and architects. This could either be a formal/informal session. The design review provide hints to the testers that help to foresee potential defects in the system. It comes with huge benefits of learning the crucial design elements especially when the discussion happens in a room of developers. At the same time, the developers also get a chance to brainstorm the design when the testers ask questions which inturn adds great business value to the product.
This article explores how fellow test engineers can contribute and influence the design decisions in a development effort. While the developers are better at Object Oriented Programming concepts, the testers can explore the feature at a deeper level identifying corner cases, exceptional conditions or error cases and recommend designs that are flexible enough to accomodate these behaviors.
Issues costing $300K occur due to bad product design that could have been prevented if there were design reviews with testers.
Below are two techniques that are explained indepth to help testers catch design flaws in a product:
1. Apply 5 basic design patterns
2. Analysis models

1. Apply 5 basic design patterns:

Design patterns are named set of problem-solution pairs for the repeated challenges faced by the programmers in the IT industry. Understanding Design Patterns requires us to “think in objects”. Being aware of the design choices and consciously making the best design choice is essentially a programmer’s skill and requires strong knowledge of Object Oriented Programming. Then, how does a tester add value to the design review of a product?
Testers can perceive where the potential issues and risks might be for a particular product design. Usually the unthinkable risks in the product design are not talked about in design reviews. This is the area where a tester can significantly add value. There are 5 basic business design patterns that can get the testers thinking about the unthinkable risks. Some of you might be already thinking on these terms while evaluating the design of a product. Making it more formal with process flow diagrams can make the design review session more useful to the stakeholders.

 

For understanding each of the below business design patterns let us consider a storyboard of a customer purchasing a product on a shopping website.
Before getting into these patterns first let us define what is an event.
An event defines under what circumstances a particular process/function gets triggered.
We can use logical connectors AND / OR called joins for joining two or more business events.

1) Dissection of events

This pattern helps the tester think: “What are the different starting events that can trigger a function/process?”
For example a Purchase Order Processing function can have different starting events as below:
Customer clicks on add to cart button on a third party shopping website
Customer places order via an email sent to his inbox
Customer places order via mobile phone

 

2) Sequential events pattern

What is the specific order in which the events should occur?
Should customers login before clicking on checkout? Can guest users also checkout?

3) Parallel Split pattern

What are the multiple processes that should be triggered by a starting event?
Lets assume that the customer clicks on the checkout button. Normally the purchase order process is triggered. What if the database crashes during the transaction? To handle this, the application should have two processes running. Periodic data migration of the transaction to backup database/servers so that the backup server can get a handle over the transaction in case of any crash and another is the payment processing function.

 

If the product is not designed to handle this crash then there would be a heavy data loss in the midst of the transaction since millions of customers might be using the site. These kinds of unthinkable risk can be better pointed out by a tester using this pattern.

 

4) Synchronization

When there are a set of activities, which of these must be finished completely before we continue the process?
Fraud check and confirmation of funding source/shipping address should be done before payment is processed.

5) Exclusive choice

What are the different mutually exclusive flows possible for a given process?
As stated in the earlier pattern, to which flow should the program transfer the control in the event of server crash?

2. Analysis Models

Analysis models could be used to probe, critique, review and approve the product design. Asking questions, even the stupid ones, during the design phase of the project can add great business value to the product.
Data models are a visual representation of the relationships between the different entities of a system. During the design phase of a project apart from the requirements specification documents there are class diagrams, use cases and entity relationship diagrams that are usually thought of as programmer’s tools. Similarly, the programmers think of it as a business analyst’s tool. Infact it can be used by both testers and developers to ask good questions that adds new perspectives to the design review discussion leading to the discovery of new facts or pitfalls in the product design.
1. Study the domain\business rules between the different entites
2. Know the right questions to ask
3. Identify pitfalls in the product design
4. Check for missing details in the requirements spec and discover new business rules
5. Filter out requirements that are likely to change

Seperation of concerns

Is the design flexible enough to accomodate the likely future change in the product? I just came across a GTAC (2010) video on Flexible Design? Testable Design? You Don’t Have To Choose! where the developer dumps multiple responsibilities in a single large class because he doesn’t like coding several small distinct behaviors each in a seperate class. This will cause issues in future when we need to change/extend/add one more behavior to the class. Whenever we modify this single large class the developer needs to modify so many methods calling them and so he will resist changes. It requires huge efforts even for a trivial change. This is an example of a bad design.
A tester need not agree with this kind of design pattern. The testers should filter out requirements that are likely to change in future using analysis models and ensure that the programmer implements “Seperation of Concerns” principle to make the design flexible for future changes for these set of requirements.
Filed Under: Software Testing, Testing practices Tagged With: , , , , , , , , , , ,

Making a successful transition to agile testing

February 18, 2011 By Leave a Comment

“Agile teams produce a continuous stream of value, at a sustainable pace, while adapting to the changing needs of the business.” – Elisabeth Hendrickson

In this definition, “business value” refers to shippable code.  Agile teams release business value in the form of shippable code frequently at a very fast pace atleast monthly once.
“Sustainable pace” means that the agile teams continuously add capabilities to the emerging system at more or less the same velocity given no increases in team size.

Thus, Agile ensures that the code is built in a sustainable fashion to a high quality. This article explains how agile manifesto might apply to testing and the roles and responsibilities of a tester in an agile project.

“If you don’t know how to work together, whatever your defined processes, I can all but guarantee that your processes are not being followed” – James Bach

Team building should take the place of processes. The whole development team should commit to delivering a high-quality product. Testing is a key activity that should involve testers,developers and customer representatives. It is not supposed to be done by the testers in isolation. The developers need to write automated unit tests before starting to code. This approach of doing testing before coding is called Test Driven Development. In agile environment the distinction between a tester and a developer is blurred. Testers are not the solely responsible or even the primary owner of quality. Quality is a shared responsibility of the whole team. Individuals in an agile team may specialise in a particular role but will take on different roles depending on the context. Testers who are out of their depth reading code or uncomfortable influencing design decisions may require some training to fit into the agile team. The estimate for test efforts should be done as a part of the overall implementation efforts of the project.

Rigid processes sometimes tend to hinder the team’s progress towards quality goals and impose friction on people interactions. Testers need to be on their gaurd to watch out for any impact of rigid processes on quality. For a long time the focus of  testing has been short-sighted with emphasis on functional correctness rather than value to people. This problem in testing is a subset of the bigger problems of rigid processes in place. For example, in company XYZ if the tester logs a defect that lacks the support of requirement specification document, the issue is considered to be a non-issue as per the process. The tester needs to support the defect with the automated test case that failed and the spec document it violates. Otherwise the DEV doesn’t fix the issue and the test managers do not treat the defect as a priority item. In these situations there needs to be a careful risk assessment of the issue through shared conversations with the developer and high support from the management if indeed the issue falls into high risk area.

 

 

Myth #1:

Manager:  Hey Team! We are agile now, therefore the requirements are not documented

Team: No problem, we can deal with it

Having no documentation is called agile…but wait…how do we come up with an elegant, high quality code without requirements documentation. It is a myth that agile eliminates all documentation. While agile testing does not support detailed specifications, the requirements should be clearly documented at a high level though not detailed.

Conversations and shared understanding will take the place of heavy-weight documentation. For testers this could mean favoring manual tests over automated tests. This gives the testers enough freedom to discover, diagnose and exploit the product while testing. This requires exploratory testing skills. Testers should be capable of looking at the big picture from multiple angles and ask good questions that the developers and customers might think of. The rigid test scripts and procedures in automation does not reveal critical issues at times. Exploratory testing can help to spot missing features or opportunities for product improvement. The testers could collaborate with the DEV and read the automated unit tests. Instead of spending energy on the specifications and requirements document, the testers can get into the code itself.

The agile testing mind-set is all about collaborating with customers, looking at the big picture, providing and obtaining feedback. Customer collaboration is at the heart of agility. Here the testers collaborate closely with the customers. Working closely with the customers is favored over becoming a customer proxy. The testers do not merely execute against a negotiated bill of work but have iterative collaboration with the customer. The testers state the user’s point of view in team meetings and bug reports. The testers even fail the release if the quality was unacceptable to protect\defend the customer.

Myth #2:

Manager: Hey Team, the management has just decided to add twenty two new features to the product!

Team: Great! That’s good for the customer!

As an agile team we never  resist changes and can accept any amount of changes anytime…What? If twenty two new features are added to the product what happens to the stability of the product?

While agile testing adapts to changing business needs, it doesn’t encourage adding many new features that can make the code unstable and break quality. Agile focuses on delivering a small set of core features with high quality one at a time and adds the other capabilites in the subsequent releases at a fast pace. The changes should be prioritised and if required deferred to the next release.

Change is at the heart of agile projects. The testing team might be expected to follow a plan when a change has happened in the product. The plan should support the change. After a change has been implemented the testers need to test the untouched areas of the changes to ensure the ongoing stability of the product.

Filed Under: Project Management Tagged With: , , , ,

Key Takeaways from WTA06 – It’s In The Cards

January 30, 2011 By 4 Comments

The American chapter of Weekend Testing is an online event focused to refine testing skills and practice the art of testing in collaboration with other passionate testers. I found this “WTA06 – It’s In The Cards” event to be a wonderful platform that enables testers to test products from different angles, share the diverse thoughts of fellow testers, enjoy the freedom to fail safely and learn; and most important – share personal observations and receive immediate feedback from other testers on the approaches. I continously questioned my testing approaches till the end and learnt a lot from this session. Thanks Michael Larsen for organising this session. In this article I would like to share my personal experience and lessons learned from this exercise.

Testing Challenge:

Here is a brief description of the testing challenge given in the session. For the full event details and session report please refer the website weekendtesting.com at http://weekendtesting.com/archives/1736

This week’s testing challenge was brought to us by Eusebiu Blindu. Thanks Eusebiu! The game was very interesting and really challenging.

This game uses playing hand of cards, and based on the hand selected an image is generated when the player clicks on the “Generate Image” button.

The primary task is to test the application, and the mission is as follows:

1) Detect if any relation exists between the selected cards and the image generated in popup (after pressing “Generate Image”)

2) Are you able to determine if the application needed some skills that you needed to improve and, if so, which is the main one?

Challenge URL:
http://test1.testalways.com/cgi-bin/cards/html/playing_cards.html

The first one hour was spent in answering the first question. We clicked on the cards and tried to detect if the image generated had any relationship with the card selected. There were 5 suits of cards. Each suit contained cards from 2,3 to King, Ace Joker of (heart, diamond, club, spade). I tried several combinations of test input data and reported the findings. The images generated where bird, cat and fish with black, blue and red background. I was trying to detect which combination of input data creates a specific output. I did detect a pattern but the behavior was random.

The variabe that was the central focus of test was the relationship between the card selected and the image generated. We made a note of the factors that influenced this variable:
Key factors influencing the variable under test:
1. Sequence of image invocation
2. Arguments passed to the url in source code
3. Card selected
4. Order of navigation: Backward, Forward

At the end of one hour all the participants agreed that there did exist a relationship between the card selected and the image generated. We ended up with some key takeaways on the testing approaches used.

Myth:
“We need to test all the combinations of input data to do complete testing”

This is a myth. Its not required to test all combinations of input data. We can seperate out the testing that is unnecessary to perform before wasting our time on all combinations. The test design should cover only the necessary combinations of input test data.

In the last one hour we started sharing our ideas and attempted to find the main testing skills needed to solve the challenge.

Listed below are the skills that I felt were key for testing after the exercise.

Automation skills:
Here I had to repeatedly test the same scenario with 253 different combinations of input test data which was monotonous. Writing a script to automatically iterate through each of these combinations to test can save lots of cost and time. Also manual testing is prone to human errors while automated tests tend to make it a lot more easier and accurate. So, automation skills are definitely required to effectively test in this context.

Analytical \ Pattern detection skills:
We need to have strong analytical skills to detect the patterns in the challenge. Test design will become easier once we are aware of the patterns. This skill is indispensable when doing testing at workplace since we cannot jump to any conclusion without thorough analysis and reasoning.

Simpifly the problem:
Even though the problem is complex, the solution can be quite simple. Through pattern detection we can generalize the solution. Quickly decompose the problem into smaller chunks for investigation and simplify it.

Focus\Defocus heuristics:
When the challenge could not be cracked with the black box approach I decided to defocus and started focusing on other approaches like boundary value analysis, viewing source code and collaboration with other testers and it helped.

Pairing and collaboration:
Some participants paired up and their consolidated findings were more than that of solos. http://testing.gershon.info/rapidreporter/pairingrr/WTA06CardsWithNancy/20110129_211619.htm is a report produced by Shmuel Gershon by pairing up with a fellow tester

Finally, these are the key takeaways from the sixth session of the American chapter of weekendtesting. I will definitely fallback on these key skills when posed with a real challenge at workplace since it really helps! Feel free to share your thoughts or question anything in this article.

References:

http://weekendtesting.com/archives/1736

Filed Under: Events Tagged With: , , ,

Software testing approaches

January 13, 2011 By 2 Comments

“When developing our test strategy, we must minimize the impact caused by changes in the applications we are testing, and changes in the tools we use to test them.”  –Carl J. Nagle

This article evaluates the effectiveness of testing approaches and sheds light on the areas we need to concentrate on while testing. In essence 20% of the test cases should account for 80% of the ROI. For this, the testers need to question the logic behind the test case development by asking: 1) What is the probability of the occurence of bugs in this area? 2) Do I really need to automate this test? 3) What is the impact of GUI changes on the test case? The testers who are short-sighted tend to focus only on the testing milestones like pass rate/deadlines and fail to achieve the long term goals of testing like maintainability, robustness and coverage.

Credit: Flickr by e-strategyblog.com

 

GUI or End-to-End testing:

In this testing approach we try to simulate a customer completing some task involving multiple features and system interactions. This is the least effective test automation approach

Advantages:
1. Exposes Behavioral issues

Disadvantages:
1. Maintenance nightmare: Whenever the GUI changes the tester needs to revise the test case. GUI test cases are tied to the very trivial details of the UI that almost any change in the GUI can cause the test cases to fail. Thus, maintenance becomes tedious. Considering the time to fix it we could rather re-create the test case from scratch
2. Last minute design changes can cause havoc to the automation team
3. Longer test case execution time
4. Does not find much bugs

While developing the test strategy we must minimize the impact of UI changes on the test cases. I would recommend executing GUI test cases later in the lifecycle when the UI is stable and after all the critical fuctional bugs are exposed by the approach “Component based” testing explained below.

 

API or Component based testing:

In this testing approach we directly hit/communicate with the component to test the application. A component is an integrated aggregate of one or more units. We parameterize the properties and pass it as the input data to the API.  The corresponding methods are triggered on the data and it tests the underlying business logic/functionality of the application.

Advantages:
1. Maintainable: As opposed to End-to-End testing this is not dependent on UI changes and hence reduces the automation efforts.
2. Faster test case execution time and hence reduces the Mean Time To Test (MTTT)
3. Finds many bugs early on in the lifecycle. Exposes functional issues

Filed Under: Software Testing, Testing practices Tagged With: , , , , , , , , , ,

Women in leadership

January 10, 2011 By 1 Comment

“Don’t leave before you leave”

-  Sheryl Sandberg, Fortune’s Most Powerful Women.

Here is a video from TED worth watching for young aspiring women in tech. The Role Model Sheryl Sandberg breaks the spell of the  inner limitations that pose barrier to young women who are geared to succeed in tech. She inspires young women to look at technology as a viable and wonderful option and tells it’s time to change the ratio of women in leadership.

Filed Under: Project Management Tagged With: , , ,

Leadership through innovation

January 3, 2011 By 2 Comments

“Unless you are breaking stuff,you are not moving fast enough.”

- Mark Zuckerberg

 

Credit: http://www.businessinsider.com

Filed Under: Project Management Tagged With: , , , , ,

Shell Scripting for beginners with a simple example Part 1

December 15, 2010 By 1 Comment

This article gives a practical exposure to beginners about shell scripting on UNIX platform. Shell script is a script for the shell command-line interpreter. Shell programming is fun and is used to automate highly repetitive time-consuming manual tasks like environment setup, post processing and configuration value changes that involve file manipulation. We can run shell scripts by installing Cygwin and use Emacs as the editor.

We point to the interpreter (i.e) the shell in the very first line of the script. This is called shebang. #! bin/sh is the default shebang if nothing is mentioned.

Writing a simple Shell Script:

Step1:  In the shell command prompt, navigate to your home directory

sh-3.2$ cd $HOME

cd means change directory. $HOME variable refers to a directory on the operating system containing the user’s files. It can be represented in short using ~ symbol.

Step2:  In the home directory create a new file named date.sh

Step3: Modify the access rights on the file so that you can edit/save it using chmod command.

sh-3.2$ chmod 777 date.sh

Everyone has read, write and execute permissions on the file

sh-3.2$ chmod 755 date.sh

Everyone can read and execute this file but I alone should be able to modify it.

Step 4: Now let us create a simple shell script to display the date and hostname.

We store the hostname value in a variable called HOST. We use the date function to display date in the format %m-%d%-%Y. And echo command is used for displaying/printing the output on the command line.

#! /bin/sh

HOST=$(hostname)

echo “——————————————————————-”

echo “Date:$(date +”%m-%d-%Y”)              Hostname:$HOST”

echo “——————————————————————-”

Step 5: Run this shell script using the command ./date.sh from the home directory where you saved it.

OUTPUT:

sh-3.2$ ./date.sh

——————————————————————-

Date:12-14-2010              Hostname:dellwin7-PC

——————————————————————-

Filed Under: Shell Scripting, Technology Tagged With: , , , , , , , , , , ,

Bug prevention – the new definition for software testing

December 13, 2010 By 4 Comments

“Bug prevention is the testing’s first goal” – B.Beizer

Beizer stated that “the act of designing tests is one of the most effective bug preventers known,” which extended the definition of testing to error prevention as well as error detection activities. This article gives insight into the power of early testing.

Most of the testers find bugs weeks/months after the designers and developers inject the bug into the product. How many testers proactively engage with their development partners to prevent the bug from ever getting into the product in the first place? If we continue to think of testing as an after-the-development process then our role is nothing more than a bug finder.

To advance our discipline of testing, the testers need to partner with the developers earlier in the lifecycle to move quality upstream and prevent issues. One of the most powerful ways of improving the quality of the product and the team effectiveness is to focus the testing efforts on defect prevention rather than defect detection. This doesn’t negate or eliminate the need for testing. Testing for defect detection alone is like simply treating the symptoms of the problem and ignoring the root cause of the real problem. Defect prevention is about addressing the real problem.

Here are some of the ways for the tester to work in a symbiotic relationship with the developer and move quality upstream:

  1. Participate in code inspection
  2. Engage in design reviews
  3. Prototyping with program manager and designers to prevent sub-optimal designs
Filed Under: Software Testing, Test Management Tagged With: , , , , , ,

Introduction to Test Coverage

December 4, 2010 By Leave a Comment

 

Test coverage is an indirect measure of quality. It measures how well the test suite actually tests the product. Test coverage analysis helps to find out areas in the program not exercised by the test suite so that we can create additional test cases to cover those gaps. It is a structural testing technique that compares the program behavior against the apparent intention of the source code. It helps finding possible pitfalls in the structure and logic of the program.

There are two broad classifications of coverage measures – Path-based and Fault-based:

Path-based coverage: This requires the execution of a particular component of the program. For example, if the scope of the testing is limited to the payments domain alone, we design the test cases in a way that it fully exercises the payments component of the program

Fault-based coverage: This requires that the test suite exercises a program in a way that reveals all the likely faults. It is used when the test strategy is focused on finding all the defects in a feature irrespective of the components involved.

Test coverage methods:

There are 6 fundamental test coverage methods as below:

Statement coverage: This reports whether each executable statement is encountered by the test suite. It is also referred to as line/segment/basic block coverage.

Decision coverage: This reports whether Boolean expressions tested in control structures are evaluated to both true and false. This is also known as branch/path coverage.

Condition coverage: This reports the true or false outcomes of each boolean sub-expressions separated by logical-OR and logical-AND if they occur

Multiple condition coverage: This reports whether every possible combination of Boolean sub-expression occurs

Function coverage: This reports whether the test suite encountered each function/procedure of the program

Call coverage: This reports whether each function call is executed

Filed Under: Software Testing, Testing Concepts Tagged With: , , , , , , , , , ,

Guidelines for writing software test plan

October 31, 2010 By 2 Comments

The test plan defines the scope, approach, resources and schedule of the intended testing activities. It identifies the features to be tested, the testing tasks and who will do the tasks.

The three main elements of a test plan are:

  1. Test Coverage: The test coverage measures the proportion of the program exercised by the test suite. It is usually expressed in percentage. It helps in finding areas that are not exercised by the test suite. Having complete test coverage ensures 100% test effectiveness and that the test suite reveals all the defects
  2. Test Methodology: The approach, techniques and tools to be used for testing
  3. Test Responsibilities: Includes the features/functions to be tested and the criteria for success/failure for the tests

The Test Plan contains the following:

  1. The Project Overview
  2. Objective and scope of testing
  3. The test methodology
  4. Resources and responsibilities
  5. The testing efforts and schedule
  6. Entry and exit criteria for testing
  7. The features to be and not to be tested
  8. Success/failure criteria
  9. Test environment
  10. Risk factors and contingency plan
  11. References to the supporting documents which includes FSD, SRS, TDD and the like.

The scheduling of the testing tasks and milestones is done considering the below factors:

  1. Complexity of the feature
  2. Competency of the resource
  3. Stability of the requirements

Each of these factors could be rated as high, medium or low.

For large projects, there will be a single test strategy document. There will be many test plans that draws its content from the test strategy. The Test Strategy is a high level document that defines the testing approach to achieve the testing objectives. It is prepared by the project manager and it will not be updated often. The test plan is derived from Software Requirements Specification, Technical Design Document, and Functional Specification Document. It includes test strategy/test approach as well. It is prepared by the test lead. The test plan needs to be updated often to reflect any deviation from the original plan.

Credit:  Jurvetson (flickr)

If a feature is not to be tested the test plan should state why it is not. For ex: It could be: a low risk feature, tested and found to be stable in the past, not pushed in the current release.

There are several inherent software risk factors in a project like complexity or newness. A well rounded test plan notes all the risks and tasks interdependencies in the test plan and communicates to the stakeholders regarding those dependencies and risks.  Dependencies could be like cross-functional impact of a new feature, delay in getting stable builds, late change in original requirements, late delivery of fix and resource availability.

Once the test plan is documented it requires review and approval by the test manager /release management team in order to move the next phase of testing.

Filed Under: Software Testing, Test Management Tagged With: , , , , , , , , , , , , ,
« Older Posts
Newer Posts »