Archive for 'Test Automation Tools'

If you have several suites to be run, you could run them in parallel at the same time in different threads.

Parallel suites

In command line the ant task suiteThreadPoolSize can be used to run several suites in parallel using org.testng.TestNG as shown below:

java org.testng.TestNG -suiteThreadPoolSize 3 testng1.xml testng2.xml testng3.xml

This command runs the three suites in 3 different threads in parallel

Parallel methods, tests, classes

In testng.xml file, using the attribute parallel inside the suite tag instructs TestNG to runs tests in parallel in different threads. The attribute parallel accepts certain values as outlined below:

<suite name="mysuite" parallel="methods" thread-count="5">
<suite name="mysuite" parallel="tests" thread-count="5">
<suite name="mysuite" parallel="classes" thread-count="5">
<suite name="mysuite" parallel="instances" thread-count="5">

1. parallel=”methods”: TestNG will run all the test methods in parallel in different threads. The dependent methods will also run in different threads but would respect the order that was specified.
2. parallel=”tests”: TestNG will run all the methods in the same test in the same thread but each test will be run in separate threads. This allows to group the thread-safe classes in the same test so that they will run in the same thread while taking advantage of TestNG’s capability to run tests in several threads.
3. parallel=”classes”: TestNG will run all the test methods in the same class in the same thread and each class will be run in separate thread.
4. parallel=”instances”: TestNG will run all the test methods in the same instance in the same thread and each instance will be run in separate threads.

Also, the attribute threadPoolSize of the @Test annotation could be used to run tests in parallel as shown below.

@Test(threadPoolSize=5, invocation-count=10, timeout=10000)
public void testMethod()

This will invoke the method testMethod 10 times in 5 different threads. The timeout of 10 seconds ensures that the thread is not blocked for more than 10 seconds. The timout applies for both parallel and non-parallel mode.

TestNG allows to perform groupings of test methods. The methods can be declared to belong to groups and TestNG can be invoked to include/exclude certain set of groups. This provides maximum flexibility to structure the tests in a more intuitive way.

Step1: Declare groups in java class

Suppose we would like to group the test methods into two categories functional and sanity. We declare the methods to belong to either of the two groups using the “groups” attribute of @Test annotation.

public class Test1
  public void testMethod1(){
  public void testMethod2(){
  public void testMethod3(){


Step2: Specify groups in TestNG.xml file

Suppose we would like to run the sanity tests alone. We specify the groups in the TestNG.xml file using the <groups> tag inside the <suite> or <test> tag as shown below:

<suite name="mySuite" verbose="1">
  <test name="Regression">
        <include name="sanity"/>
      <class name="sample.Test1"/>

TestNG is a testing framework designed to simplify the testing needs.


Quick overview of TestNG.xml file

  • A suite is an xml file represented by <suite> tag. The TestNG suite can contain one or more TestNG tests
  • A test is represented by the <test> tag. A TestNG test can contain one or more TestNG classes
  • A class is a java class having TestNG annotations. It is represented by the <class> tag. A TestNG class can contain one or more TestNG methods
  • A method is method configured with @Test annotation


Sample TestNG.xml file



Also, it is possible to use packages instead of classes in TestNG.xml file. For this package tag is used as shown below:


Is selenium the future of test automation? That leads us to yet another important question, what was the past of test automation? Well…,I was just kidding. There are numerous amazing test automation tools out there but none of them could takeover the field of test automation. Why? It’s because, when you take a closer look at any testing project, the complexity increases so much that it takes a real tester with highly sophisticated testing practices and coding skills more than a mere tool to handle those situations. Yes, any tool no matter how good it is, cannot simply provide the foundation for a sound test automation framework. It is we, the “testers”, who can do it! But how?

This article shows step-by-step how selenium with a hybrid of other tools and practices could be used by the tester to develop a highly maintainable, robust, sophisticated and sound test automation framework.


This article gives a 30-feet overview of how a test automation framework with selenium looks like.


The little secrets of test automation:

Would you be interested to know how I can automate tests in just six weeks while others usually take over six months? I automate far faster and much cheaper than anyone else can do! Come on, my little secrets are so little.

Often in a testing project we tend to handle things in our own way that we see fit. The delicate test automation framework gets upset due to these gimmicks. Let me introduce you to the few main things that really counts during test automation:

  1. Customer focus
  2. Easy-to-understand test case listing the user actions
  3. Decoupling of test data from test execution
  4. Intelligent exception handling
  5. Data-driven testing framework
  6. Modular testing framework
  7. Separation of test verification from test script

Even if you are new to automation you can easily get a hang of it as you follow the steps outlined. Let me illustrate these principles with a simple demo.


To execute the practical demonstration in this article we require the installation of JDK, Selenium Server, Selenium client driver for java, ant, TestNG, eclipse and TestNG plugin for eclipse.

For installation/setting up of Selenium, please refer the article “Selenium RC configuration for java – all in a nutshell”.

In this demo I am using Ubuntu as platform. You could use your favorite platform for coding. So, let us walkthrough the steps for developing a robust and highly maintainable test automation framework with selenium.


Step 1: Create java project and add references to external jars:

Suppose we want to dress perfectly for the weather with a rain hat, a rain coat and an umbrella. Now let’s develop a selenium test automation framework to test a very simple application that I have created called Weather Monster – “”.

  1. Open eclipse → Create a new java project called weatherMonster
  2. Click on Next button → click on the Libraries tab
  3. Click on Add External JARs button and add the respective jar files for selenium server, selenium java client driver and TestNG as shown in the diagram below → Click Finish



Step 2: Automatic configuration of test environment

A configuration script contains the key value pair needed to setup the test environment. These values could be changed dynamically using a shell script.

Let’s create a shell script to dynamically start the selenium server.

Create a new folder called “config” in the weatherMonster project and place the below shell script named in it.

cd ~/Documents/working/installable/selenium/jar\ files/
gnome-terminal --title="config" -x bash -c "java -jar selenium-server-standalone-2.0rc2.jar"

This shell script navigates to the folder path containing the selenium server jar file (Replace this path with respective path on your machine). It then opens a new GNOME-TERMINAL named config and runs the selenium server by execution of the command “java -jar selenium-server-standalone-2.0rc2.jar”

To grant execution privileges to the script use chmod command as below:

chmod 777

Then run the script using the below command:


We need to run this script each time before test suite execution to start the selenium server.

Right click the src folder and create a package named config. Right click the config package and create a java class called

Let’s assign values to the environment variables such as host, port, browser and so on.
Since we do not want to hard code the test data and we would like to decouple test data from test execution let’s assign the test data to the variables separately in the config folder and organize them as shown below:

package config;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.HttpCommandProcessor;
public class seleniumTest
	/*Environment variables*/
	public final String HOST = "localhost";
	public final int PORT = 4444;
	public final String BROWSER = "firefox";
	public final String BASE_URL = "";
	//To set the baseURL and server port
	public final HttpCommandProcessor PROC = new HttpCommandProcessor(HOST,PORT, BROWSER, BASE_URL);
	//To create a selenium object for passing as variable
	public final DefaultSelenium Selenium = new DefaultSelenium(PROC);
	public final String TIMEOUT="60000";
	/*Weather Monster home page*/
	public final String HOME_MSG = "This is weathermonster, bringing you the weather";
	public final String HOME_PAGETITLE = "Weather Monster";


Step 3: Modularization of commonly used methods

A module is a small independent script that represents AUT(Application Under Test). The most commonly used methods in the test case should be created as modules in separate folders that can be called and reused by many test cases. This increases the maintainability of the test suite. If there is a change, it needs to be applied only in one single place as opposed to all the places thus saving lots of time and cost.

Modularization also makes the test case easy-to-read and understand.

Consider the below test automation code. Can you understand it? It is a technical stack containing all the configuration details which is very hard to understand.


The test case should list the user actions not the technical details. This makes the test case customer-focused and easy-to-read. Now, let’s move out all the technical stack from the test case to a separate module.

Right click the src folder and create a package named module. Right click the module package and create a java class called

Here we code all the technical details of starting selenium server for test initialization so that the actual test case is freed from these unnecessary details and can cover the main area to test.

package module;
import com.thoughtworks.selenium.*;
import org.testng.annotations.*;
import config.*;
public class testInit extends SeleneseTestBase
	//Declare a new selenium test
	seleniumTest test = new seleniumTest();
	public void testSetup(DefaultSelenium selenium)
			//Open, maximize and focus on the facebook page. Delete all cookies
			selenium.waitForPageToLoad(test.TIMEOUT); //Handles timeout exception
			selenium.deleteAllVisibleCookies(); //Clear all session cookies before testing
		catch (Exception e)

Step 4: Writing a selenium test case

Now that we already have modules for test setup and initialization, the writing of a selenium test case doesn’t take much effort. All we need to do is just call these modules and reuse them in our test case. This allows us to focus more on the customer.

A good automation script should contain the actual code logic to perform and validate a business scenario.

Right click the src folder and create a package named testSuite. Right click the testSuite package and create a java class called

package testSuite;
/* Test case for weather monster home page*/
import module.*;
import config.*;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
public class weatherMonsterHome extends SeleneseTestBase
	//Declare a new selenium test
	seleniumTest test = new seleniumTest();
	testInit setUp = new testInit();
	public void testWeatherMonster()
			/*Test the home page of Weather Monster
			 * Pass the expected message and page title of the home page as parameters
			weatherMonster(test.Selenium, test.HOME_MSG,test.HOME_PAGETITLE);
	public void weatherMonster(DefaultSelenium Selenium, String strHomeMsg, String strExpectedHomePageTitle)
			/*Verification of Weather Monster home page*/
			System.out.println("Message on the Weather Monster home page is displayed successfully");
			System.out.println("Title of Weather Monster home page is correct");
		catch(Exception e)

This decoupling of test script from all the test execution details has made the test case a very easy-to-read list of user actions.

Ultimately the test structure is as below:

Step 5: Run the test case

  1. Navigate to the config folder and give the below command to start selenium server:
  • Right click on and select “Run as TestNG test”.



RemoteTestNG starting

Message on the Weather Monster home page is displayed successfully

Title of Weather Monster home page is correct

PASSED: testWeatherMonster


We have missed out on a few little secrets of test automation like intelligent exception handling and data-driven testing. We shall get introduced to them in our next article, so stay tuned!

Feel free to ask any question or post comments.

What do we need to get started with selenium RC?

Configuration of Selenium RC test automation framework with Java client driver is not that simple as it requires the installation of so many packages and it takes a die-hard effort to get all the required information on getting started at one place. It’s sometimes boring and you might ask yourself “Is this the only way to get started?“. This article is not a shortcut but still presents all in a nutshell on how to get started with selenium RC. The below nutshell picture portrays all the software packages needed to set up Selenium RC for java. So now lets get started on the long road of setting up our test automation framework using selenium RC and java.

Below are the step-by-step instructions for the installation and configuration of Selenium RC for java client driver on ubuntu platform. To start selenium server we need JAVA. So, let’s start with the installation of java.

Installation of java and setting class path:

Download Java JDK from the below path:

To find the exact path where java is installed give the below command:

which java

OUTPUT: /usr/bin/java Then set the path for JDK using the below commands:

export PATH=$PATH:/usr/bin/java
export JAVA_HOME=/usr/bin/java

To check whether java path is set properly give the below command

java -version

It should display the version of the Java installed.

Installation of selenium RC with java client driver:

Selenium Server: This is required to run the Selenium RC style scripts. Download and extract the selenium server jar file from:

Selenium Client Driver for java: To create scripts that interact with the Selenium Server we need the selenium client driver. There are language-specific client drivers for ruby,c# and so on. Let us install the selenium client driver for java. Download the client-driver for the programming language java:

Installation of ant:

Install ant on ubuntu using the below command

sudo apt-get install ant ant-optional

To find where ant has been installed give the below command:

which ant

Set the path of ant using the PATH variable as below:

export PATH=$PATH:/usr/bin/ant

Installation of Eclipse:

Install eclipse in ubuntu using the below command:

sudo apt-get install eclipse

TestNG plugin for eclipse:

1. Open eclipse. 2. Click Help → Install New Software.. 3. Then click on Available Software Sites link 4. Click on the Add button. Enter the site url as to add TestNG plugin to eclipse. Now the TestNG plugin for eclipse is added as below:

Installation and configuration of TestNG on ubuntu:

The below command installs TestNG on ubuntu:

git clone git://

Building TestNG with ant: Download from the below site into the home directory unzip using the below command:


Now navigate to the testng directory. Copy ivy-2.1.0.jar into /ant/lib directory as shown below:

sudo cp ivy-2.1.0.jar /usr/share/ant/lib

Now give ant command in the testng directory:



Now we have installed all that we need to get started with Selenium RC using the client driver for java. In the next article, we will create a test suite using this setup.


The earlier article “Crucial aspects of browser compatibilty testing with Selenium Grid” gives insight into browser compatibility testing andthe architecture of one of the most popular browser compatibility testing tools, selenium grid. Nowlet‘sget our hands dirty with test automation using Selenium Grid.



The practice exercise in this article requires Eclipse, TestNG, ant, Java, Selenium RC and Selenium Grid. You can execute the code on your favorite platform. For this demo I am using ubuntu as platform.

Installation of Selenium Grid:

Please refer the documentation in seleniumhq for selenium grid download and installation instructions

Unzip the Selenium Grid folder and execute the below command in the Selenium Grid root directory to check whether it is installed correctly or not:

ant sanity-check

The output should be BUILD SUCCESSFUL

Now let’s get started.


Step 1. Launch of selenium hub:

Open a terminal and launch the selenium hub using the below command:

ant launch-hub

The hub is where the selenium remote controls providing the different environments like chrome on ubuntu or IE on Windows register themselves. We can view the list of remote controls registered with the hub at http://localhost:4444/console

Now the available remote controls section in the hub console displays nothing  since we did not start any remote control yet.


Step 2. Launch of selenium remote controls:

To start a remote control open a new terminal and issue the below command:

ant launch-remote-control

Then, open the hub console http://localhost:4444/console . The selenium grid hub snapshot displays a remote control on port 5555 for firefox. This is the default environment if we do not specify our own.

We can configure more than one selenium remote control with the same port but more than one remote control cannot run tests on the same port. The reason behind this is simple. For example, more than one person can call the same phone number but cannot talk over the phone to the same person at the same time. Only one call can be answered on a single phone at a time. Similarly it is impossible to run tests in parallel on two selenium remote controls configured with the same port number. We need to configure each remote control with unique port numbers. How do we do this?

If we want to specify our own environment we need to use the Dport and Denvironment options as below:

ant -Dport=5556 -Denvironment=*chrome launch-remote-control


Step 3. Automatic environment setup:

Let’s create a bash script for automatically opening up new terminals and launching several remote controls with the desired environment. If you are working on windows you need to create a batch file instead of a bash script. The below script works only on ubuntu platform.

Navigate to the selenium grid directory and create the below bash script. Let’s call it

gnome-terminal --title="Hub" -x bash -c "ant launch-hub"
sleep 150
echo "10 seconds wait over"
gnome-terminal --tab --title="Chrome" -x bash -c "ant -Dport=5555 -Denvironment=*chrome launch-remote-control" &amp;
gnome-terminal --tab --title="Firefox" -x bash -c "ant -Dport=5556 -Denvironment=*firefox launch-remote-control" &amp;
gnome-terminal --tab --title="Opera" -x bash -c "ant -Dport=5557 -Denvironment=*opera launch-remote-control"

The bash script opens a new gnome terminal called “Hub” and launches the selenium hub. It sleeps for 150 seconds waiting for the hub to get started. Depending on the time taken you can change the sleep period. The script then opens three other new terminals and registers three remote controls with the respective environments specified. On execution of the script the below output is got on http://localhost:4444/console page.

Now the environment is setup for selenium grid automation. Let’s write a simple selenium grid program.



Step 4. Creating a java project called giftShop:


Suppose you would like to cheer up your friend on her birthday and give her the best birthday party of all time. You are planning  to celebrate your friend’s birthday with balloons, cakes, ice creams and of course a birthday gift!. You are going to surprise your friend with “I have got something for you” from in such a way that no one can ever put a price tag on the friendship. Let’s write a selenium program to test the display of birthday gifts page on website on multiple browsers.


  1. In Eclipse create a new java project called “giftShop”. Click on Next button and click on the “Libraries” tab. Click on “Add External JARs” button. Select the testng and selenium server JAR files as below and click on finish button.



2. Now create a new TestNG class inside the giftShop project. In the source folder field, browse to /giftShop/src. Give the package name as giftShop and class name as giftBox. Click on finish button.

Step 5. Enable testing in parallel mode in multiple browsers:

Right click on the giftShop project. Click on “Convert into TestNG” option. “Generate testng.xml” window will get displayed. This option will be available only if the TestNG plugin for eclipse is installed. Select “tests” in parallel mode dropdown and click on Finish button.


To organize the project structure create a folder called config and place the generated testng.xml file in it.

In the testng.xml file we just created, specify the browser and port names with which we have configured our selenium remote control as in the below piece of code.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "">
<suite name="Suite" parallel="tests">
	<test name="Chrome">
		<parameter name="" value="localhost"/>
		<parameter name="selenium.port" value="5555"/>
		<parameter name="selenium.browser" value="*chrome"/>
		<parameter name="selenium.url" value=""/>
		<parameter name="bdayLink" value="//li[@id='nav-oc']/div/ul[3]/li[3]/a/b"/>
			<class name="giftShop.giftBox" />
	<test name="Firefox">
		<parameter name="" value="localhost"/>
		<parameter name="selenium.port" value="5556"/>
		<parameter name="selenium.browser" value="*firefox"/>
		<parameter name="selenium.url" value=""/>
		<parameter name="bdayLink" value="//li[@id='nav-oc']/div/ul[3]/li[3]/a/b"/>
			<class name="giftShop.giftBox" />
	<test name="Opera">
		<parameter name="" value="localhost"/>
		<parameter name="selenium.port" value="5557"/>
		<parameter name="selenium.browser" value="*opera"/>
		<parameter name="selenium.url" value=""/>
		<parameter name="bdayLink" value="//li[@id='nav-oc']/div/ul[3]/li[3]/a/b"/>
			<class name="giftShop.giftBox" />

Step 6: Writing selenium grid program:

This selenium grid program will start selenium on chrome, firefox and opera. It would open up the website and click on birthday as occasion. It asserts whether the landing page displays the birthday gifts in all the three browsers as expected.


package giftShop;
import org.testng.annotations.Test;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Parameters;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;
public class giftBox 
	Selenium selenium;
	//Declaration of variables used in this program
	String strTimeout="30000";
	String bdayTitle="Birthday Gifts - Birthday Gift Ideas from";
	int intsleepTime=15000;
    //Reads the values of host, port, browser and url "" from testng.xml
    @Parameters({ "", "selenium.port", "selenium.browser", "selenium.url" })
    public void beforeClass(String host, String port, String browser, String url)
        selenium = new DefaultSelenium(host, Integer.parseInt(port),browser, url);
    @Parameters({ "bdayLink" })
    public void testB(String bDayGift)
        	//Click on birthday as occasion on website;
        	//Verify that the birthday gift items page is displayed
        	Assert.assertEquals(bdayTitle, selenium.getTitle());
        catch (Exception e)
            // TODO Auto-generated catch block
    public void afterClass() 

Running the test case:

Please do the following to run the selenium grid test case (after the hub and RCs are launched):
1. Right click on the testng.xml file
2. Click on Run as “TestNG Suite”

Please feel free to post any comments or ask questions.

What is browser compatibility for software? In different browsers the same text, graphics and color can appear differently. Some of the browsers do not support Javascript or Active-X control which are required by the software to provide the desired functionality to the end-user. The critical areas of the software is expected to appear and function as desired on all the commonly used browsers like safari, IE, firefox etc. This ensures that the code developed supports browser compatibility, one of the crucial aspects of software development.

Examples of browser compatibility issues:
1. The date input field is validated for incorrect date-formats using javascript. If javascript is not supported by the browser, the software will allow the customer to enter invalid date which is a defect

2. A country drop-down is sorted alphabetically using a client-side javascript. The same rule applies as one. If it is not supported, the user finds it very difficult to select a country from the dropdown

3. Misalignment of controls on the web page which looks frustrating to the customer

4. Web page looking cluttered on mobile browsers

Having elaborated on browser compatibility, let us discuss a very popular tool called selenium grid that provides a very easy way to execute multiple tests in parallel on different browsers. By running the test suite concurrently on several browsers it reduces the cost for browser compatibility testing and also dramatically speeds up the feedback cycle. This article is a great head-start on how selenium grid makes browser compatibility testing fun and easy. It explores the boundless potential it offers when used in conjunction with Selenium RC. In the next article “Six steps to complete test automation with selenium grid” we will get our hands dirty with selenium grid programming and execution.

Architecture of Selenium Grid:

Here we can see three main components in the diagram, Test, Selenium Hub and Selenium Remote Control.

The Test is a series of Selenium commands

The Selenium hub routes the selenese requests from the Test to the appropriate Selenium Remote Control.

The Selenium Remote Control is an instance of Selenium Server that registers itself to the hub when it starts describing the environment provided…….for example chrome on ubuntu, IE on windows.

Selenium grid is a distributed grid of Selenium Remote Controls. The Remote Control is an instance of Selenium Server that registers itself to the hub when it starts describing the environment provided. When the Selenium Remote Control is started we define which environment it provides to the hub. In the selenium test, when instantiating the Selenium driver, the hub environment is passed in the place of the browser string.

The Remote Control instances that have been registered with the grid show up in http://localhost:4444/console. The Selenium hub routes the selenese requests to the appropriate Selenium Remote Control. The hub puts the request on hold if there is no available Remote Control on the grid. As soon as the suitable Remote Control becomes available the hub will serve the request. The hub parses the tests that target specific environments. We can specifically target a particular browser and a particular platform for testing the software.

This is how Selenium grid works and it makes browser compatibility testing a breeze.

Junit and TestNG differ in the core design. Junit is a unit testing framework while TestNG addresses testing at a higher level. This article discusses the three main differences between TestNG and Junit.

1.     In Junit, one test case failure can cause a bunch of test cases to fail in the test suite. There is no option of skipping the set of dependent test cases. The dependent test cases are also reported as failures. For example, suppose there is a test case to test login and the next 10 test cases need to perform a transaction after login. If the login test case fails the other 10 test cases will also fail.

TestNG handles dependency between test cases. If one test case failure causes the failure of a group of test cases it skips that group and executes the rest of the test suite. The group that has dependency on the failed test cases is reported as skipped NOT failed.

2.     In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific subset of the testsuite say database related test cases instead of running the entire test suite. This can be done as below:

In the test case we define two groups DBTestcase and deprecated as below:

@Test(groups = {"DBTestcase", "deprecated"})
public void testMethod2()

In TestNG.xml config file we write the below piece of code to include only database related test cases and exclude the deprecated ones.

<test name="Sample">
			<include name="DBTestcase" />
			<exclude name="deprecated" />
		<class name="example1.Test1" />

In Junit for a long time it was not possible to run a specific subset of the test cases. We can either run the entire suite or run each test case individually. Junit 4.8 introduced a new feature called “Categories” to overcome this limitation. However groups are much easier to configure in TestNG.

3.     TestNG supports parameterization for objects. For example I can execute a single test case for multiple test data sets through the parameterization of DataProvider object. This makes the implementation of data driven testing more flexible in TestNG as below:

@DataProvider(name = "DP1")
public String[][] testData() throws Exception
String[][] sdataArray = {{"james","moosecat$", "","selenigr","selenigr1$", ""}};
return sdataArray;
@Test (dataProvider = "DP1")
public void evernoteSharing(String user1, String password1, String email1, String user2, String password2, String email2)

In junit test data is not parameterized using a DataProvider hook. Data-driven testing is implemented differently and involves more coding.

In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific groups instead of running the entire test suite.