TestNG – running tests in parallel

If you have several suites to be run, you could run them in parallel at the same time in different threads.

Parallel suites

In command line the ant task suiteThreadPoolSize can be used to run several suites in parallel using org.testng.TestNG as shown below:

java org.testng.TestNG -suiteThreadPoolSize 3 testng1.xml testng2.xml testng3.xml

This command runs the three suites in 3 different threads in parallel

Parallel methods, tests, classes

In testng.xml file, using the attribute parallel inside the suite tag instructs TestNG to runs tests in parallel in different threads. The attribute parallel accepts certain values as outlined below:

<suite name="mysuite" parallel="methods" thread-count="5">
<suite name="mysuite" parallel="tests" thread-count="5">
<suite name="mysuite" parallel="classes" thread-count="5">
<suite name="mysuite" parallel="instances" thread-count="5">

1. parallel=”methods”: TestNG will run all the test methods in parallel in different threads. The dependent methods will also run in different threads but would respect the order that was specified.
2. parallel=”tests”: TestNG will run all the methods in the same test in the same thread but each test will be run in separate threads. This allows to group the thread-safe classes in the same test so that they will run in the same thread while taking advantage of TestNG’s capability to run tests in several threads.
3. parallel=”classes”: TestNG will run all the test methods in the same class in the same thread and each class will be run in separate thread.
4. parallel=”instances”: TestNG will run all the test methods in the same instance in the same thread and each instance will be run in separate threads.

Also, the attribute threadPoolSize of the @Test annotation could be used to run tests in parallel as shown below.

@Test(threadPoolSize=5, invocation-count=10, timeout=10000)
public void testMethod()
{
  System.out.println("Hello");
}

This will invoke the method testMethod 10 times in 5 different threads. The timeout of 10 seconds ensures that the thread is not blocked for more than 10 seconds. The timout applies for both parallel and non-parallel mode.

TestNG groups using include/exclude

TestNG allows to perform groupings of test methods. The methods can be declared to belong to groups and TestNG can be invoked to include/exclude certain set of groups. This provides maximum flexibility to structure the tests in a more intuitive way.

Step1: Declare groups in java class

Suppose we would like to group the test methods into two categories functional and sanity. We declare the methods to belong to either of the two groups using the “groups” attribute of @Test annotation.

public class Test1
{
  @Test(groups={"functional","sanity"})
  public void testMethod1(){
  }
 
  @Test(groups={"sanity"})
  public void testMethod2(){
  }
 
  @Test(groups={"functional","sanity"})
  public void testMethod3(){
  }
}

 

Step2: Specify groups in TestNG.xml file

Suppose we would like to run the sanity tests alone. We specify the groups in the TestNG.xml file using the <groups> tag inside the <suite> or <test> tag as shown below:

<suite name="mySuite" verbose="1">
  <test name="Regression">
    <groups>
      <run>
        <include name="sanity"/>
      </run>
    </groups>
    <classes>
      <class name="sample.Test1"/>
    </classes>
  </test>
</suite>

TestNG.xml – A Quick Overview

TestNG is a testing framework designed to simplify the testing needs.

 

Quick overview of TestNG.xml file

  • A suite is an xml file represented by <suite> tag. The TestNG suite can contain one or more TestNG tests
  • A test is represented by the <test> tag. A TestNG test can contain one or more TestNG classes
  • A class is a java class having TestNG annotations. It is represented by the <class> tag. A TestNG class can contain one or more TestNG methods
  • A method is method configured with @Test annotation

 

Sample TestNG.xml file

 

 

Also, it is possible to use packages instead of classes in TestNG.xml file. For this package tag is used as shown below:

 

30-feet view of test automation framework with selenium

Is selenium the future of test automation? That leads us to yet another important question, what was the past of test automation? Well…,I was just kidding. There are numerous amazing test automation tools out there but none of them could takeover the field of test automation. Why? It’s because, when you take a closer look at any testing project, the complexity increases so much that it takes a real tester with highly sophisticated testing practices and coding skills more than a mere tool to handle those situations. Yes, any tool no matter how good it is, cannot simply provide the foundation for a sound test automation framework. It is we, the “testers”, who can do it! But how?

This article shows step-by-step how selenium with a hybrid of other tools and practices could be used by the tester to develop a highly maintainable, robust, sophisticated and sound test automation framework.

 

This article gives a 30-feet overview of how a test automation framework with selenium looks like.

 

The little secrets of test automation:

Would you be interested to know how I can automate tests in just six weeks while others usually take over six months? I automate far faster and much cheaper than anyone else can do! Come on, my little secrets are so little.

Often in a testing project we tend to handle things in our own way that we see fit. The delicate test automation framework gets upset due to these gimmicks. Let me introduce you to the few main things that really counts during test automation:

  1. Customer focus
  2. Easy-to-understand test case listing the user actions
  3. Decoupling of test data from test execution
  4. Intelligent exception handling
  5. Data-driven testing framework
  6. Modular testing framework
  7. Separation of test verification from test script

Even if you are new to automation you can easily get a hang of it as you follow the steps outlined. Let me illustrate these principles with a simple demo.

Pre-requisites:

To execute the practical demonstration in this article we require the installation of JDK, Selenium Server, Selenium client driver for java, ant, TestNG, eclipse and TestNG plugin for eclipse.

For installation/setting up of Selenium, please refer the article “Selenium RC configuration for java – all in a nutshell”.

In this demo I am using Ubuntu as platform. You could use your favorite platform for coding. So, let us walkthrough the steps for developing a robust and highly maintainable test automation framework with selenium.

 

Step 1: Create java project and add references to external jars:

Suppose we want to dress perfectly for the weather with a rain hat, a rain coat and an umbrella. Now let’s develop a selenium test automation framework to test a very simple application that I have created called Weather Monster - “http://weathermonster.heroku.com/”.

  1. Open eclipse → Create a new java project called weatherMonster
  2. Click on Next button → click on the Libraries tab
  3. Click on Add External JARs button and add the respective jar files for selenium server, selenium java client driver and TestNG as shown in the diagram below → Click Finish

 

 

Step 2: Automatic configuration of test environment

A configuration script contains the key value pair needed to setup the test environment. These values could be changed dynamically using a shell script.

Let’s create a shell script to dynamically start the selenium server.

Create a new folder called “config” in the weatherMonster project and place the below shell script named config.sh in it.

#!/bin/bash
cd ~/Documents/working/installable/selenium/jar\ files/
gnome-terminal --title="config" -x bash -c "java -jar selenium-server-standalone-2.0rc2.jar"

This shell script navigates to the folder path containing the selenium server jar file (Replace this path with respective path on your machine). It then opens a new GNOME-TERMINAL named config and runs the selenium server by execution of the command “java -jar selenium-server-standalone-2.0rc2.jar”

To grant execution privileges to the script use chmod command as below:

chmod 777 config.sh

Then run the script using the below command:

./config.sh

We need to run this script each time before test suite execution to start the selenium server.

Right click the src folder and create a package named config. Right click the config package and create a java class called seleniumTest.java

Let’s assign values to the environment variables such as host, port, browser and so on.
Since we do not want to hard code the test data and we would like to decouple test data from test execution let’s assign the test data to the variables separately in the config folder and organize them as shown below:

package config;
 
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.HttpCommandProcessor;
 
public class seleniumTest
{
	/*Environment variables*/
	public final String HOST = "localhost";
	public final int PORT = 4444;
	public final String BROWSER = "firefox";
	public final String BASE_URL = "http://weathermonster.heroku.com";
	//To set the baseURL and server port
	public final HttpCommandProcessor PROC = new HttpCommandProcessor(HOST,PORT, BROWSER, BASE_URL);
	//To create a selenium object for passing as variable
	public final DefaultSelenium Selenium = new DefaultSelenium(PROC);
 
	/*Timeout*/
	public final String TIMEOUT="60000";
 
	/*Weather Monster home page*/
	public final String HOME_MSG = "This is weathermonster, bringing you the weather";
	public final String HOME_PAGETITLE = "Weather Monster";
}

 

Step 3: Modularization of commonly used methods

A module is a small independent script that represents AUT(Application Under Test). The most commonly used methods in the test case should be created as modules in separate folders that can be called and reused by many test cases. This increases the maintainability of the test suite. If there is a change, it needs to be applied only in one single place as opposed to all the places thus saving lots of time and cost.

Modularization also makes the test case easy-to-read and understand.

Consider the below test automation code. Can you understand it? It is a technical stack containing all the configuration details which is very hard to understand.

 

The test case should list the user actions not the technical details. This makes the test case customer-focused and easy-to-read. Now, let’s move out all the technical stack from the test case to a separate module.

Right click the src folder and create a package named module. Right click the module package and create a java class called testInit.java.

Here we code all the technical details of starting selenium server for test initialization so that the actual test case is freed from these unnecessary details and can cover the main area to test.


package module;
 
import com.thoughtworks.selenium.*;
import org.testng.annotations.*;
import config.*;
 
public class testInit extends SeleneseTestBase
{
	//Declare a new selenium test
	seleniumTest test = new seleniumTest();
 
	@Test
	public void testSetup(DefaultSelenium selenium)
	{
		try
		{
			//Open, maximize and focus on the facebook page. Delete all cookies
			selenium.start();
			selenium.open("/");
			selenium.waitForPageToLoad(test.TIMEOUT); //Handles timeout exception
			selenium.windowMaximize();
			selenium.windowFocus();
			selenium.deleteAllVisibleCookies(); //Clear all session cookies before testing
		}
		catch (Exception e)
		{
			 e.printStackTrace();
		}
	}
}


Step 4: Writing a selenium test case

Now that we already have modules for test setup and initialization, the writing of a selenium test case doesn’t take much effort. All we need to do is just call these modules and reuse them in our test case. This allows us to focus more on the customer.

A good automation script should contain the actual code logic to perform and validate a business scenario.

Right click the src folder and create a package named testSuite. Right click the testSuite package and create a java class called weatherMonsterHome.java


package testSuite;
 
/* Test case for weather monster home page*/
 
import module.*;
import config.*;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
 
public class weatherMonsterHome extends SeleneseTestBase
{
	//Declare a new selenium test
	seleniumTest test = new seleniumTest();
	testInit setUp = new testInit();
 
	@Test
	public void testWeatherMonster()
	{
			/*Test the home page of Weather Monster
			 * Pass the expected message and page title of the home page as parameters
			 */
			setUp.testSetup(test.Selenium);
			weatherMonster(test.Selenium, test.HOME_MSG,test.HOME_PAGETITLE);
	}
 
	public void weatherMonster(DefaultSelenium Selenium, String strHomeMsg, String strExpectedHomePageTitle)
	{
		try
		{
			/*Verification of Weather Monster home page*/
			assertTrue(Selenium.isTextPresent(strHomeMsg));
			System.out.println("Message on the Weather Monster home page is displayed successfully");
 
			assertEquals(Selenium.getTitle(),strExpectedHomePageTitle);
			System.out.println("Title of Weather Monster home page is correct");
		}
		catch(Exception e)
		{
			e.printStackTrace();
		}
	}
}


This decoupling of test script from all the test execution details has made the test case a very easy-to-read list of user actions.

Ultimately the test structure is as below:

Step 5: Run the test case

  1. Navigate to the config folder and give the below command to start selenium server:
./config.sh
  • Right click on weatherMonsterHome.java and select “Run as TestNG test”.

 

OUTPUT:

RemoteTestNG starting

Message on the Weather Monster home page is displayed successfully

Title of Weather Monster home page is correct

PASSED: testWeatherMonster

 

We have missed out on a few little secrets of test automation like intelligent exception handling and data-driven testing. We shall get introduced to them in our next article, so stay tuned!

Feel free to ask any question or post comments.

Selenium RC configuration for java – all in a nutshell

What do we need to get started with selenium RC?

Configuration of Selenium RC test automation framework with Java client driver is not that simple as it requires the installation of so many packages and it takes a die-hard effort to get all the required information on getting started at one place. It’s sometimes boring and you might ask yourself “Is this the only way to get started?“. This article is not a shortcut but still presents all in a nutshell on how to get started with selenium RC. The below nutshell picture portrays all the software packages needed to set up Selenium RC for java. So now lets get started on the long road of setting up our test automation framework using selenium RC and java.

Below are the step-by-step instructions for the installation and configuration of Selenium RC for java client driver on ubuntu platform. To start selenium server we need JAVA. So, let’s start with the installation of java.

Installation of java and setting class path:

Download Java JDK from the below path: http://www.oracle.com/technetwork/java/javase/downloads/index.html

To find the exact path where java is installed give the below command:

which java

OUTPUT: /usr/bin/java Then set the path for JDK using the below commands:

export PATH=$PATH:/usr/bin/java
export JAVA_HOME=/usr/bin/java

To check whether java path is set properly give the below command

java -version

It should display the version of the Java installed.

Installation of selenium RC with java client driver:

Selenium Server: This is required to run the Selenium RC style scripts. Download and extract the selenium server jar file from: http://code.google.com/p/selenium/downloads/detail?name=selenium-server-standalone-2.0rc2.jar&can=2&q=

Selenium Client Driver for java: To create scripts that interact with the Selenium Server we need the selenium client driver. There are language-specific client drivers for ruby,c# and so on. Let us install the selenium client driver for java. Download the client-driver for the programming language java: http://seleniumhq.org/download/

Installation of ant:

Install ant on ubuntu using the below command

sudo apt-get install ant ant-optional

To find where ant has been installed give the below command:

which ant

Set the path of ant using the PATH variable as below:

export PATH=$PATH:/usr/bin/ant

Installation of Eclipse:

Install eclipse in ubuntu using the below command:

sudo apt-get install eclipse

TestNG plugin for eclipse:

1. Open eclipse. 2. Click Help → Install New Software.. 3. Then click on Available Software Sites link 4. Click on the Add button. Enter the site url as http://beust.com/eclipse to add TestNG plugin to eclipse. Now the TestNG plugin for eclipse is added as below:

Installation and configuration of TestNG on ubuntu:

The below command installs TestNG on ubuntu:

git clone git://github.com/cbeust/testng.git

Building TestNG with ant: Download cobertura-1.9.4.1-bin.zip from the below site into the home directory http://cobertura.sourceforge.net/download.html unzip using the below command:

unzip cobertura-1.9.4.1-bin.zip

Now navigate to the testng directory. Copy ivy-2.1.0.jar into /ant/lib directory as shown below:

sudo cp ivy-2.1.0.jar /usr/share/ant/lib

Now give ant command in the testng directory:

ant

The OUTPUT should be BUILD SUCCESSFUL.


Now we have installed all that we need to get started with Selenium RC using the client driver for java. In the next article, we will create a test suite using this setup.

Six steps for complete test automation with selenium grid

 

The earlier article “Crucial aspects of browser compatibilty testing with Selenium Grid” gives insight into browser compatibility testing andthe architecture of one of the most popular browser compatibility testing tools, selenium grid. Nowlet‘sget our hands dirty with test automation using Selenium Grid.

 

Pre-requisites:

The practice exercise in this article requires Eclipse, TestNG, ant, Java, Selenium RC and Selenium Grid. You can execute the code on your favorite platform. For this demo I am using ubuntu as platform.


Installation of Selenium Grid:

Please refer the documentation in seleniumhq for selenium grid download and installation instructions

http://selenium-grid.seleniumhq.org/step_by_step_installation_instructions_for_windows.html
http://selenium-grid.seleniumhq.org/get_started.html

Unzip the Selenium Grid folder and execute the below command in the Selenium Grid root directory to check whether it is installed correctly or not:

ant sanity-check

The output should be BUILD SUCCESSFUL

Now let’s get started.


 

Step 1. Launch of selenium hub:

Open a terminal and launch the selenium hub using the below command:

ant launch-hub

The hub is where the selenium remote controls providing the different environments like chrome on ubuntu or IE on Windows register themselves. We can view the list of remote controls registered with the hub at http://localhost:4444/console

Now the available remote controls section in the hub console displays nothing  since we did not start any remote control yet.

 

Step 2. Launch of selenium remote controls:

To start a remote control open a new terminal and issue the below command:

ant launch-remote-control

Then, open the hub console http://localhost:4444/console . The selenium grid hub snapshot displays a remote control on port 5555 for firefox. This is the default environment if we do not specify our own.

We can configure more than one selenium remote control with the same port but more than one remote control cannot run tests on the same port. The reason behind this is simple. For example, more than one person can call the same phone number but cannot talk over the phone to the same person at the same time. Only one call can be answered on a single phone at a time. Similarly it is impossible to run tests in parallel on two selenium remote controls configured with the same port number. We need to configure each remote control with unique port numbers. How do we do this?

If we want to specify our own environment we need to use the Dport and Denvironment options as below:

ant -Dport=5556 -Denvironment=*chrome launch-remote-control

 

Step 3. Automatic environment setup:

Let’s create a bash script for automatically opening up new terminals and launching several remote controls with the desired environment. If you are working on windows you need to create a batch file instead of a bash script. The below script works only on ubuntu platform.

Navigate to the selenium grid directory and create the below bash script. Let’s call it run.sh

#!/bin/bash
gnome-terminal --title="Hub" -x bash -c "ant launch-hub"
sleep 150
echo "10 seconds wait over"
gnome-terminal --tab --title="Chrome" -x bash -c "ant -Dport=5555 -Denvironment=*chrome launch-remote-control" &amp;
gnome-terminal --tab --title="Firefox" -x bash -c "ant -Dport=5556 -Denvironment=*firefox launch-remote-control" &amp;
gnome-terminal --tab --title="Opera" -x bash -c "ant -Dport=5557 -Denvironment=*opera launch-remote-control"

The bash script opens a new gnome terminal called “Hub” and launches the selenium hub. It sleeps for 150 seconds waiting for the hub to get started. Depending on the time taken you can change the sleep period. The script then opens three other new terminals and registers three remote controls with the respective environments specified. On execution of the script the below output is got on http://localhost:4444/console page.

Now the environment is setup for selenium grid automation. Let’s write a simple selenium grid program.

 

 

Step 4. Creating a java project called giftShop:

 

Suppose you would like to cheer up your friend on her birthday and give her the best birthday party of all time. You are planning  to celebrate your friend’s birthday with balloons, cakes, ice creams and of course a birthday gift!. You are going to surprise your friend with “I have got something for you” from www.gifts.com in such a way that no one can ever put a price tag on the friendship. Let’s write a selenium program to test the display of birthday gifts page on www.gifts.com website on multiple browsers.

 

  1. In Eclipse create a new java project called “giftShop”. Click on Next button and click on the “Libraries” tab. Click on “Add External JARs” button. Select the testng and selenium server JAR files as below and click on finish button.

 

 

2. Now create a new TestNG class inside the giftShop project. In the source folder field, browse to /giftShop/src. Give the package name as giftShop and class name as giftBox. Click on finish button.

Step 5. Enable testing in parallel mode in multiple browsers:


Right click on the giftShop project. Click on “Convert into TestNG” option. “Generate testng.xml” window will get displayed. This option will be available only if the TestNG plugin for eclipse is installed. Select “tests” in parallel mode dropdown and click on Finish button.

 

To organize the project structure create a folder called config and place the generated testng.xml file in it.

In the testng.xml file we just created, specify the browser and port names with which we have configured our selenium remote control as in the below piece of code.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests">
	<test name="Chrome">
		<parameter name="selenium.host" value="localhost"/>
		<parameter name="selenium.port" value="5555"/>
		<parameter name="selenium.browser" value="*chrome"/>
		<parameter name="selenium.url" value="http://www.gifts.com/"/>
		<parameter name="bdayLink" value="//li[@id='nav-oc']/div/ul[3]/li[3]/a/b"/>
		<classes>
			<class name="giftShop.giftBox" />
		</classes>
	</test>
	<test name="Firefox">
		<parameter name="selenium.host" value="localhost"/>
		<parameter name="selenium.port" value="5556"/>
		<parameter name="selenium.browser" value="*firefox"/>
		<parameter name="selenium.url" value="http://www.gifts.com/"/>
		<parameter name="bdayLink" value="//li[@id='nav-oc']/div/ul[3]/li[3]/a/b"/>
		<classes>
			<class name="giftShop.giftBox" />
		</classes>
	</test>
	<test name="Opera">
		<parameter name="selenium.host" value="localhost"/>
		<parameter name="selenium.port" value="5557"/>
		<parameter name="selenium.browser" value="*opera"/>
		<parameter name="selenium.url" value="http://www.gifts.com/"/>
		<parameter name="bdayLink" value="//li[@id='nav-oc']/div/ul[3]/li[3]/a/b"/>
		<classes>
			<class name="giftShop.giftBox" />
		</classes>
	</test>
</suite>

Step 6: Writing selenium grid program:

This selenium grid program will start selenium on chrome, firefox and opera. It would open up the www.gifts.com website and click on birthday as occasion. It asserts whether the landing page displays the birthday gifts in all the three browsers as expected.

 

package giftShop;
 
import org.testng.annotations.Test;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Parameters;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;
 
public class giftBox 
{
	Selenium selenium;
 
	//Declaration of variables used in this program
	String strTimeout="30000";
	String bdayTitle="Birthday Gifts - Birthday Gift Ideas from Gifts.com";
	int intsleepTime=15000;
 
    @BeforeClass
    //Reads the values of host, port, browser and url "www.gifts.com" from testng.xml
    @Parameters({ "selenium.host", "selenium.port", "selenium.browser", "selenium.url" })
    public void beforeClass(String host, String port, String browser, String url)
    {
        selenium = new DefaultSelenium(host, Integer.parseInt(port),browser, url);
        selenium.start();
        selenium.open("");
        selenium.windowMaximize();
        selenium.windowFocus();
        selenium.deleteAllVisibleCookies();
    } 
 
    @Test
    @Parameters({ "bdayLink" })
    public void testB(String bDayGift)
    {
        try
        {
        	//Click on birthday as occasion on www.gifts.com website
        	selenium.click(bDayGift);
        	selenium.waitForPageToLoad(strTimeout);
 
        	//Verify that the birthday gift items page is displayed
        	Assert.assertEquals(bdayTitle, selenium.getTitle());
            Thread.sleep(intsleepTime);
        }
        catch (Exception e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    } 
 
    @AfterClass
    public void afterClass() 
    {
        selenium.close();
        selenium.stop();
    } 
}

Running the test case:

Please do the following to run the selenium grid test case (after the hub and RCs are launched):
1. Right click on the testng.xml file
2. Click on Run as “TestNG Suite”

Please feel free to post any comments or ask questions.

Crucial aspects of browser compatibilty testing with Selenium Grid

What is browser compatibility for software? In different browsers the same text, graphics and color can appear differently. Some of the browsers do not support Javascript or Active-X control which are required by the software to provide the desired functionality to the end-user. The critical areas of the software is expected to appear and function as desired on all the commonly used browsers like safari, IE, firefox etc. This ensures that the code developed supports browser compatibility, one of the crucial aspects of software development.

Examples of browser compatibility issues:
1. The date input field is validated for incorrect date-formats using javascript. If javascript is not supported by the browser, the software will allow the customer to enter invalid date which is a defect

2. A country drop-down is sorted alphabetically using a client-side javascript. The same rule applies as one. If it is not supported, the user finds it very difficult to select a country from the dropdown

3. Misalignment of controls on the web page which looks frustrating to the customer

4. Web page looking cluttered on mobile browsers

Having elaborated on browser compatibility, let us discuss a very popular tool called selenium grid that provides a very easy way to execute multiple tests in parallel on different browsers. By running the test suite concurrently on several browsers it reduces the cost for browser compatibility testing and also dramatically speeds up the feedback cycle. This article is a great head-start on how selenium grid makes browser compatibility testing fun and easy. It explores the boundless potential it offers when used in conjunction with Selenium RC. In the next article “Six steps to complete test automation with selenium grid” we will get our hands dirty with selenium grid programming and execution.

Architecture of Selenium Grid:

Here we can see three main components in the diagram, Test, Selenium Hub and Selenium Remote Control.

The Test is a series of Selenium commands

The Selenium hub routes the selenese requests from the Test to the appropriate Selenium Remote Control.

The Selenium Remote Control is an instance of Selenium Server that registers itself to the hub when it starts describing the environment provided…….for example chrome on ubuntu, IE on windows.

Selenium grid is a distributed grid of Selenium Remote Controls. The Remote Control is an instance of Selenium Server that registers itself to the hub when it starts describing the environment provided. When the Selenium Remote Control is started we define which environment it provides to the hub. In the selenium test, when instantiating the Selenium driver, the hub environment is passed in the place of the browser string.

The Remote Control instances that have been registered with the grid show up in http://localhost:4444/console. The Selenium hub routes the selenese requests to the appropriate Selenium Remote Control. The hub puts the request on hold if there is no available Remote Control on the grid. As soon as the suitable Remote Control becomes available the hub will serve the request. The hub parses the tests that target specific environments. We can specifically target a particular browser and a particular platform for testing the software.

This is how Selenium grid works and it makes browser compatibility testing a breeze.

junit vs TestNG – core differences

Junit and TestNG differ in the core design. Junit is a unit testing framework while TestNG addresses testing at a higher level. This article discusses the three main differences between TestNG and Junit.

1.     In Junit, one test case failure can cause a bunch of test cases to fail in the test suite. There is no option of skipping the set of dependent test cases. The dependent test cases are also reported as failures. For example, suppose there is a test case to test login and the next 10 test cases need to perform a transaction after login. If the login test case fails the other 10 test cases will also fail.

TestNG handles dependency between test cases. If one test case failure causes the failure of a group of test cases it skips that group and executes the rest of the test suite. The group that has dependency on the failed test cases is reported as skipped NOT failed.

2.     In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific subset of the testsuite say database related test cases instead of running the entire test suite. This can be done as below:

In the test case we define two groups DBTestcase and deprecated as below:

@Test(groups = {"DBTestcase", "deprecated"})
public void testMethod2()
{
 
}

In TestNG.xml config file we write the below piece of code to include only database related test cases and exclude the deprecated ones.

<test name="Sample">
	<groups>
		<run>
			<include name="DBTestcase" />
			<exclude name="deprecated" />
		</run>
	</groups>
	<classes>
		<class name="example1.Test1" />
	</classes>
</test>

In Junit for a long time it was not possible to run a specific subset of the test cases. We can either run the entire suite or run each test case individually. Junit 4.8 introduced a new feature called “Categories” to overcome this limitation. However groups are much easier to configure in TestNG.

3.     TestNG supports parameterization for objects. For example I can execute a single test case for multiple test data sets through the parameterization of DataProvider object. This makes the implementation of data driven testing more flexible in TestNG as below:

@DataProvider(name = "DP1")
public String[][] testData() throws Exception
{
String[][] sdataArray = {{"james","moosecat$", "james@gmail.com","selenigr","selenigr1$", "selenigr@yahoo.co.uk"}};
return sdataArray;
}
 
@Test (dataProvider = "DP1")
public void evernoteSharing(String user1, String password1, String email1, String user2, String password2, String email2)
{
 
}

In junit test data is not parameterized using a DataProvider hook. Data-driven testing is implemented differently and involves more coding.

In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific groups instead of running the entire test suite.

Is your code fit for testing?

This article brainstorms how design choices for better code testability fits into the big picture of providing a continuous stream of business value to the customer. Exploring the possible software design options even before the coding starts enables decoupling and dependency injection which makes the software all the more robust and flexible to absorb future changes in the customer requirements.

What is a unit? A unit is the smallest testable part of an application. Unit tests are written from the specification (FSD/TDD) based on how the product would be used by the end customer. It should be tested in isolation without any dependency. All the dependencies should be mocked up using stubs so that the unit tests are simple and fast. These tests are supposed to focus only on a specific action without any coding for the dependencies.

For example, the coder develops a method called validate_zip_code. When I enter  zip code as 00000 it should return false. This is a unit test.  As per the requirement spec, on the shipping form if the customer enters invalid zip code he should be re-directed to the help page that explains what is a zip code.  Imagine that this help page is yet to be developed by a different team called consumer team. Testing both the zip code field and the display of help page is not unit testing but functional/integration testing. This dependency on the help page display needs to be separated from the validation of zip code. This separation is called dependency injection and it decouples the code making it more robust. The dependencies could be mocked using tools like mockito or easymock. Thus the coder needs to create a mockup or a simulator to substitute for the dependency.

Why dependency injection can ease unit tests? With dependency injection writing unit tests becomes easy and the execution of unit tests becomes simple and fast. This is because the coders are not required to write test code for dependencies as well. Writing code for dependencies makes unit tests complicated and it is also unnecessary since we are not going to do any functional testing with it.

Advantages of Dependency Injection:
1. Unit testing before coding adopts Test Driven Development. The coders can brainstorm the possible execution paths and corner cases that he had never thought of before. This results in a higher quality code
2. Code with lots of dependencies violates the Law of Demeter or the Hollywood principle. With dependency injection the code becomes loosely coupled
3. Determines if the code is fit for testing and increases the testability of the code

How can testers influence design choices?

As a Quality Engineer it’s a great experience being able to plan and facilitate design review sessions with developers and architects. This could either be a formal/informal session. The design review provide hints to the testers that help to foresee potential defects in the system. It comes with huge benefits of learning the crucial design elements especially when the discussion happens in a room of developers. At the same time, the developers also get a chance to brainstorm the design when the testers ask questions which inturn adds great business value to the product.
This article explores how fellow test engineers can contribute and influence the design decisions in a development effort. While the developers are better at Object Oriented Programming concepts, the testers can explore the feature at a deeper level identifying corner cases, exceptional conditions or error cases and recommend designs that are flexible enough to accomodate these behaviors.
Issues costing $300K occur due to bad product design that could have been prevented if there were design reviews with testers.
Below are two techniques that are explained indepth to help testers catch design flaws in a product:
1. Apply 5 basic design patterns
2. Analysis models

1. Apply 5 basic design patterns:

Design patterns are named set of problem-solution pairs for the repeated challenges faced by the programmers in the IT industry. Understanding Design Patterns requires us to “think in objects”. Being aware of the design choices and consciously making the best design choice is essentially a programmer’s skill and requires strong knowledge of Object Oriented Programming. Then, how does a tester add value to the design review of a product?
Testers can perceive where the potential issues and risks might be for a particular product design. Usually the unthinkable risks in the product design are not talked about in design reviews. This is the area where a tester can significantly add value. There are 5 basic business design patterns that can get the testers thinking about the unthinkable risks. Some of you might be already thinking on these terms while evaluating the design of a product. Making it more formal with process flow diagrams can make the design review session more useful to the stakeholders.

 

For understanding each of the below business design patterns let us consider a storyboard of a customer purchasing a product on a shopping website.
Before getting into these patterns first let us define what is an event.
An event defines under what circumstances a particular process/function gets triggered.
We can use logical connectors AND / OR called joins for joining two or more business events.

1) Dissection of events

This pattern helps the tester think: “What are the different starting events that can trigger a function/process?”
For example a Purchase Order Processing function can have different starting events as below:
Customer clicks on add to cart button on a third party shopping website
Customer places order via an email sent to his inbox
Customer places order via mobile phone

 

2) Sequential events pattern

What is the specific order in which the events should occur?
Should customers login before clicking on checkout? Can guest users also checkout?

3) Parallel Split pattern

What are the multiple processes that should be triggered by a starting event?
Lets assume that the customer clicks on the checkout button. Normally the purchase order process is triggered. What if the database crashes during the transaction? To handle this, the application should have two processes running. Periodic data migration of the transaction to backup database/servers so that the backup server can get a handle over the transaction in case of any crash and another is the payment processing function.

 

If the product is not designed to handle this crash then there would be a heavy data loss in the midst of the transaction since millions of customers might be using the site. These kinds of unthinkable risk can be better pointed out by a tester using this pattern.

 

4) Synchronization

When there are a set of activities, which of these must be finished completely before we continue the process?
Fraud check and confirmation of funding source/shipping address should be done before payment is processed.

5) Exclusive choice

What are the different mutually exclusive flows possible for a given process?
As stated in the earlier pattern, to which flow should the program transfer the control in the event of server crash?

2. Analysis Models

Analysis models could be used to probe, critique, review and approve the product design. Asking questions, even the stupid ones, during the design phase of the project can add great business value to the product.
Data models are a visual representation of the relationships between the different entities of a system. During the design phase of a project apart from the requirements specification documents there are class diagrams, use cases and entity relationship diagrams that are usually thought of as programmer’s tools. Similarly, the programmers think of it as a business analyst’s tool. Infact it can be used by both testers and developers to ask good questions that adds new perspectives to the design review discussion leading to the discovery of new facts or pitfalls in the product design.
1. Study the domain\business rules between the different entites
2. Know the right questions to ask
3. Identify pitfalls in the product design
4. Check for missing details in the requirements spec and discover new business rules
5. Filter out requirements that are likely to change

Seperation of concerns

Is the design flexible enough to accomodate the likely future change in the product? I just came across a GTAC (2010) video on Flexible Design? Testable Design? You Don’t Have To Choose! where the developer dumps multiple responsibilities in a single large class because he doesn’t like coding several small distinct behaviors each in a seperate class. This will cause issues in future when we need to change/extend/add one more behavior to the class. Whenever we modify this single large class the developer needs to modify so many methods calling them and so he will resist changes. It requires huge efforts even for a trivial change. This is an example of a bad design.
A tester need not agree with this kind of design pattern. The testers should filter out requirements that are likely to change in future using analysis models and ensure that the programmer implements “Seperation of Concerns” principle to make the design flexible for future changes for these set of requirements.