30-feet view of test automation framework with selenium

Is selenium the future of test automation? That leads us to yet another important question, what was the past of test automation? Well…,I was just kidding. There are numerous amazing test automation tools out there but none of them could takeover the field of test automation. Why? It’s because, when you take a closer look at any testing project, the complexity increases so much that it takes a real tester with highly sophisticated testing practices and coding skills more than a mere tool to handle those situations. Yes, any tool no matter how good it is, cannot simply provide the foundation for a sound test automation framework. It is we, the “testers”, who can do it! But how?

This article shows step-by-step how selenium with a hybrid of other tools and practices could be used by the tester to develop a highly maintainable, robust, sophisticated and sound test automation framework.

 

This article gives a 30-feet overview of how a test automation framework with selenium looks like.

 

The little secrets of test automation:

Would you be interested to know how I can automate tests in just six weeks while others usually take over six months? I automate far faster and much cheaper than anyone else can do! Come on, my little secrets are so little.

Often in a testing project we tend to handle things in our own way that we see fit. The delicate test automation framework gets upset due to these gimmicks. Let me introduce you to the few main things that really counts during test automation:

  1. Customer focus
  2. Easy-to-understand test case listing the user actions
  3. Decoupling of test data from test execution
  4. Intelligent exception handling
  5. Data-driven testing framework
  6. Modular testing framework
  7. Separation of test verification from test script

Even if you are new to automation you can easily get a hang of it as you follow the steps outlined. Let me illustrate these principles with a simple demo.

Pre-requisites:

To execute the practical demonstration in this article we require the installation of JDK, Selenium Server, Selenium client driver for java, ant, TestNG, eclipse and TestNG plugin for eclipse.

For installation/setting up of Selenium, please refer the article “Selenium RC configuration for java – all in a nutshell”.

In this demo I am using Ubuntu as platform. You could use your favorite platform for coding. So, let us walkthrough the steps for developing a robust and highly maintainable test automation framework with selenium.

 

Step 1: Create java project and add references to external jars:

Suppose we want to dress perfectly for the weather with a rain hat, a rain coat and an umbrella. Now let’s develop a selenium test automation framework to test a very simple application that I have created called Weather Monster - “http://weathermonster.heroku.com/”.

  1. Open eclipse → Create a new java project called weatherMonster
  2. Click on Next button → click on the Libraries tab
  3. Click on Add External JARs button and add the respective jar files for selenium server, selenium java client driver and TestNG as shown in the diagram below → Click Finish

 

 

Step 2: Automatic configuration of test environment

A configuration script contains the key value pair needed to setup the test environment. These values could be changed dynamically using a shell script.

Let’s create a shell script to dynamically start the selenium server.

Create a new folder called “config” in the weatherMonster project and place the below shell script named config.sh in it.

#!/bin/bash
cd ~/Documents/working/installable/selenium/jar\ files/
gnome-terminal --title="config" -x bash -c "java -jar selenium-server-standalone-2.0rc2.jar"

This shell script navigates to the folder path containing the selenium server jar file (Replace this path with respective path on your machine). It then opens a new GNOME-TERMINAL named config and runs the selenium server by execution of the command “java -jar selenium-server-standalone-2.0rc2.jar”

To grant execution privileges to the script use chmod command as below:

chmod 777 config.sh

Then run the script using the below command:

./config.sh

We need to run this script each time before test suite execution to start the selenium server.

Right click the src folder and create a package named config. Right click the config package and create a java class called seleniumTest.java

Let’s assign values to the environment variables such as host, port, browser and so on.
Since we do not want to hard code the test data and we would like to decouple test data from test execution let’s assign the test data to the variables separately in the config folder and organize them as shown below:

package config;
 
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.HttpCommandProcessor;
 
public class seleniumTest
{
	/*Environment variables*/
	public final String HOST = "localhost";
	public final int PORT = 4444;
	public final String BROWSER = "firefox";
	public final String BASE_URL = "http://weathermonster.heroku.com";
	//To set the baseURL and server port
	public final HttpCommandProcessor PROC = new HttpCommandProcessor(HOST,PORT, BROWSER, BASE_URL);
	//To create a selenium object for passing as variable
	public final DefaultSelenium Selenium = new DefaultSelenium(PROC);
 
	/*Timeout*/
	public final String TIMEOUT="60000";
 
	/*Weather Monster home page*/
	public final String HOME_MSG = "This is weathermonster, bringing you the weather";
	public final String HOME_PAGETITLE = "Weather Monster";
}

 

Step 3: Modularization of commonly used methods

A module is a small independent script that represents AUT(Application Under Test). The most commonly used methods in the test case should be created as modules in separate folders that can be called and reused by many test cases. This increases the maintainability of the test suite. If there is a change, it needs to be applied only in one single place as opposed to all the places thus saving lots of time and cost.

Modularization also makes the test case easy-to-read and understand.

Consider the below test automation code. Can you understand it? It is a technical stack containing all the configuration details which is very hard to understand.

 

The test case should list the user actions not the technical details. This makes the test case customer-focused and easy-to-read. Now, let’s move out all the technical stack from the test case to a separate module.

Right click the src folder and create a package named module. Right click the module package and create a java class called testInit.java.

Here we code all the technical details of starting selenium server for test initialization so that the actual test case is freed from these unnecessary details and can cover the main area to test.


package module;
 
import com.thoughtworks.selenium.*;
import org.testng.annotations.*;
import config.*;
 
public class testInit extends SeleneseTestBase
{
	//Declare a new selenium test
	seleniumTest test = new seleniumTest();
 
	@Test
	public void testSetup(DefaultSelenium selenium)
	{
		try
		{
			//Open, maximize and focus on the facebook page. Delete all cookies
			selenium.start();
			selenium.open("/");
			selenium.waitForPageToLoad(test.TIMEOUT); //Handles timeout exception
			selenium.windowMaximize();
			selenium.windowFocus();
			selenium.deleteAllVisibleCookies(); //Clear all session cookies before testing
		}
		catch (Exception e)
		{
			 e.printStackTrace();
		}
	}
}


Step 4: Writing a selenium test case

Now that we already have modules for test setup and initialization, the writing of a selenium test case doesn’t take much effort. All we need to do is just call these modules and reuse them in our test case. This allows us to focus more on the customer.

A good automation script should contain the actual code logic to perform and validate a business scenario.

Right click the src folder and create a package named testSuite. Right click the testSuite package and create a java class called weatherMonsterHome.java


package testSuite;
 
/* Test case for weather monster home page*/
 
import module.*;
import config.*;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
 
public class weatherMonsterHome extends SeleneseTestBase
{
	//Declare a new selenium test
	seleniumTest test = new seleniumTest();
	testInit setUp = new testInit();
 
	@Test
	public void testWeatherMonster()
	{
			/*Test the home page of Weather Monster
			 * Pass the expected message and page title of the home page as parameters
			 */
			setUp.testSetup(test.Selenium);
			weatherMonster(test.Selenium, test.HOME_MSG,test.HOME_PAGETITLE);
	}
 
	public void weatherMonster(DefaultSelenium Selenium, String strHomeMsg, String strExpectedHomePageTitle)
	{
		try
		{
			/*Verification of Weather Monster home page*/
			assertTrue(Selenium.isTextPresent(strHomeMsg));
			System.out.println("Message on the Weather Monster home page is displayed successfully");
 
			assertEquals(Selenium.getTitle(),strExpectedHomePageTitle);
			System.out.println("Title of Weather Monster home page is correct");
		}
		catch(Exception e)
		{
			e.printStackTrace();
		}
	}
}


This decoupling of test script from all the test execution details has made the test case a very easy-to-read list of user actions.

Ultimately the test structure is as below:

Step 5: Run the test case

  1. Navigate to the config folder and give the below command to start selenium server:
./config.sh
  • Right click on weatherMonsterHome.java and select “Run as TestNG test”.

 

OUTPUT:

RemoteTestNG starting

Message on the Weather Monster home page is displayed successfully

Title of Weather Monster home page is correct

PASSED: testWeatherMonster

 

We have missed out on a few little secrets of test automation like intelligent exception handling and data-driven testing. We shall get introduced to them in our next article, so stay tuned!

Feel free to ask any question or post comments.

junit vs TestNG – core differences

Junit and TestNG differ in the core design. Junit is a unit testing framework while TestNG addresses testing at a higher level. This article discusses the three main differences between TestNG and Junit.

1.     In Junit, one test case failure can cause a bunch of test cases to fail in the test suite. There is no option of skipping the set of dependent test cases. The dependent test cases are also reported as failures. For example, suppose there is a test case to test login and the next 10 test cases need to perform a transaction after login. If the login test case fails the other 10 test cases will also fail.

TestNG handles dependency between test cases. If one test case failure causes the failure of a group of test cases it skips that group and executes the rest of the test suite. The group that has dependency on the failed test cases is reported as skipped NOT failed.

2.     In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific subset of the testsuite say database related test cases instead of running the entire test suite. This can be done as below:

In the test case we define two groups DBTestcase and deprecated as below:

@Test(groups = {"DBTestcase", "deprecated"})
public void testMethod2()
{
 
}

In TestNG.xml config file we write the below piece of code to include only database related test cases and exclude the deprecated ones.

<test name="Sample">
	<groups>
		<run>
			<include name="DBTestcase" />
			<exclude name="deprecated" />
		</run>
	</groups>
	<classes>
		<class name="example1.Test1" />
	</classes>
</test>

In Junit for a long time it was not possible to run a specific subset of the test cases. We can either run the entire suite or run each test case individually. Junit 4.8 introduced a new feature called “Categories” to overcome this limitation. However groups are much easier to configure in TestNG.

3.     TestNG supports parameterization for objects. For example I can execute a single test case for multiple test data sets through the parameterization of DataProvider object. This makes the implementation of data driven testing more flexible in TestNG as below:

@DataProvider(name = "DP1")
public String[][] testData() throws Exception
{
String[][] sdataArray = {{"james","moosecat$", "james@gmail.com","selenigr","selenigr1$", "selenigr@yahoo.co.uk"}};
return sdataArray;
}
 
@Test (dataProvider = "DP1")
public void evernoteSharing(String user1, String password1, String email1, String user2, String password2, String email2)
{
 
}

In junit test data is not parameterized using a DataProvider hook. Data-driven testing is implemented differently and involves more coding.

In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific groups instead of running the entire test suite.