You can download this for free from the App store. Once you install it, you can either use your saucelabs account or try it for free with a few limitations. Currently they support various versions of Internet Explorer, Firefox, Chrome and Opera on Windows, Linux and OSX. They also have options for OSX/iPhone Simulator testing.

The way it works is that once you choose a url to navigate to and the OS/Browser combination, sauce creates a new vm on demand within seconds. On the vm you have access to the browser and the desktop. In the browser you have full keyboard and mouse control so you can perform all actions that are needed to simulate end user testing. Even with the limited access to the desktop, you can do things like change the resolution of the screen.

Video

Before describing it further you might want see it in action:

Free Version limitations

If you are starting with the free version, there are a few limitations that you will encounter. Some of them are

• Access to 15 of the 95 currently supported os/browser combinations
• Session time is limited to 10 mins and after that there is a period of time when you cannot create another vm. You can always find workarounds but I wont get into that.
• You can have only one active session at a time.

As demonstrated in the video, you can create screenshots as well. At the moment though, I was unable to access them later though. I got an error page at saucelabs website instead.

Internal Testing

One question that you might be wondering at this point is how would you be able to achieve internal testing with this tool. For that, first you would have to register with saucelabs and then use something called Sauce Connect to connect to your internal web apps securely.

Hope you find this useful.

My attempt to share “what’s new” in the world of test automation and QA.

— Rahul Poonekar (@QAAutomationNet) March 2, 2012

Wait for page transitions

If you have worked with Selenium 1, you would expect to find a waitForPageToLoad equivalent in WebDriver. You wont find one. WebDriver does a pretty good job waiting for a page transition. In situations that I have run into redirects, WebDriver waited for the page that we were redirected to to load as well. This was something I had to specially handle with Selenium 1.

When we talk about a “page load” we aren’t talking about Ajax loading though. There can be elements on the page that gets loaded using AJAX after the browser is in a “ready state”. If your next interaction with the page required these elements to be present, you would need to handle it with a wait for a condition.

By the way, if you find that the page transition wait’s do not work reliably for you, please share the case by adding a comment below. It is worth noting that WebDriver documentation does say:

“Dependent on several factors, including the OS/Browser combination, WebDriver may or may not wait for the page to load. In some circumstances, WebDriver may return control before the page has finished, or even started, loading. To ensure robustness, you need to wait for the element(s) to exist in the page using Explicit and Implicit Waits.”

Again, I didn’t run into the situation where WebDriver didn’t detect the page transition correctly yet.

A simple test that demonstrates this is:

@Test
public void simpleTest() {
    WebDriver driver = new FirefoxDriver();
    driver.get("http://referencewebapp.qaautomation.net/");
    driver.findElement(By.linkText("Register")).click();

    // Check for body header on new page.<br />
    WebElement bodyHeader = driver.findElement(By.id("body_header"));
    assertEquals("Page body header match failed", "Register", bodyHeader.getText());
    driver.close();
}

Here you can see I didn’t have to specify a wait between lines 5 and 8.

Now what if we want to specify how much the max wait time for a page transition should be? To do this in WebDriver you will have to add the following line:

    driver.manage().timeouts().pageLoadTimeout(90, TimeUnit.SECONDS);

We will look into what driver.manage().timeouts() returns a little later, but the key thing to note here is that there is a mechanism to specify the page load timeout. You could set this anywhere you want but I would imagine for most scenarios you may want to specify this right after initializing the driver.

Wait for a condition

When we have asynchronous processing either on the web page or even in a different system, before you can do your validation or next browser interaction you will need to wait. If you don’t wait for these process(es) to complete, you will see unreliable results.

Let us see an example of such an error. Let us write a test that navigates to http://referencewebapp.qaautomation.net/progressbar.php and validates the progress text of "Process completed!".

@Before
public void initialize() {
    WebDriver driver = new FirefoxDriver();
    driver.get("http://referencewebapp.qaautomation.net/progressbar.php");
}

@Test
public void getProcessTextWithoutWaiting() {
    WebElement progressBar4CommentElement = driver.findElement(By.id("pb4_comment"));
    assertNotNull("Could not find progress bar 4", progressBar4CommentElement);
    assertEquals("Insufficient progress (" + driver.findElement(By.id("pb4_pbText")).getText() + ")",
            "Process completed!", progressBar4CommentElement.getText());
}

You will notice that we don’t get the expected text of “Process completed!” after the page is loaded. Instead, you will see failures with varying progress bar values. On my machine I see:

org.junit.ComparisonFailure: Process running (5/100) 
Expected : Process completed!
Actual   : Process running...

If you look at the DOM of this page, you will notice that there is an element with id=pb4_status that appears after the progress bar is completely loaded. If you were just to add

driver.findElement(By.id("pb4_status"));

in the above test, it would fail with an exception

org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"id","selector":"pb4_status"}

So, in this case, the validation of the process completion message can be fixed if we wait for this progress status element. Broadly you have two choices on how you can implement this wait for condition. Let us look at them:

Implicit Waits

WebDriver has the ability to poll the DOM for element(s). Use of this capability of WebDriver, as opposed to building your own logic to wait for the element(s), is called an Implicit Wait.

The way you add this implicit wait is by instantiating an implementation of the WebDriver.Timeouts interface. This implementation allows you to specify how long web driver should keep looking (poll) for an element whenever you use try retrieving an element using findElement or findElements method in the page before failing. The way you would set this implict wait is by calling the implicitlyWait method.

So using this let us create a new test:

@Test
public void implicitWaitForProcessStatus() {
    driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    WebElement progressBar4CommentElement = driver.findElement(By.id("pb4_comment"));
    assertNotNull("Could not find progress bar 4", progressBar4CommentElement);

    driver.findElement(By.id("pb4_status"));
    assertEquals(progressBar4CommentElement.getText(),
            "Process completed!", progressBar4CommentElement.getText());
}

You will now notice that your test passes! By adding the implicit wait in line 3, WebDriver now does not throw an exception on line 7 and instead polls for the pb4_status element on the page. So once the element exists only then it goes to the next statement where the assertion passes. The maximum time WebDriver polls for this element is 10 seconds, which is specified in the implicitWait call in line 3.

You can always remove this implict wait by resetting the timeout to zero:

driver.manage().timeouts().implicitlyWait(0, TimeUnit.SECONDS);

Explicit Waits

When you build your own logic on how to wait for conditions and you call it between browser interactions, it is called an explicit wait. In this case, since it is your logic, you can build it to do more than just look for an element or elements that match a single criteria. For example you could cater to

• wait for multiple elements on a page that need a different way to look up i.e. different locators
• wait for other types of events on the page, like new dialogs (alerts, confirmation), browser windows etc
• wait for an external system to complete processing etc

This can be done in different ways. Let us look at some options:

Fixed delays

The most common problem that I have seen in tests is when they  have a fixed amount of wait time. An example of how it could be implemented in java is Thread.sleep(milliseconds). If you compare this with waiting for a specific condition, like appearance or disappearance of an element on the page, it is obvious that there could be a lot of wasted time because of all these delays in your tests. I have dedicated a special section at the end of this post with my thoughts on how bad this practice is and why people end up doing this anyways. But for now let’s just move on to options that you would actually want to consider.

Explicit wait using WebDriver

Within the WebDriver API you will find WebDriverWait. There are several very good code designs in WebDriver. WebDriverWait is another example. If you understand how it works, you can appreciate how much flexibility it can provide. Let us see an example of how you can use it.

@Test
public void explicitWaitUsingWebDriver() {
    WebElement progressBar4CommentElement = driver.findElement(By.id("pb4_comment"));
    assertNotNull("Could not find progress bar 4", progressBar4CommentElement);

    Wait<WebDriver> wait = new WebDriverWait(driver, 10);
    // Wait for search to complete
    wait.until(new ExpectedCondition<Boolean>() {
        public Boolean apply(WebDriver webDriver) {
            System.out.println("Searching...");
            return webDriver.findElement(By.id("pb4_status")) != null;
        }
    });

    assertEquals(progressBar4CommentElement.getText(),
            "Process completed!", progressBar4CommentElement.getText());
}

Here we have used WebDriverWait to wait until an ExpectedCondition implementation we created as an anonymous class passes. In this implementation, on line 10, we added our condition – which is to wait for the pb4_status element.

Another class that WebDriver comes with is ExpectedConditions (note the s at the end of the class name). Using this class you can avoid writing your own implementation of ExpectedCondition (no s at the end of this class name). Using the method presenceOfElementLocated of this class the above test would look like this:

@Test
public void explicitWaitUsingExpectedConditions() {
    driver.get(REFERENCE_APP_URL + "progressbar.php");
    WebElement progressBar4CommentElement = driver.findElement(By.id("pb4_comment"));
    assertNotNull("Could not find progress bar 4", progressBar4CommentElement);

    Wait<WebDriver> wait = new WebDriverWait(driver, 10);
    // Wait for search to complete
    wait.until(ExpectedConditions.presenceOfElementLocated(By.id("pb4_status")));

    assertEquals(progressBar4CommentElement.getText(),
            "Process completed!", progressBar4CommentElement.getText());
}

I could write an entire post on how to use the WebDriverWait and what type of conditions we could specify. If this is something that interests you, let me know by adding a comment at the bottom of this post. If I see a lot of interest, I will add it soon.

Let us look at one other option …

Build your wait logic not using WebDriver

There may be some situations when you can’t or don’t want to use use WebDriver API. Some such situations are:

Since Selenium 1 didn’t provide a good way to wait for complex conditions, you might have created your own polling logic … I certainly did. If you think that would still do what you need and you don’t want to take on the risk to porting it to WebDriver’s implementation, you could.

Let us say you want to poll on external systems to complete some asynchronous processing, you may or may not be able to use WebDriverWait (or its parent FluentWait).

FluentWait is not thread safe. If you need something thread safe, you will have to build it yourself.

Some challenges with Waits

I have witnessed multiple times test frameworks being scrapped because they were slow and/ or unreliable. The biggest factor … improper wait logic implementations. The list of mistakes framework developers make with waits is pretty long, but here are some of the most common reasons and ways to fix it:

• Most of the times framework developers are volunteering their time to work on this project or they are under a lot of time pressure and take shortcuts. Several people think adding fixed waits requires a lot lesser work than building a wait for a condition logic. Hopefully with this post you can see it doesn’t have to be much work for majority of the cases. There are several ways to work in such high pressure environments including learning how to build incrementally. The trick here is to never loose focus on speed and reliability of what you build even if what you build might be fewer in capabilities at the beginning.
• There are several conditions you need to wait for which causes some complications in building wait logic … for instance there could be multiple widgets on a page. Couple of the most simplest solutions people should consider are
• you don’t have to wait for all the dynamic elements on the page but only the one that you need to interact with
• you don’t have to only wait for an element to appear, you could also wait for an element to disappear. A simple example is the hourglass or spinners that ajax widgets usually have when they are loading. You could just wait for all spinners to stop being visible on the page.
• It could be that the functionality needs to be wrapped in a more domain specific api/ reusable function and you want to hide the details of what to wait for from the users and it is different for each situation. In some cases waiting for spinners idea mentioned above could solve the problem. The other option is to ask the UI developers to add some hint on the page – for instance a loadcomplete javascript variable on the page which turns to “true” after the dynamic element is loaded.

This topic is pretty vast so if you think there are some key aspects that this post should cover, please send me your feedback below and I will try to incorporate them.

1. « Back : Interacting with page elements
2. Window, frames and Popup dialog handling : Next »

The next action you will most likely want to perform are adding validation points, or assertions, in your test automation solution. The common types of validations are:

• Is the page title as expected
• Validations against an element on the page
• Does text exist on the page
• Does a javascript call return an expected value

Let us look at how to achieve each of these using the WebDriver API.

Page Title

You can get the current page title simply by calling getTitle() on the WebDriver instance. Here is how a simple test would look like:

@Test
public void pageTitle() {
    driver.get("http://referencewebapp.qaautomation.net/");
    String pageTitle = driver.getTitle();
    assertEquals("Current page title", "Reference Web App - QA Automation", pageTitle);
}

Validations against an element

Text within an element

There are two approaches you can use for validating text within an element. The first approach is find the element using one of the locator strategies and then call getText() on the element object returned.

driver.get("http://referencewebapp.qaautomation.net/");
WebElement header = driver.findElement(By.id("header"));
assertEquals("Header text", "Reference Web App", header.getText());

The second approach is to search for an element using the text in the locator and check if it is not null.

driver.get("http://referencewebapp.qaautomation.net/");
WebElement header = driver.findElement(
        By.xpath("//div[@id=\"header\" and normalize-space(.)=\"Reference Web App\"]"));
assertNotNull("Header text", header);

In this example I used xpath but you could use other locators like linkText or partialLinkText.

Attributes of the element

This is similar to validating text in the element. You would find the element on the page and then use getAttribute

driver.get("http://referencewebapp.qaautomation.net/");
WebElement submit = driver.findElement(By.name("submit"));
assertEquals("Submit button value", "Login", submit.getAttribute("value"));

Does text exist in the entire page

For those who start with Selenium 2, or who have used Selenium 1.x, you would expect to easily find a doesTextExist method. It will come as a bit of a shock that there isn’t something as obvious in the API.

One option to get the text representation of the entire page is getting the text of the body element using something like

driver.findElement(By.tagName("body")).getText()

Unfortunately, getting text using this mechanism is extremely slow for large pages. The second option is to get the html content using driver.getPageSource() and then pass it to a html parser. One such a parser is JSoup.

• easy to use,
• is better performant for this sort of thing, and
• the output text is cleaner and easier do validations against

. Here is how you would use it:

    public String getPageContent() {
        Document doc = Jsoup.parse(driver.getPageSource());
        return doc.text();
    }

Just to compare the two approaches, getting document text in msn.com took over 5 secs using first approach and less than a second for the second.

Ofcourse for this code to work, you will have to download the jsoup library and put it in your classpath. Also if you are not developing in java you will have to find a different parser because jsoup is a java library that hasn’t been ported to other languages.

Validating Javascript return values

There are various reasons why you might want to directly call a javascript method or get the value of a javascript variable on the page. You can do this using only those WebDriver instances that implement the JavascriptExecutor interface. The JavascriptExecutor interface has a executeScript method that you would pass your javascript call. If you expect a return value, your javascript snippet must have a return.

Let’s look at an example:

driver.get(REFERENCE_APP_URL + "jsmethods.php");
String returnValue = (String) ((JavascriptExecutor) driver).executeScript("return getString();");
assertEquals("JS method with string return type", "foo", returnValue);

The return type for executeScript method is Object. That is because this script would return an object of different types based on what is returned. The various types of objects returned are Sting, Integer, Boolean, etc. It also can return a WebElement as we saw in the Locating page elements using WebDriver under DOM locator.

Besides these types of validations, there are certainly others you can have, like if a new window was opened, an alert dialog popped up, a cookie was set etc. We will cover those in later sections of the Selenium 2 User Guide.

1. « Back : Locating page elements using WebDriver
2. Interacting with page elements : Next »

Once we get a handle to the element, we can perform

• Keyboard actions like type, multi select etc and
• Mouse actions like clicking, dragging-dropping, left click etc

Here are some of the common actions your framework/ tests would have to perform.

Clicking Elements

The WebElement interface has a click() method. So if you would want to click an element, you would have to use findElement or findElements to find the element and call the click() method.

Let us look at this example:

driver.get("http://referencewebapp.qaautomation.net/");

WebElement registerLink = driver.findElement(By.linkText("Register"));
registerLink.click();

Here in line 3 we got the handle to the anchor page element by looking it up using the link text. On the WebElement object we then called click() which allows us to navigate to the Registeration page. You will notice that in this case selenium handled waiting for the page transition. This is different from Selenium 1′s API where you had to explicitly call waitForPageToLoad.

As per Selenium 2′s documentation, there are cases when the click method would not wait for the page transition. Also, if there are a lot of asynchronous calls on the new page, most of WebDriver click implementation doesn’t wait for them to complete. It would be upto you to manage the appropriate waiting logic. This is covered in more details in the next section.

Setting form fields

Enter text in input fields

WebElement usernameField = driver.findElement(By.id("username"));
usernameField.sendKeys("rpoonekar");

Selecting value in a Drop-down/ select fields

Drop-down fields use the same method to set values in them. The main difference is that in the sendKeys method you could pass the value or the option text as parameter to the method. For instance if your select field looked like this:

<select name="state">
    ...
    <option value="CA">California</option>
    ...
</select>

you could set the field using either

driver.findElement(By.name("state")).sendKeys("CA");

or

driver.findElement(By.name("state")).sendKeys("California");

Selecting Radio and checkbox fields

With choice elements, once you get a handle to the WebElement, you would need to call the click() method.

driver.findElement(By.id("email")).click();

Form Submit

There are two way you can submit a form. The first option is locating the submit button and clicking it

driver.findElement(By.xpath("//input[@type='submit']")).click();

The second option is to call the submit() method on any field you had a reference to. If this element is within a form, WebDriver will walk up the DOM until it finds the enclosing form and then calls submit on that. If the element isn’t in a form, then the NoSuchElementException will be thrown

WebElement state = driver.findElement(By.name("state"));
state.sendKeys("California");
state.submit();

Of these two approaches, I recommend the first one because you are specifying precisely which element to look for and click and so it will execute faster. You are also inherently asserting if the submit element exists in the form.
I view the second approach more than just a convenience method. The second approach can be useful when you don’t want your test to break if there are certain types of changes in the mechanism to submit the form. For instance, if the submit button is replaced with a link that submits the form. This reduces the maintenance cost of your test(s).

Mouse actions

Drag and Drop

When you need to do mouse actions you could use the Mouse class. There are some common composite actions available in the Actions class. Drag and Drop is one of them. The dragAndDrop method essentially has three actions – click and hold/ mouse down, mouse move and release/ mouse up.

Let us look at this example:

@Test
public void dragAndDrop() {
    driver.get("http://referencewebapp.qaautomation.net/dragndrop.php");

    Actions actions = new Actions(driver);
    WebElement drag = driver.findElement(By.id("draggable"));
    WebElement to = driver.findElement(By.id("droppable"));

    actions.dragAndDrop(drag, to);
    actions.perform();
    assertEquals("Drag-n-Drop didn't work.", "Dropped!", to.getText());
}

As you can see, the dragAndDrop methods takes two arguments, the element that you want to drag, and the element you want to drop it on.

One of the key differences in the way Mouse is used and Actions is used is that when you call the mouse methods, it gets executed immediately. With Actions, you build the different steps and they all get executed when you call perform().

1. « Back : Common Assertions using WebDriver
2. Waiting after web page interactions : Next »

Locators have changed with Selenium 2 with the WebDriver API. Out of the box WebDriver supports several ways to locate elements on the page using the By abstract class.

Finding elements by ID

ids are the preferred way to locate elements on a page for 2 main reasons:

• According to w3c ids are supposed to be unique on an html page. This makes ids a very explicit and reliable way to locate elements on the page.
• Also, all browsers also have highly efficient methods to get an object on the page using their ids. This makes id locators the fastest type of locator in Selenium.

Let us now look at how to use id locators using this html code as an example:

<form name="loginForm">
    Login Username: <input id="username" name="login" type="text" />
    Password: <input id="password" name="password" type="password" />
    <input name="login" type="submit" value="Login" />
</form>

In the above code, the username and password text fields are can be set using their ids. The locators for them would be

driver.findElement(By.id(username));

Even though this is a great locator, it is not realistic for all elements on a page to have ids. The developers add ids to key elements on the page to better control the look and feel or provide the dynamic user interaction. However, ids should also be added to elements that are frequently interacted within tests … to make the pages more testable. Automated test script authors should consider adding, or requesting addition of, ids to these key elements on the page.

In some cases, the ids on an element cannot be reliably used in a test. For instance, if you are displaying objects stored in the database, the objects ids could contain the database id in it, for instance book1347 could be the id for book who’s database id is 1347. In such situations it may not be possible to predict and use the ids and so you may need to use other ways described below to find elements.

Finding elements by name

Generally ids are added to elements when they want to be referenced from css or javascript and names are added to form fields. When referencing element from javascript either can be used. From a test automation standpoint, whenever id is not available/ usable, you should try to use the name instead.

Using the same example above, the way you would find the submit button would be:

driver.findElement(By.name("login"));

There is one big difference between the id and name attributes though … name attributes don’t have to be unique in a page. If there are multiple elements with the same name, then the first element in the page is selected. So, in this example, if another button or form named “login” was present of added later, it could cause the test to fail.

Finding links by text

This locator identifies links by the text in them. Let us look at an example:

 <html>
 <body>
    ...
    <a href="signin.html">Sign In</a> to modify your account details.
    ...
</body>
</html>

To click this hyperlink using the anchor tag’s text, you can use the By.linkText() locator:

driver.findElement(By.linkText("Sign In"));

If there are multiple elements with text “Sign In”, the first one is selected.

Btw, this is called linkText because it is used for hyperlinks. In Selenium if you used the link=textPattern locator, you could use it to locate other elements like div, span, td etc. In WebDriver, this locator works only for links.

Another common case is when we need to find links by a portion of the text it contains. In such cases you can find it by specifying the partial text. For example:

driver.findElement(By.partialLinkText("Sign"));

Finding elements by XPath

XPath is a very powerful language to express which element to identify. If you use it correctly, it can produce very reliable and low maintenance locators, but if you use it incorrectly, it can create very brittle test cases.

Let us see some examples:

 <table name="cart">
    <tr id="item1">
        <td class="name item">Mp3 Download: foobar.mp3</td>
        <td class="name item"><input name="qty" class="name item formfield disabled" /></td>
    </tr>
    <tr id="item2">
        <td class="name item">Mp3 Player</td>
        <td class="name item"><input id="item2_quantity" name="qty" class="name item formfield required" type="text" /></td>
    </tr>
    ...
</table>

You can target the highlighted input field using the following XPath expressions:


//table/tr[2]/td/input
//input[@id='item2_quantity']
(//table[@name='cart']//input)[2]
//input[contains(@class,'required')]
//input[contains(@class,'required') and type='text']


And for the last option, your java call would look like:
driver.findElement(By.xpath("//input[contains(@class='required') and type='text']"));

As you can guess, some of these expressions will not be as reliable as others. Of these //table/tr[2]/td/input is the worst because it would break even with slightest modification to the page structure. It can take some time to learn XPath if you aren’t familiar with it, but it is worth the time if you plan to spend a lot of time writing UI automated tests. In any case do not rely on tools, including selenium IDE, to generate the right xpath expression for you. They can help you get started but they are usually bad at identifying the more reliable XPaths.

XPath namespaces

There are some special cases you should be aware of when you work with XPath, like when you are trying to interact with SVG. Like in the example here, the html looks something like this:

<div id="svgchart">
  ...
    <svg xmlns="http://www.w3.org/2000/svg">
        <g>
            <path .../>
        </g>
        ...
    </svg>
  ...
</div>

Here, if you use XPath like:

//svg

you will get ERROR org.openqa.selenium.NoSuchElementException: Unable to locate element. This is because the svg element is in a different namespace. You will have to specify your xpath with the namespace uri like this instead:

//*[local-name()='svg' and namespace-uri()='http://www.w3.org/2000/svg']


XPath performance

So, if XPath’s are so versatile, why doesn’t everyone prefer these? It’s because they are often the slowest, especially in older versions of IE! There are some ways you can make XPath’s faster, but they still are a few times slower than ids or names.

Finding elements by CSS

css locators can be used to identify a large number of elements on a page. Let us look at this html snippet for instance:

 <table name="cart">
    <tr id="item1">
        <td class="label">Mp3 Download: foobar.mp3</td>
        <td class="item"><input name="qty" class="formfield disabled" /></td>
    </tr>
    <tr id="item2">
        <td class="label">Mp3 Player</td>
        <td class="item"><input id="item2_quantity" name="qty" class="formfield required" type="text" /></td>
    </tr>
    ...
</table>

In this case the css locator variations possible for the highlighted input field are:


input.required
input[class~='required']
input.required[type='text']
#item2_quantity
input#item2_quantity


The css locator may not be as expressive as XPath, but it generally executes faster.

Finding elements by class

This is more of a convinience mechanism to identify elements. In the css example above, instead of using

driver.findElements(By.css("input[class~='required']"));

you could use

driver.findElements(By.class("required"));

Getting elements by DOM

DOM stands for Document Object Model. DOM is convention for representing objects in HTML documents.

<form id="loginForm">
    Login Username: <input id="username" name="username" type="text" />
    Password: <input name="password" type="password" />
    <input name="login" type="submit" value="Log in" />
</form>

In this page, the dom expression for the highlighted input field would be:


document.forms[0].elements[0]
document.forms['loginForm'].elements['username']
document.forms['loginForm'].username
document.getElementById('username')


For those who have used Selenium 1 API, you would expect to find a By.dom() equivalent api, but it doesn’t exist. However, you still can get a handle to these elements using dom expressions by using the following code snippet instead:

...
    driver.findElement(byDom("document.forms[0].elements[0]"));
...

public By byDom(String domExpression) {
    final Object o = ((JavascriptExecutor) driver).executeScript("return " + domExpression + ";");

    if (o instanceof WebElement) {
        return new By() {
            @Override
            public List<WebElement> findElements(SearchContext searchContext) {
                return new ArrayList<WebElement>() {
                    {
                        add((WebElement) o);
                    }
                };
            }
        };
    }
}

The three key aspects of this code that you should note,

• The driver can be casted to JavascriptExecutor and other interfaces which will provide additional capabilities.
• executeScript() in JavascriptExecutor returns an object which can be casted to a WebElement if the javascript expression returns a dom element.
• You can create your own implementation of By

Some things to note for Selenium 1 users

• You will notice that Implicit locators are not supported in WebDriver.
• When using name in Selenium 1, you could add other attributes in the expressions to filter the results. For example name=login type=text was valid. You cannot do that any more with the By.name() call.

findElement() and findElements()

As you might have noticed in the examples above, the By locators were used within findElement and findElements methods. The purpose of these methods is to return a WebElement, in case of findElement, and a list of WebElements in the case of findElements.

The WebElement represents an html element on the page. In the next section you will see that it is this WebElement object that you would interact with or read attributes of for validations etc.

When you use these methods on the WebDriver object, the scope within with the elements are located is the entire page.

Getting child elements

The WebElement interface also has findElement() and findElements() methods. In this case, it is within the scope of this parent element that child elements are located. This is useful when you want to narrow the scope to find several elements that you would interact with. In most cases, narrowing the scope improves the execution times for locating the elements.

1. « Back : Getting started with Selenium 2 and WebDriver
2. Common Assertions using WebDriver : Next »

To begin adding Selenium to your project, you will have to first download Selenium for the language you develop your framework on.

The download comes as a zip file so you will first have to unzip it. In this unzipped folder contents, you will see selenium-java-2.x.y.jar and several other dependent jar files in the lib directory. Assuming you have a java development environment with the setup similar to the one defined in the Creating your Java project workspace post, you will have to copy these jar files in your lib directory as well. Most people just copy all the jars over, however, I recommend copying just the libraries you need.

Starting and stopping Selenium server

For those who have used earlier versions of Selenium, you are probably familiar with starting and stopping selenium using ant. In Selenium 2, you don’t need to start the server. Infact, the selenium-java-2.x.y.jar cannot be used to start the server in the first place. You need a separate jar to start the selenium 2 server, which is used only when you need to run Selenium 1 APIs on Selenium 2.

Selenium-backed WebDriver

For reasons described in the Selenium 2 introduction, if you plan to use the selenium server you will need to download the selenium-server-2.x.y.zip instead of the selenium-java-2.x.y.jar. You can find the selenium-server-2.x.y.zip on the Selenium project download google page.

Instantiating WebDriver

WebDriver is an interface in the new Selenium 2 API. The different implementations of this WebDriver interface allows programmatically controlling different browsers.

Using the interface you could start a new browser instance, navigate to a url and close the browser using this snippet.

WebDriver driver;
Wait wait;

public void instantiateBrowser(BrowserType browserType, String url) {
    driver = getWebDriver(browserType);
    wait = new WebDriverWait(driver, 30);
    driver.get(url);
}

public void closeBrowser() {
    driver.close();
}

public WebDriver getWebDriver(BrowserType browserType) {
    switch (browserType) {
        case FIREFOX:
            // Need to implement
            return null;
        case IE:
            // Need to implement
            return null;
        case CHROME:
            // Need to implement
            return null;
        case HTMLUNIT:
            // Need to implement
            return null;
        default:
            throw new RuntimeException("Browser type unsupported");
    }
}

public enum BrowserType {
    FIREFOX, IE, CHROME, HTMLUNIT
}

The following sections shows how you can instantiate and configure each browser type:

Firefox

In Firefox, it is recommended to create a firefox profile for test automation. I have an entire post dedicated to describing the steps to create a firefox profile.

Once you have created this profile, you can instantiate a new firefox session using this code snippet.

File profileDirectory = new File(profileDirectory);
FirefoxProfile profile = new FirefoxProfile(profileDirectory);
return new FirefoxDriver(profile);

Here profileDirectory is the path to the directory where the profile is created.

Internet Explorer

With IE the first challenge I encountered was with security. The error you would get is:

org.openqa.selenium.WebDriverException: Unexpected error launching Internet Explorer. Protected Mode must be set to the same value (enabled or disabled) for all zones.

There are two ways to solve this:

The first option is that in your Internet Explorer browser set the protected mode enabled or disabled for all zones (Internet, Local Intranet, Trusted Sites and Restricted sites). The problem with this option is that if you are testing a system which does not work with this security setting or you do not have control over the machines where your test code will execute, then you cannot use this approach.

The second option to get around it is by using the INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS capability. You can set this capability by using the following snippet:

DesiredCapabilities ieCapabilities = DesiredCapabilities.internetExplorer();
ieCapabilities.setCapability(
        InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, true);
return new InternetExplorerDriver(ieCapabilities);

The problem with this option is that there is a potential that you might get some flakiness/ unreliability while performing certain actions in IE. I haven’t encountered this flakiness yet though.

Chrome

With Chrome, you will have to download the chrome driver first. If you don’t you will see the error:

java.lang.IllegalStateException: The path to the chromedriver executable must be set by the webdriver.chrome.driver system property; for more information, see http://code.google.com/p/selenium/wiki/ChromeDriver. The latest version can be downloaded from http://code.google.com/p/chromium/downloads/list

I recommend downloading all three binaries maintained by the Chromium project i.e. for windows, linux and mac and renaming them to chromedriver-win.exe, chromedriver-lin and chromedriver-mac. This way you can refer to them on the appropriate OS using the following code.

DesiredCapabilities chromeCapabilities = DesiredCapabilities.chrome();

String chromeBinary = System.getProperty(" ");
if (chromeBinary == null || chromeBinary.equals("")) {
    String os = System.getProperty("os.name").toLowerCase().substring(0, 3);
    chromeBinary = "lib/chromedriver-" + os + (os.equals("win") ? ".exe" : "");
    System.setProperty("webdriver.chrome.driver", chromeBinary);
}

return new ChromeDriver(chromeCapabilities);

One of the things that threw me off initially when I was looking for the binaries of the chrome driver was that I saw a linux 32 and 64 bit binary, but I only saw a win32 bit binary. I wondered if that meant that win64 was not supported. However, when I used the win32 binary on my windows 64bit machine, it worked.

One other thing to keep in mind is that if you do not have Chrome installed in the default location, you could also tell WebDriver where to look for it by adding

chromeCapabilities.setCapability("chrome.binary", "/path/to/where/chrome/is/installed/chrome.exe");

before creating a new instance of the ChromeDriver.

1. « Back : What is new in Selenium 2
2. Locating page elements : Next »

I have used Selenium for several years, and continue to use it, at work. I am a fan and so I decided to contribute to it’s adoption by writing some documentation on Selenium for everyone’s benefit. My hope is that because I am a user of the tool and I have experienced problems that real users face, this guide will capture the essentials of how to use the tool.

A lot of people look for Selenium tutorials and this guide is written in a manner to help newbies learn how to develop tests in Selenium. This guide also contains code examples that should help both novice and experienced users.

The code examples are currently in Java. I am sure it will still give you the key information you need to allow you to develop in the programming language of your choice, but, if you have snippets in other languages that you think would be useful, please feel free to share them.

Table of Contents

1. Introduction to Selenium
2. What is new in Selenium 2
3. Getting started with Selenium 2 and WebDriver
4. Locating page elements
5. Common Assertions using WebDriver
6. Interacting with page elements
7. Waiting after web page interactions
8. Window, frames and Popup dialog handling
9. Mobile automation

Feedback on these topics would be much appreciated!

• Introduction to Selenium : Next »

Installation Steps

If you are not on Snow Leopard, you will have to upgrade first. Once you are on Snow Leopard and have all the latest updates installed, open App Store application and buy the Mac OSx Lion. For $29.99, I consider this a very good deal!

The installer is about 3.74GB in size and it took me 35 mins to download it. Once it is downloaded and you click the Install button, you will see this screen.

Then click Continue.

Click Agree. You will be prompted for your mac user password. Enter it.

Click Install and first the installer will start preparing for the install.

Once it’s complete you will have to restart the OS to begin the installation.

Once you click Restart, it will close any open applications.

Once it restarts, the installation automatically starts, which takes about 30 mins.

Once the installation is complete, you will greeted at first by the Setup Assistant.

Here you will have to move your mouse over the text that describes how scrolling works and using two fingers scroll down to a button that  says something like continue using Lion, which when you click will finish the Setup Assistant.

Post Installation thoughts

As a software developer I first checked if all my developer apps still worked. When I started IntelliJ, I was prompted to install java. Once I did that, IntelliJ worked just fine.

I also use NX Client quite a bit at work and the current GA version is not supported in Lion anymore. Nomachine released a preview version that is already working for me on Lion though. So, I am all set.

As for the Lion look and feel, Mission control and the ability to create and remove desktops are the most useful to me. I have several apps running at any given point of time and now it does get a lot more convenient to organize them in a way that I can get to them quickly.

The scroll direction is opposite though and I don’t like it. It was easy to revert that by going to System Preferences > Trackpad > Scroll & Zoom and disable the “Scroll direction: natural”.

Finally, it was time for me to check if my test automation tools (what this site really focuses on) continued to work on Lion.  As you might remember Selenium 1.0.1 stopped working after Snow Leopard was released. It was eventually fixed with the release of Selenium 1.0.3. I was glad to see that Selenium 2.3.0 did have any problems with Lion.

Over time I found that building tests in a certain way made the tests more reliable. I also started discussing with others what they did to keep their test suite robust to validate what I had learnt the hard way. Unfortunately searching for test automation principles on the internet wasn’t as useful as I hoped. Some of those principles, like “minimize untestable code” seemed like it was a product development principle as opposed to a test automation principle. Some others like “verify one condition per test” are good for unit testing but impractical for functional/ system tests. I also found some key principles missing in some of those lists. So, I am taking a stab here at describing what I think should be the Principles of Test Automation.

Choose the right type of test automation methodology

I broadly categorize automated testing as Unit, Component and System testing.

Unit testing

Unit testing is validation of the smallest part of the application. This smallest part in procedural languages is a function and in object oriented languages is a method in a class.

Since the focus of the unit test is very small, these tests can cover the entire code base, even if it’s not exercised by user flows. Also, they need the least amount of setup and teardown, so fewer things can go wrong … making them the least brittle. A combination of fewest assertions and least setup and teardown, make them the fastest as well. For these reasons, you should strive to have as much coverage as possible using unit tests.

There are several unit testing frameworks. In Java for instance, the commonly used ones are JUnit and TestNG. What causes confusion sometimes is that people use these frameworks to write tests that test a lot more than a function or a method. For example you could write a UI test using JUnit. That does not mean this is a unit test though. What advantages I described earlier about unit testing pertains to those tests that validate whether a function or method works correctly.

Component testing

When there are several components in the system, testing each component in isolation is called component testing. Most people are aware of the design practice of separating the Model View and Controller in the system. If you were to test the entire system integrated together, you would not be able to test capabilities in the Controller, for instance, that the Model and View weren’t utilizing. So, to test the Controller thoroughly, you would have to write component tests without Model and View.

The problem with writing tests against a component is that you need some system to provide the input to the component and/ or consume and validate the output of the component. In the real world, that would be what other dependent components would do. But for component testing, you have to add mocks or simulators for these dependent components that would mimic their expected behavior. The other big benefit of this is that it removes the possibility of failures due to bugs in those components.

As you can tell, component testing gives you more coverage of various execution paths than functional testing but not as much statement level coverage as unit testing.

System or Functional testing

The purpose of System testing, also called as Functional testing, is that it tests the entire system interacting with each other from an end user’s perspective.

When designing System tests and frameworks to execute system tests, you should be thinking black box testing. If the end user never interacts with the database the application uses for instance, ideally the system tests should not be validating data in the database. It should validate what the users see. This prevents the possibility of a false positive when the internals of the application is undergoing change even when the end user shouldn’t notice a change.

When running reliably, these tests tend to have the highest ROI. Especially when you are running the use cases that bring the highest value to the end user or generates revenue for the application/ service. The challenge ofcourse is to keep Functional tests reliable is the tricky part. Besides being brittle, they are slow making it hard to scale these tests. Also, they can cover only the code that is exposed to the end user. For these reason it’s best to automate only the high value flows and understand what the point of diminishing returns is.

Keep the tests short

Most of the times people write a unit test that tests one function. But frequently people forget that a unit test should contain a single assertion. They add multiple assertions for a single function, each time passing different input data combinations to the function under test. In this case when any assertion fails, it does not run the rest of the assertions and so the results don’t provide the granularity into all possible failures that exist.

For the same reason, keep the number of assertions as less as possible in component and system tests as well. Also, the longer the test, the more brittle it gets. The main reason why people tend to write long tests is because they want to cover an entire use case. Also they think while we are at this step, we can validate a few other things you normally would with manual testing. Don’t fall for these temptations and break your scripts into multiple tests with as few steps as possible in each of them.

Keep tests independent

No test should be dependent on another.

This isn’t very hard to follow but is one of the most common mistakes people make. When a use case has several steps, often people write every step as a separate test and then make the tests dependent on each other. The problem with this approach is ofcourse that you need to cannot run any one test in isolation, or you can’t easily choose a category of tests based on which area of the product is changing. Also, you need to track dependencies between them. One of the ways, I have seen people achieve time and time again, is by simply naming them in alphabetical order. This then leads to several problems when the workflow changes and the tests have to be reworked, or if your tests scale and you realize you can’t easily run the tests concurrently.

The right solution is to ensure that each test is independent and validates one step. Several times you can avoid the previous steps in the workflow by setting up the dependent data in the database baseline. When that is not possible, you can make them part of the setup of the test.

If you make it part of the setup, I prefer that the reports don’t show them as test failures. If I can’t show them as “not run” for any reason, I prefer to not have them in the report at all. This way you can avoid analysis paralysis and steps that really have failures are the only ones that are investigated.

Tests should be idempotent

If the tests can be executed over and over again and with the same results, they are considered idempotent. If the tests aren’t designed to be idempotent, then you have to take a lot of measures to ensure they are executed just once and then the test environment is rebuilt. This makes it cumbersome to reproduce failures while debugging and can lead to undesired inefficiency and/or inflexibility in the way tests are executed.

Tests should be deterministic

Having indeterminacy in tests undermines any test automation effort because no body trusts the results. One of the easy ways to tell if a test is indeterminant is if it fails because of a false positive. A false positive is when a test case fails not because of a bug in the System Under Test (SUT) or Application Under Test (AUT) but because of a problem in the test or the framework/ tools it is built upon.

What most people don’t realize is that tests that never fail dues to false positives can also be indeterminant. Imagine the AUT is an application that allows you to make purchases using credit cards or reward points. If the tests were written in such a way that you would use the reward points on a test credit card to make a purchase if you had enough points and use credit if not, the test wouldn’t fail because one or the other scenario would work. But, you wouldn’t know for sure if both the code paths were covered. If you had a million points before running the suite, it could be that every test ran using the rewards code path and it would fail to catch potential bugs in the credit code path. A determinant test on the other hand would always test one code path or the other no matter how many times you ran the test.

Minimize incidental test coverage

Incidental test coverage means that you are exercising code of the SUT that you aren’t intending to test. This happens when you essentially cover code by performing necessary steps to reach the validation point (assertion) that you really want to test. If you use code coverage tools, this gives you a false feeling of confidence that you have good coverage. In such situations, you should add tests for those areas of your product that you are covering as part of setup or teardown steps.