• Jobs
  • About
  • Making Functional Automation successful in your team February 15, 2010

    Who should read this

    Here the focus is on describing how to build a foundation for Functional Test Automation in a software company. This article is for you if you have never tried test automation before, or even if you have had very little luck with it in the past. Most websites talk a lot about technological choices for automation tooling but there is a lot more to automation to make it successful. In this, and a series of follow on articles, I will focus on all the aspects of making an automation initiative successful.
    By the way, if you have no experience with automation and want to know how to get started, you might find the career in Test Automation useful as well.

    What you need to know before you begin


    Lets first start with the very obvious – what is automation’s purpose? In the most simplest sense, it is used for improving product quality and employee productivity. Higher product quality leads to higher referencible customers and lower product support costs. Higher employee productivity leads shorter time to market, lower cost to doing business and higher employee retention.


    Automation does not come cheap. Yes you can avoid the obvious license costs of automation tools in several cases, but that’s not the only cost we are talking about here. The time it takes to hire the right talent, time to build the automation solution, the impact to business processes, the cost of adoption are all real costs to be considered.
    Building an automation solution for your team(s) is not a particularly daunting task, however, I have seen too many cases where companies were unsuccessful. In some cases to the point where they completely gave up and fell back to manual testing. I very purposefully chose the word automation solution. This is not just a technology choice of a tool to be used for automation. Making automation successful requires a change in all engineering departments and at several levels in the organization. While scoping a project, automation has to be accounted for, while development testability has to be considered, while testing there needs to be a concrete plan around what should be manually tested and what should be automated. Now, in some departments it might be a small change and some it might large, but the point is that this initiative should not be considered as something that a small team, within QA or Dev, can solve by their own.

    Bottom line

    There needs to be a constant focus on ROI (Return On Investment).

    Where to begin

    Here are first set of things you should be thinking about:

    Understand current hurdles

    Before you can build a solution, you need to understand what the current software development lifecycle is, what testing approaches are employed, what the execution challenges are for each approach, what solutions were proposed in the past, with them what worked and what didn’t. Often this information cannot be collected by talking to one person. Its best to interview a few people from every engineering department.

    Team structure and hiring

    In some organizations it makes sense to have the individual teams build their own automation tooling, but in most cases it makes sense to have a single team that focuses on building the automation tooling that is used by other QA team(s). There are many factors that can influence this decision. Some of them are, do the different teams in the organization have very different tooling needs, or are they very similar (which is when you would want to avoid duplication of effort); do the teams have the technical know-how or the experience they need to be successful with automation themselves; do you have to make do with the resources you already have or can you afford hiring. Assuming you are already inclined to create a separate team to drive the success of this initiative, the next decision you would have to make is whether to have this team should report into the QA org or under Development. There are pros and cons to both.

    Generate requirements

    Once you understand the hurdles and understand the logistics around who will be developing and using the solution, you can start generating requirements your tooling solution will eventually need to meet. Here are some things to consider:

    • What interfaces does your tooling need to interface with. For instance, do you need to drive a browser, send SOAP/ REST requests and validate its responses, validate email notifications, do database validations, do document or image validations etc.
    • Are there any tooling or even just libraries that either QA or Dev created that can be used.
    • Is the primary focus automating new functionality tests (which is especially a challenge with agile development) or catching up with regression tests (which usually happens when there are several years of development with very little focus on automating).
    • Which systems, like the test case tracking system, that the automated tests need to be integrated with.
    • When and how do these automated tests need to be triggered.
    • Who need to contribute to the test scripts development? QA, Dev or both?

    Setting goals

    Don’t make the mistake of trying to build an elaborate solution that attempts to achieve all requirements at once. It’s necessary to understand what you eventually want to achieve through automation, but to get there, plan to have your Test automation development as an iterative process. Try to first figure out solving which problem would give you the biggest bang for your buck. After that you can always build upon your successes. Almost always building the Build Acceptance tests and automatically running them with, either every check-in, or every code deployment on a QA system, ends up being a good choice as the first goal.

    Technology choices

    It’s finally time to talk about implementation and tools. Often people want to talk about tooling when they want to start automating but only once you understand what your goals are and what resources you have to achieve them, the technology choices become a lot simpler. For instance, if you intend to use APIs created by the development team or if they are directly contributing to the solution, then often the programming/ scripting language choice needs to be something developers are comfortable with. If you are working in a java shop, you can’t expect the developers to create VBScript libraries to be used in QTP. Several tools these days, like Selenium and FitNesse, support multiple language choices.

    Once you narrow down on the tools that fit the requirements, the next steps are to start evaluating and prototyping them using a few simple test cases. On this website I intend to put several posts around how to get started with several of the popular tools. This should help you with your evaluation.

    In closing

    The items I mentioned above deserve an entire post each! I plan to add them in the near future so stay posted. Also, after making your technology choices, you would have just completed the “getting started” phase of automation. There are other challenges you will face when you try to scale your solution beyond, say, your automated build acceptance tests. So I will cover them in future posts as well.

    If you have any comments or suggestions about this post, do feel free to send them my way. I am always very happy to learn from others experiences and consolidate them here for everyone’s benefit.

    Further Reading

    Posted by Rahul Poonekar in : Concepts

    Leave a Reply

    Your email address will not be published. Required fields are marked *