FUNDAS ABOUT TEST AUTOMATION FRAMEWORKS :

Historically, test automation has not met with the level of success that it could. Time and again test automation efforts are born, stumble, and die. Most often this is the result of misconceived perceptions of the effort and resources necessary to implement a successful, long-lasting automation framework. Why is this, we might ask? Well, there are several reasons.

The commercial automation tools have been chiefly marketed for use as solutions for testing an application. They should instead be sought as the cornerstone for an enterprise-wide test automation framework. And, while virtually all of the automation tools contain some scripting language allowing us to get past each tool’s failings, testers have typically neither held the development experience nor received the training necessary to exploit these programming environments

Test automation must be approached as a full-blown software development effort in its own right. Without this, it is most likely destined to failure in the long term.

In order to make the most of our test strategy, we need to make it reusable and manageable. To that end, there are some essential guiding principles we should follow when developing our overall test strategy:

• Test automation is a fulltime effort, not a sideline.
• The test design and the test framework are totally separate entities.
• The test framework should be application-independent.
• The test framework must be easy to expand, maintain, and perpetuate.
• The test strategy/design vocabulary should be framework independent.
• The test strategy/design should remove most testers from the complexities of the test framework.

Data Driven Automation Frameworks

Over the past several years there have been numerous articles done on various approaches to test automation. Anyone who has read a fair, unbiased sampling of these knows that we cannot and must not expect pure capture and replay of test scripts to be successful for the life of a product. We will find nothing but frustration there

1.2.1 Data Driven Scripts

Data driven scripts are those application-specific scripts captured or manually coded in the automation tool’s proprietary language and then modified to accommodate variable data. Variables will be used for key application input fields and program selections allowing the script to drive the application with external data supplied by the calling routine or the shell that invoked the test script.

Variable Data, Hard Coded Component Identification:
These data driven scripts often still contain the hard coded and sometimes very fragile recognition strings for the window components they navigate. When this is the case, the scripts are easily broken when an application change or revision occurs. And when these scripts start breaking, we are not necessarily talking about just a few. We are sometimes talking about a great many, if not all the scripts, for the entire application.

1.2.2 Keyword or Table Driven Test Automation

Nearly everything discussed so far defining our ideal automation framework has been describing the best features of "keyword driven" test automation. Sometimes this is also called "table driven" test automation. It is typically an application-independent automation framework designed to process our tests. These tests are developed as data tables using a keyword vocabulary that is independent of the test automation tool used to execute them. This keyword vocabulary should also be suitable for manual testing, as you will soon see.

Action, Input Data, and Expected Result ALL in One Record:
The data table records contain the keywords that describe the actions we want to perform. They also provide any additional data needed as input to the application, and where appropriate, the benchmark information we use to verify the state of our components and the application in general.

Reusable Code, Error Correction and Synchronization:
Application-independent component functions are developed that accept application-specific variable data. Once these component functions exist, they can be used on each and every application we choose to test with the framework.

1.2.3 Hybrid Test Automation (or, "All of the Above")

The most successful automation frameworks generally accommodate both keyword driven testing as well as data driven scripts. This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture.

The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been. The utilities can also facilitate the gradual and manageable conversion of existing scripts to keyword driven equivalents when and where that appears desirable.

1.2.4 Commercial Keyword Driven Frameworks

Some commercially available keyword driven frameworks are making inroads in the test automation markets. These generally come from 3^rd party companies as a bridge between your application and the automation tools you intend to deploy. They are not out-of-the-box, turnkey automation solutions just as the capture\replay tools are not turnkey solutions.

Keyword Driven Automation Framework Model

The model focuses on implementing a keyword driven automation framework. It does not include any additional features like tracking requirements or providing traceability between automated test results and any other function of the test process. It merely provides a model for a keyword driven execution engine for automated tests.

The commercially available frameworks generally have many more features and much broader scope. Of course, they also have the price tag to reflect this.

I) Project Guidelines

The test strategy will allow each test to include the step to perform, the input data to use, and the expected result all together in one line or record of the input source.

Implement a framework that will integrate keyword driven testing and traditional scripts, allowing both to benefit from the implementation.

Implement the framework to be completely application-independent since it will need to test at least 4 or 5 different applications once deployed.

The framework will be fully documented and published.

The framework will be publicly shared on the intranet for others to use and eventually (hopefully) co-develop.