In the first article – Test Encapsulation – published in the Q3/2010 edition of Testing Experience, I explained how the design of a test automation framework (from now on referred to as TAF) should start by thinking of tests first and then the surrounding features. In this article I’m going to address another challenging area in the design of TAFs – Distributed Testing.
Distributed testing is about enabling the TAF to distribute and run the same or different tests across multiple test machines at the same or different time(s), monitor their status and consolidate test results from all test machines.
In distributed testing, the test framework is logically split into two types of machines:
- The Test Controller (from now on referred to as Controller) – The machine which distributes the tests, monitors their overall status and is responsible for results consolidation.
- The Test Runner Machines (from now on referred to as Runners) – The machines which run the tests.
In certain situations, a machine could very well be the Controller as well as a Runner. Accordingly, it can have logic corresponding to either one or both functional areas.
Some typical uses of distributed testing are:
- OS Platform Coverage – Running tests on various platforms
- Hardware Coverage – Running tests on machines with different hardware or resources
- Performance Testing – Generating simultaneous requests to a target application because a single machine would not be able to generate the target load
- Security Testing – Separating the Runner from the Controller for the basic reason that such tests can result in operating system crashes
- Reducing Test Cycle Time – Even though the tests can be run on a single machine, multiple similar machines are used to allow parallel execution
- Common Test Framework – Running various types of test on different machines because they need different configurations.
Through the course of the next sections, in this article, we will try to understand the challenges faced in the distributed testing mechanism and various ways of developing an automated solution for the same.
The article is available as a PDF file here (~ 0.5 MB): Notes on Test Automation – Distributed Testing Models
You can also download the complete magazine from the Testing Experience website after a small registration process.
Rahul Verma