Test automation is one of my favourite subjects as a software tester. I have been planning for long to write a series – “Notes on Test Automation” – through which I can document and share my learning from designing and implementing test automation frameworks as well based on the literature I read for the purpose and test automation tools/frameworks that I analyse & use.
Initially it was supposed to be a series of blog posts, but later I thought to get it published in a testing magazine.
I approached Jose Diaz, the editor of Testing Experience magazine with the idea. He showed a lot of interest in publishing this as a series.
The first “episode” of the series has been published in the Q3’10 edition of Testing Experience magazine as – Test Encapsulation – Enabling Powerful Automated Test Cases.
I have copy pasted some contents from the published article in this blog post to give you an overview of what the series is about and what Test Encapsulation is about.
Introduction about the Series – Notes on Test Automation
Test Automation Frameworks (TAFs) do not get the treatment in terms of design that they deserve. Spending time on understanding the precise requirements and then designing/choosing the framework becomes much more important when a general purpose framework is required. Such a situation is unavoidable when your organization is looking at the possibility of a single underlying TAF that could be employed by multiple teams involved in strikingly different products; their testers are conversant with different programming languages, and they choose or needto employ different tools to support testing.
This series is focused on the technical aspects of designing a scalable, general-purpose, object-oriented TAF. This in turn means that the contents would mainly comprise concepts that directly translate into an implementation in an object-oriented language. One could choose to write the same thing in the traditional programming way, but there would be further work required on part of the reader to translate the design.
The series will not attempt to answer the questions of why to automate, when to automate, which tests to automate, automated tests versus manual tests, ROI of automation, etc. There are quite a few books available on the subject of test automation, and these topics are discussed at length in them. So, if the reader chooses to follow this series, it is assumed that he/she already understands the importance of test automation. This series focuses on how to automate, whereby the “how part” is not about using an existing tool or framework, but rather about designing one.
The name of the series “Notes on Test Automation” purposefully does not use any adjectives to indicate how universally efficient or effective its ideas are. This series will evolve over time, and I am going to share practical nuggets of test automation knowledge with you that I have learnt from my practical experience. We will occasionally strike out a sentence or two which revokes our initial understanding, and you might find me challenging my own ideas from the previous or even from the same article.
The Problem of Dumb Tests (or Why We Need Test Encapsulation)
In the design of TAFs, most of the important information that a test requires is kept outside the TAF, as many times such frameworks are designed using traditional programming rather than employing object-oriented programming. A simple example is grouping tests into platform-based folders, after which the scheduling logic copies in only the tests meant for a test execution machine based on a specific platform. There are multiple problems with this approach. One is redundancy, multiple copies of tests across folders. If you think about it, we could have a common test platform and then specific platform folders, but my question is how many such “commons” would you have? Even if you havethe patience, what happens when a test gets modified and is not meant to be run on Platform A? How do you know to which folder you should go to change it? So, there should be an external means to map tests to folders! By not making the information available within the test about which platforms it should (or should not) run on, you have the following situation:
- You need a complex folder structure, which is virtually impossible to maintain as the number of supported platforms grows
- You need an external means to map tests to the above folder structure.
- If the test writer is not the test framework administrator, it means the person who requires this is different from the one who implements it.
- You have a “dumb” test, which does not know on which platform it can run. So, if anything goes wrong, you have an incorrect test case that could crash the system or, at worst, lead to an incorrect result.
So far, we have only talked about the platforms. We have not talked about categorization of tests, priority values associated with tests, known bugs, API version checks. Can you see the complex and impossible folder structures that these options would lead to if kept outside the test?
This article focuses on the concept that a test is meant to be the most complex part of the framework in terms of its power and flexibility. Test encapsulation is at the heart of building a general purpose framework, which can be used by testing and development teams as a common test automation platform for whitebox and black-box tests. To know about what exactly is meant by test encapsulation, its approach and benefits, keep reading!
