Rahul Verma: My Petty Bits

When it comes to asking some pertinent questions, I have interacted with hundreds of testers closely in my work & workshops and thousands briefly when it comes to conferences . That is not an exaggeration.

Personally, I am an introvert with just 4-5 close friends (that’s an exaggeration I think :-), the number could be lesser.). Professionally though, I’ve broken this personal barrier to serve my profession.

Testers and the Craft of Test Design – The Ground Reality

One of the key questions I have frequently asked from testers is – “How do you design tests? What are the test design techniques you know about?”

Here are some observations:

• Some have no clue. I can read their faces and tell, they don’t know what is a test design technique. Unfortunately, testing is still treated by some, not as a skill but something taken for granted – any one can do it. So, one can become a tester, clear a “Testing” interview without knowing about testing or cramming some question/answers (these at some point were called “Dumping” in India, I don’t know what they are called these days – “Influencing?”)
• Good News. Most testers mention “<random prefix> Boundary <random suffix>” in some way or the other as the first technique that comes to their mind. It does not matter what they call it really. It indicates most testers know the importance of testing around boundaries. However, digging a little deeper quickly tells that most of them have a very vague idea, even of the basic arithmetic involved in boundary value analysis. Change the question a little bit, talk about numbers with decimals, talk about delta apart from 1 and you get answers that are partially wrong or incomplete. Go beyond a style of question that says “x has this range rule(s)” to an actual software/device and the gaps are even more evident. Internal boundaries? Forget it.

I once asked “What is Boundary Value Analysis?” in an interview from a person with around 14 years of experience. I was a youngster at the time (yes, I was that way once :-)) who offered interview-as-a-service consulting. This manager was visibly offended. Instead of answering he started commenting on the quality of the question. I said I understand, how about this – this is the only interview question I’ll ask and we are done. The answer he gave was – oh it’s simple and went on with the the +/- arithmetic which a kid can do (no, not a testing kid, like a kid who has been just taught subtraction and addition in primary school). I said, now make this answer worth your 14 years of experience. It is the same answer a tester with less than 1 hour of study can give me. I know you have guessed the answer he gave – silence and then again challenging the question :-). That’s what happens when one confuses ageing with experience. Contrary to the popular belief – Zyaadatar baal dhoop mei hi safed ho jaate hain (sorry, untranslatable. literally: most hair become gray in sunlight – my goodness, that just killed one of the best things I wrote in this article).

• A few testers mention something about Equivalence classes.

Please note that most of these testers mention this *after* they mention about boundaries. Will come back to this error later in the series.

This is where at the surface level, because of the inherent ambiguity of Equivalence Class Partitioning, the answers *may* look right. Dig a little deeper and you can see through how unstructured and vague an answer is.

• After this point, it’s open game. I consider the testers who are here and talking about further techniques in the “hope” zone – they give me hope that learning the craft of testing still has value in some minds. However there is no consistency. Some typical answers by testers in this zone include state testing, tours, fault injection and so on.

Testing is an art. At the core of this art is the craft piece too. Testing is a craft.

Some testers treat it as neither – they are still not really happy about being a tester. The testers who treat it just as a craft haven’t really found the happiness they deserve, so they end up exploring the craft part itself partially. My focus area has been the testers who treat testing as an art. Unfortunately, many of them have stopped recognising that testing is a craft too. They are like painters with great artistic potential and self-confidence, but don’t know what a brush is. In all this noise, fortunately there are those who helped me in my journey so far by just being the beautiful testers they are.

The Single-Variable Bias and the Disconnect Between Learning and Applying a Technique

When one learns the basic techniques BVA/ECP etc, they are often taught by taking a single variable example. Of course many texts out there go beyond this point, but I don’t see most testers reading these techniques till that point.

That’s the disconnect.

We are increasingly living in a society which thinks they know about a movie by watching its trailer. We are in a society where everyone is in a hurry. Testers are no exceptions. We skim through posts/articles/books or worse yet, just read the heading of an article and form an opinion before even going further/deeper.

A sub-set of testers can talk about what the techniques are. Give them a single variable and range problem, they will solve it too (howsoever wrong/incomplete, they will solve it.)

Give them a real problem and very few of them can correlate technique to practically applying it.

Professional Tip: When you ask a vague “How to ..”, “What is ..” question in Google search, the articles and source exactly/closely matching this title are often wrong and cheap sources of information. E.g. as of writing this sentence, I searched for – “What is Boundary Value Analysis?”. The first search result is from Guru99. The next 5 are from GeeksForGeeks, Wikipedia, Software Testing Help, Javatpoint, eduCBA. Maybe Wikipedia entry is good, may be not. But I hope you see how a Google search like this constantly directs you away from the great minds in testing. Want to be good at testing? In searching, beat those who are good at search engine optimisation. Ask better questions. At times, skipping the first few search results is the only trick you need.

I’ve done this experiment all the time. You can try this out too.

1. Step 1 – Ask a tester about what is BVA. You’ll most likely get some answer.
2. Step 2. Ask about ECP/other techniques. You’ll get some more answers.
3. Step 3 – Now give a testing problem, not broken down into its variables – you’ll most likely see random test ideas, with no structure to them, no sense of coverage using the techniques that they talked about earlier.

On a related note, why do some people out there think that ChatGPT is generating good test ideas? Why do they think that it’s better than most testers out there? Because at a glance, it infact does give better answers – more comprehensive, more structured than most testers. It’s only when you look deeper that you find gaps in its answers. The first impression it gives, is better than an answer which most testers can give. There’s nobody else to blame for this – we can not blame why somebody created ChatGPT. If nothing else, let’s accept the challenge. And no, it is not better than any of the authors I’ve read/testers or developers I respect, in their space. It’s a good tool though and is increasingly raising its own bar. Forget excellence for one – can we as testers respect and reach the foundational bar that our profession demands?

My Motivation to Write a Series on Sampling

There is wonderful work done by great minds in testing in this area. I am making no claim to do better or even equivalent to them.

This series is my take on test design where the craft and art goes together. I hope that it will fill some gaps in the knowledge of testers who haven’t explored this area as well as those who have been working on it for many years.

I’ve got many teachers – the people with whom I’ve debated, the books that I’ve read, the courses that I’ve attended, the testers whom I’ve coached, the projects and products I’ve been a part of and so on.

I’ll try to club whatever I know with all the inherent contradictions in a single place.

The least this series will do is helping me put my thought process in a coherent manner in one place.

Let’s see how it turns out.

That’s all for now.