Rahul Verma: My Petty Bits

When I was thinking about writing the next entry in the AEIOU model explanation, I thought of writing about Input/Output. I soon realised that it is too big a topic for a single entry and could get too complex too quickly.

I’ve taken some time to think about breaking down this problem in multiple parts. No such attempts are perfect. However, I hope it builds on where I left the previous discussion in a coherent manner.

Before reading further, here’s the AEIOU model for your reference and if you haven’t already done this, please go through the previous articles on this subject to make a proper sense of this article.

What is Input/Output (I/O)?

This is a vague question. So, any detailed answer won’t help as it will be biased.

I’ll attempt to crack this ambiguity with ambiguity :-).

What is a Mental Model to Me?

It’s how I make sense of things in my mind. I, like everyone else, end up borrowing knowledge from the world and knowingly/unknowingly interpret problems. I am almost obsessive about having my own interpretation.

A mental model in my opinion is a one step further. It’s a conscious choice of arranging these interpretations.

I am a person who connects to visual and abstract stuff better. My mind thinks in terms of boxes and arrows.

AEIOU model is one such model, or rather a documented incomplete representation to make sense of my corresponding mental model.

In the subsequent sections, pay attention to the boxes and arrows – that’s all that is to explaining this otherwise difficult but under-explored concept in testing.

Model 1 – The Usual Mental Model of I/O

Let’s start with a basic model which I think is registered in the mind of most testers for what is I/O:

1. There is something we want to test – the Test Object.
2. Whatever goes into the Test Object that a tester provides is input.
3. The Test Object does some magic.
4. Whatever comes out of the Test Object is output.

This model later inspired a famous dialogue in Sacred Games without due credit:

Loose English Translation: “Sometimes I think I am God”. There it goes again. All the fun lost in translation. Back to the topic, weasel. Me and my mental models surely need some re-arrangement.

On a serious note, this model, although a good start is extremely limiting. The tester thinks that s/he in full control, and fails to understand that s/he is just a small piece in a complex game laid out in front of him/her with many hidden players.

It’s not a humble model in itself although if one is considering the involved complexity, the magic could be humbling :-).

Anyways let’s build on it.

Model 2 – A Basic I/O Model from Control Perspective in a Test Transaction

As a Tester, you need 2 fundamental points of control – to provide input and to observe output. Let’s map this and Point of Action from AEIOU model here:

Even weak models can help you. This model highlight an important point about point of output observation as being different from point of input. However, the model is too simple to explain its context and importance. So, for now, I am going to ignore it and leave it for a later discussion.

Model 3 – A Basic I/O Model from Point of Primary Control Perspective in a Test Transaction

The Model 2 fails to explain the most basic concept of how a single atomic interaction includes both Input and Output variables in it. It also does not highlight the concept of interface for the interaction to take place. Also, it does not give the sense of a primary point of control.

Let’s re-arrange the boxes and arrows:

Even weak models can help you. This model highlight an important point about point of output observation as being different from point of input. However, the model is too simple to explain its context and importance. So, for now, I am going to ignore it and leave it for a later discussion.

Model 3 – A Basic I/O Model from Point of Primary Control Perspective in a Test Transaction

The Model 2 fails to explain the most basic concept of how a single atomic interaction includes both Input and Output variables in it. It also does not highlight the concept of interface for the interaction to take place. Also, it does not give the sense of a primary point of control.

Let’s re-arrange the boxes and arrows:

However, this helps in understanding many crucial aspects:

• A tester will need points of control on the right side of primary point of control for many types of tests. That’s what Embedded Test Control in AEIOU model depicts.
• The concept of action taking place in layers within and outside of the test object becomes evident.

Another critical aspect to note is that because of the complexity of I/O involved, the sense of direction/ arrows is lost in most of part of the problem unless we want to introduce biases.

Interaction as Input/Output

Model 4 highlights another important point which needs as a separate section.

It’s not just data that’s input/output. An interaction can also be an input/output as far as testing is concerned.

Tip: Whether an interaction happened, how many such interactions happened and in what sequence are variables that can be observed and subjected to oracles as a part of a test for deciding whether there is a problem or not.

This is counter-intuitive for many testers who think of input/output only in terms of data.

Model 5 – The AEIOU Model

If I continue building on this line of thought, I will reach the Interactions & Observations section of the AEIOU model. So, you can think of the above models as deconstructions of this model.

Discovering a New Model: Model X – Attempt 1

Yes, that’s true.

Which of the above models is THE model? None! All of them above co-exist in my mind and all of them help.

Frankly, till this point of writing this article, I’ve not been able to arrive at the model which I thought I will arrive at. I want a model that I can use as a simple basis for the things that I want to write in Sampling series.

If you haven’t realised by this point, this article is not to tell you what are the models. This article is about discovering your own.

Let’s have a look at an abstracted revision of Model 4:

The sphere with the dual shade of red and blue shows you that a test has a small, partial relationship to the interface of a test object and in its logic flow it hits small, partial pieces in test object and participants. There is a need of test control everywhere for precise testing.

Discovering a New Model: Model X – Attempt 2

Some further points:

• Some I/O is transient. It moves through test object and participants and is lost.
• Some I/O is persistent and becomes a part of the system of test object and participants.

This is an interim model that comes to my mind:

As I created this model, it explained to me how to break down the concept of Test Transaction into stages.

There are three primary stages to a Test Transaction: the begin stage, the engagement stage and the conclusion stage.

Begin Stage of a Test Transaction: The test object and participants are at a certain state. Is this state the one you need for a test? Mostly not. So, you bring the totality of a test object and participants by changing the values of the relevant variables that define this state.

Engage Stage of a Test Transaction: You make various interactions with the test object relevant to your test. During the same time, you also make interactions to other participants for controlling the test as well as making interim observations.

Conclude Stage of a Test Transaction: You make observations related interactions with the test object and participants. You could also reset the input variables to bring them to the state where they were before the Test Transaction.

Other critical points of interest:

• It acknowledges the role of unknown actors that have an impact on variables and the state, not just a test actor.
• It acknowledges the importance of output variables at the begin stage too, which a is little ticklish concept to understand but is the basis of goal based testing.
• It acknowledges the role of a Test Actor in all stages.

Variables and Direction

It is established so far that the input/output variables are infinite in nature.

The meaning of direction for majority of the model is ambiguous and open ended.

So, can we simplify the model?

Model 6 – An “Infinite” State Model for Input/Output in a Test Transaction

I use the word infinity in my work in an abstract sense for an “unknown maximum”. From math classes, I remember this was represented as “n something” e.g. n inputs where n is the max number of inputs possible in a context. In my thought because the word context itself is fluent, saying ‘n’ does not work for my mental models. So, I’m happy with a liberal use of infinity. You can decide what works for you.

Most of these input/output variables of persistent nature define the state of the totality of test object and all participant objects.

It means this “system” has a begin state, then infinite internal states and then the final state as mapped to the 3 stages of a test transaction:

A tester would like to think as if s/he is in full control of this infinity, but s/he’s obviously not. So, the model also needs to acknowledge the role of unknown actors.

The model highlights another point now w.r.t. I/O variables:

• Some variables remain relevant throughout the test transaction.
• Some variables are relevant only in a specific stage.
• Additional variables become relevant as me move through the 3 stages.

Also, it splits the concept of states of test object and participants into 3 zones:

• Test Transaction: Begin Stage: We need a controlled state here. This means we need to know exactly which variables are relevant and what values they should take. We need to bring the system to a stage where this becomes true.
• Test Transaction: Engage Stage: Though many atomic/composite interactions and the indirect interactions they trigger, the system moves through infinite states. A tester typically considers a sub-set of these states for observation.
• Test Transaction: Conclude Stage: This is the final state of the system. A tester identifies with this state most commonly as the observation stage where readings are taken for relevant output variables. This is also a stage where resetting the state to the pre-test transaction state can become important.

Model 7 – A Dual State Model for Input/Output in an Interaction

Model 6 helps to appreciate the overall complexity of variables across the test transaction. However, it also is too complex to think about simple tests.

Each model has a purpose. Complex models explain complex situations however they can also block thinking for simple situations. We need both.

So, let’s say a test transaction consists of many Composite Interactions which in turn comprise of many Atomic Interactions. Many a times, a singular interaction thinking can help a tester for a mental model.

This simple model is able to explain how a tester should be thinking in terms of I/O at any point in time when actively engaged with a test object or participant.

The dotted red arrow for the Reset indicates that it can be optional as the locus of execution in a test moves from one interaction to another.

Ideally, reset of state should be done after each test transaction. In practice, this needs trade-offs.

Atomic vs Composite Interaction

I defined these in previous article.

However, these are just think-words. We don’t really know what is atomicity and what is the final edge of composition.

It will help you to think about Interaction as a Continuum starting from Atomic to Infinite Composition.

Model 8 – A Simplified Dual State Model

Here’s the single simple model which I think I use when I think in terms of Model 8. For complex situations, I just multiply it:

Compare it with Model 1 and you’ll find that the above model, though simpler is a different outlook to how a tester should look at Input/Output in a test transaction.

Rather than thinking from the tester’s angle, it puts Test Object and Participants in focus.

It considers the tester’s role as a thinker who makes conscious choices and observations that result from his direction/indirect interactions with the Test Object and Participants.

It has no other arrow apart from the arrows of movement of states, as it is assumed that a tester will use whatever is in his/her capacity and knowledge to introduce himself/other testers/automation to do controlled test transactions.

The role of unknown actors is assumed as a reality and being cautious of that another assumption for a tester.

Model 9 – Input/Output Model for a Test Transaction

Model 8 in itself does not acknowledge that some input/output variables are transient i.e. their impact starts and ends with the Test Transaction. Some of them have more persistent impact and hence they become a part of the state of Test Object and Participants.

Following is an abstract model that clubs various aspects of I/O discussed so far and leaves the details open-ended:

Summary of what this model indicates:

• Possibility of multiple Test Actors at multiple points of control and I/O amongst these actors.
• Direct I/O from Test Object (which Model 1 was all about).
• Transition of states of Test Object and Participants because of this Direct I/O
• A reset of state during and at the end of a Test Transaction
• I/O interactions at the begin, engage and conclude stages of an interaction or transaction.
• The symbol of infinity acknowledges the complexity and how intricate tests can be.
• Last but not the least, it highlights the importance of Test Actors. The human-robot face is about opportunities for combining human and automation roles in activities in the same Test.

Further Thoughts

The truth about models is that once you have worked out the details, for all practical purposes you end up using a very simple model in your mind as a front to more complex models in adhoc combinations. As you learn more, the models should get modified and re-arrange themselves.

All the previous models have value. However, in my thinking, my mind tends to hide details behind simple imagery. It’s difficult for me to put this in words. So, I wrote this article in a very raw form so that I can show how no model is perfect.

Simpler models make way more assumptions than the complex ones. Complex models add so many details that they become brittle.

Each model could highlight an aspect which others don’t. That’s true for any model that you create. So, rather than aiming to create one perfect model that explains it all, create a base and its extensions. That’s what AEIOU model and the extensions in this article mean to me.

Try developing your own models based on the information above and elsewhere. It’s essential that we connect the dots in testing, in a way tailor-made for us. That’s the personal connect and system I think every tester should target between him/her and testing.

PS: These models are only partially helpful for thinking in terms of I/O when it comes to machine learning models, especially during training/testing phase. That’s a subject I don’t fully understand yet and the corresponding tweaking can happen only post that.

That’s all for now.