What is a Test Oracle?
Is it a source of expected result? Is it the mechanism to validate the observed against the expected? Is it a means for producing expected result? Or is it a combination of all of these including the mechanism to bring the Component Under Test to a state where the expected results can be verified?
I have heard the definition of the word for all of the above contexts. All of the above are points that need attention from a tester. But which one is the test oracle? Should I focus on anatomy of this or spend the same time for learning more stuff and accept test oracle as a term in its multi-personality avatar?
Also, what is its relation to a heuristic? In the light of a blog post by Michael Bolton, heuristics are the super-set of oracles i.e. all oracles are heuristic. But then Doug Hoffman categories Heuristic Test Oracles as separate from 4-5 other types. Conceptually, I agree to Michael & the argument he made. I also agree to Doug as in his article he gives a very useful way of determining what can/should be automated. The confusion arises when we start using the word heuristic which symbolizes the fallible nature of whatever we do. So, even when I use the phrase “deterministic oracle”, it is fallible because I could go wrong in a lot of ways while using it. So, deterministic is heuristic ?!
I feel heuristic is another word which for me is going into the same basket as the word “context”. These words are the most powerful words when it comes to a testing skills. They are so generic and useful that they have become the backdrop of whatever I do as a tester. There they stand. The other side of the story is that at any point of time, there is no use saying that “based on the context,…” or say “let me give you a heuristic…”, as these are a given. They are so useful that they are paradoxical. “Can you give me a context when you should not assess the context?”. “There is only one infallible principle for testing – consider every principle as heuristic.”
After reading various pieces of literature I have picked up these words and they have become a part of my testing lingo. Infact I have been teaching about a heuristic of my own ( The GOLD Heurisitc ). But now every time I use these words, they cause more confusion than communication. I have decided to capture the essence and reduce the usage of these words.
[…] On Oracles, Heuristics and Context Written by: Rahul Verma […]
Thanks for the post.
I learnt new things : Oracles in Test and Heuristics – Contexts.
It would be helpful,with real time examples.
Heuristic is a specific word with a specific meaning. The fact that it applies to a lot of things does not make it generic.
Context is a specific word with a specific meaning, too.
In the Context-Driven circle, we tend to take our words seriously. One of the problems with the other schools, as I see them, is that they use words like ceremonial banners.
@James,
If usage of these words helps certain individuals, I guess they should continue using them to any extent they want.
They help me more on the mindset aspect rather than communication. So, I have chosen to use them very selectively.