While teaching a student from my SDET Lab programme, we had a good debate over what a good test is. I find Cem Kaner’s book on software testing very useful for beginners and I had given my student a copy of it as a reading exercise. A point mentioned in one of the initial chapters in Kaner’s book triggered the discussion.
From Cem Kaner’s book, a good test case is the one which finds a bug otherwise it is useless.
This is in contradiction with Kaner’s excellent definition of software testing: “Testing is an Empirical technical investigation done to provide stakeholders, information about quality of a product or a service”.
If the purpose of a testing is to provide information about the quality of something under test, IMO it holds true for a test as well.
The purpose of a test is to provide information.
So, labeling a test case as wasteful based on the outcome is wrong in my opinion. A test case that proves that there is no problem is as important as a test that proves otherwise.
Thoughts?
very true…i am agree with your opinion…passed test case is equally important as failed one.
“a good test case is the one which finds a bug otherwise it is useless”
Very fine and accurate;
also
“a collection of good and effective test cases are describe the qualities in terms of all aspects of Testing.
The investigation of software consists out of more than test cases. Only with executing pre defined test cases you are missing information about the quality. The aim of a test case is checking value of expected quality. And expectations can be wrong, so a good test case should be aimed to be a critical checklist on stuff that might differ from expectations. If you find differences, you probably have a bug, the testcase that found bugs had the most value.
The other part of testing, investigation via exploring does not use test cases, but it is also equally or maybe even more important, because in this part you will find non expected behaviour, that nobody thought about. The risks that you cannot think of at first, but are real and can only be found by exploration.
So, there is no contradiction. Testing is Checking and Exploring. This should lead to valuable information abt quality. The most valuable information tells something about risks (could be bugs or possible issues in future). That something works fine, is also good to know, but It is second in line for valuable information.
So: If a tester after a week reports: i’ve executed 500 test cases and I found no bugs, does that really say enough about the quality?
@Rob,
Thanks for the comment.
Testing provides confidence (or lack of it/level of it) about software quality.
About the 500 test cases, this is not sufficient enough information. If along with this number, I get a mind map of what all test ideas and their variants have been tried out, I can assess what level of coverage we are talking about. Now if the coverage is promising, is sufficient based on risk analysis given the timelines, and still not bug is found, I guess I have no issues with that and I would say that it was good testing work.