Test Coverage - Contd

Vasu,

Lets assume that in your project, customers are finding more production defects, if you ask me , then i can say

1.I am finding your test strategy is wrong.

2.Whenever production bug is found by user, just analyse whether the bug they found (and you lost) is because of carelessness / negligance of your testers or because of the your nature of testing i.e your test strategy

If first is the case, then we need to make sure it wont happen and we may practice techniques like pair testing (two tester in same test area) or more tight supervision of test execution etc. If second is the case, you have to revisit the strategy.

3. If you keep on getting production bug then you need to get the list of production bug and do the gap analysis with your test cases. It is not simply adding the defect as test cases, but you do the 'gap analysis'. For e.g take X number of defects found in production and see the test cases in that area , the question should be asked is 'Why they havent added as test cases before?'. Finding a pattern between "{Z number of defects found by production users} vs {Y number of (probably ineffective) test cases} will also throw some light.

4.Diversify the testing techniques.This is one of the important area that should be concentrated. Change the testing techniques often during test life cycle.According ot context driven testing school there are four dimensions of testing,
-People oriented (beta testing, UAT etc)
-Risk oriented (Risk based testing)
-Coverage based (like documentation based testing, testing page by page of product documentation, or combination testing etc).
-Activity based (like regression testing, Long sequence testing etc)

There are many techniques in each section. But one of the important technique that i would like to quote is "Map and test all the ways to edit a field". You can change a field value in different ways. Depending upon the context, this technique may prove powerful.You have to test in all the possible ways.

5.Use gray box testing: do the black box testing by applying white box testing techniques like Decision coverage and condition coverage.

6.Clear the old test cases. See what are the existing test cases are obsolete and delete it or make them as non-executable. In this way you can save time & energy which can be used for new powerful attacking test cases.

7. Get powerful test data, it depends upon the project, some project real time data will act as powerful agent.

8.Get the help of end user (subject matter expert / business analyst). Schedule a meeting and go with important questions, like pattern of usage, pattern of real time data, more use cases / user stories etc. based on which we can create more powerful test cases. If it is product, then ask product manager, or marketing person to gain this knowledge on common usage.

9.Get the use of support team, use support team's knowledge on their calls in your testing.

Iteration by iteration, in this way we can reduce the production bugs.