Testing Is Method Oriented…
When we interview new candidates we often ask them if the repetitive nature of retesting the same features would bother them.
Our aim in this repetition is to catch the errors, the bugs, the non-compliance to specification or expectation and report on it in a manner that allows the issue to be reproduced and resolved.
The mission statement I give to my testers, especially the new ones is, we are here to ensure the customer pays good money for a product that delivers satisfaction/ happiness and solves their problem.
Tell a non-martial artist about Kata (roughly translated as form) and you will probably end up demonstrating instead of explaining. Kata is intended as a means of focused, attentive walk through, a systematic approach to improving the execution of specific combat techniques either alone or in unison with fellow practitioners.
As someone who has learnt Judo and Hapkido I can attest that performing Kata makes certain moves a reflexive response but also allows the person practicing the Kata to learn how the smallest change can deliver the largest improvement. I was lucky enough to study with a teacher who understood the bio-mechanics involved at this level and who saw the point of in-unison Kata in class as a time to observe and offer critique on each student’s Kata.
Most testing doesn’t offer the downtime for an exact allegory to Kata; having the team run through simulated test cycles is unrealistic. Observing and analyzing the nature and detail of Test Cases (STC) your testers create provides a Kata of sorts.
On a project in the last year I decided to run through the Test Cases and Coverage with the tester responsible for testing that module. Let’s call him Jim. Jim had been responsible for updating the Test Cases based on a major rewrite of the specifications. I asked Jim to walk me through the test cases and a pattern emerged.
I asked Jim how many times he had tested this particular module and he responded that this was the third time. While the focus of the testing was Business Logic Driven I saw no evidence of systematic testing of the UI. Knowing that a specific feature had just been changed to incorporate a search feature with results displayed in a grid UI element I asked Jim to show me the test cases involved in testing the new feature.
Jim walked me through the tests displaying extensive knowledge of the business logic and use cases behind the feature. Several questions occurred as I listened: -
Was this a case where too much Kata was the cause? No, because Kata is not simply rote movement either in Martial Arts or Testing; it’s the progressive understanding of what lies behind the repetition and practice. I’ve referred to holistic testing in the context of Automated and Manual Testing but it equally applies to running through Test Cases while understanding that coverage can always be improved through self-observation.
I asked Jim to speak about how we arrived at this fix to his testing kata together at the next daily team meeting and he agreed to review this.
After the team heard Jim’s review I suggested they consider if their testing kata had focused on one aspect whilst missing the fundamentals of another.
The Warptest POV
By providing a framework where the form and systematic approach to testing are monitored, evaluated and critiqued in a constructive manner you can ensure optimal test coverage.
Your testing team will be more prone to self-observation and attention to not just what they test but how they are doing it.
Instead of trying to abuse metrics to evaluate testing quality you are able to better improve on test cycles before they actually occur.
All this presumes that testing is not performed at an ad hoc level. It’s all about applying the right methodology to maturity of the App under test.
Are you ready to test?