You need to know what's on the cutting-edge of technology. Find out what's coming and the unique Warptest POV with just one click on the "Blog" tile.

3 Reasons Automated QA Is A Pox On Our House

3 Reasons Automated QA Is A Pox On Our House

Automated QA Yada Yada Yada

Another year of recurring discussions online how the silver bullet of automated QA is killing manual QA. This ongoing trend has had several impacts on the market, not all of them positive.

The Impact of Automated QA

1. Craftmanship is dying not manual testing. Do you know one person who is a craftsman in anything or have they been replaced by mass industrialized process? This is progress you say?

Junior testers feel driven to dive head-first into automated QA before they have learnt the craft of testing. They see automation as the end product, not as a means to implementing the tests and methodology they never had a chance to learn about. What do they spend their time learning, how the product works under the hood or the intricacies of their chosen automated QA framework?

2. QA Managers are super-powered, high octane, ninjas but, at a high cost. If you are a QA Manager then you had best be a full-stack test professional, who can manage a team, function as mentor, be an oracle for the product technology, generate dashboards, manage DevOps / ALM tools but also dedicate yourself to hands-on implementation of automated QA yourself.

Why? Because QA professionals have sold to companies that it is possible and optimal that one person is capable of doing this without compromising quality (ironic much?) of management. QA Managers are being forced to sacrifice their strategic responsibilities for tactical operations for what? Burn rate or simply because this idea stands unopposed?

Analogous to this, when is the last time your Development Manager spent a meaningful portion of their work day writing code?

3. Don’t shoot the messenger. Did we forget that the essence of our job is to report our findings? Often we report defects, overall quality of a build but also we raise red flags. We provide preventive treatment for issues before they go to production but if our underlying methodology is flawed where does that leave us?

When the silver bullet doesn’t live up to expectations, the messenger is not always viewed favorably.

The Warptest POV

We frequently cite Michael Bolton on his philosophy regarding testing vs checking and where automated QA falls into this. Here is one of his tweets, yet do we really work this way, evangelize this philosophy to our companies? It doesn’t seem so.

In recent days they asked Elon Musk about delivery problems in his company Tesla and his answer, I built too much on automation capabilities. If Elon Musk can admit that automation is not the magical solution (read the tweet below) then isn’t it time we considered the same?

Elon Musk’s admission has much further reaching implications than simply admitting over-reliance on automation. Musk admits that the role of human workers in successful delivery is underrated.

How do we refactor our methodology to redeem the human role in testing and QA?

In a nutshell, the perceptual bias towards automated QA needs a stake driving thru its heart.

Whilst I have used existing terminology and semantics to avoid confusion (manual vs automated QA, testing, etc.) this is clearly a major reason why Michael Bolton goes to great lengths to use accurate and appropriate terminology (again see his tweet above and many other of his posts, tweets etc for more).

In fact, there are technologies that cannot be tested with Automated QA, the existing frameworks aren’t mature enough to provide a solution. We can create partial solutions but we run the risk of becoming over-enamored with the solution as a product and not executing our tests on the actual product.

If we can get past this misconception about how we test, then maybe we can get to meaningful discussion about how we reframe optimal use and understanding of automated QA.

Two ideas I discussed today on a Facebook group for Israeli testers (in Hebrew) were: –

  • Test planning and design is an umbrella that provides coverage for your tests:

automated QA - umbrella

  • Testing debt – testing should be agile but not just in the sense of testers in the agile team. Not just in the sense of testing as part of the sprint. Every tester, regardless of the tools they use must be aware of how they and their tests integrate into the testing process.

It’s 2018, are you ready to focus on the optimal and correct way to use automated QA?

 
Comments

Early on in your post, you said “Junior testers feel driven to dive head-first into automated QA before they have learnt the craft of testing”. Here’s a rather good analogy: here in the UK, if you apply to go to university to learn photography, the first thing they do is to give you an old-fashioned camera with film in it and teach you the traditional “wet” processes before they let you anywhere near a digital camera. That way, you get to understand the craft of photography in depth.

I recently had an encounter with a piece of customer-facing IT in my local supermarket which was so frustrating, I complained to the CEO of the company. The application had so obviously been put through automated testing because all the code was executing properly; but it had never been subjected to any manual testing because the user interface worked in ways that were completely counter-intuitive to the average human being. As a result, it was almost completely useless.

That’s a great analogy and the issue you raise is actually a big one. Nothing can replace either the gut feeling that the graphical user experience is poor or flawed nor the visual perception of wrongness. Automation frameworks have no capacity for this nor are we anywhere near an AI that can explore or intuit such concepts.
Thank you for reading and commenting. Jonathan.

Hi All im newbie here. Good article! Thx! Thx!

Hey there, thanks I really appreciate you comments and welcome to Warptest. Come on back and keep commenting and if you have a tech or testing subject then feel free to ask.

Trackbacks for this post

Leave a Reply