One of the most important challenges in test automation is figuring out what is actually worth automating. Not all things can be automated, not all things should be automated. But some can, and presumably some should. So inevitably we need to ask, why automate this thing and not that thing?
That question came up in passing today, and in the off-the-cuff answer I gave I realized that there was actually a paradox. Initially, when asked why I was working to automate a particular thing, I started into what is probably a fairly typical answer: it will be faster, less error prone, yadda yadda. But when pressed further, I changed tack: saving time on this task frees up time for us to work on more important things.
In retrospect, that was probably poor phrasing. It’s not that the other things I might work on are more important, per se, but perhaps more deserving of human attention.
So, what was the paradox?
One of the heuristics I use to decide if something should be automated, especially when it comes to CI/CD pipelines, is whether it’s important enough to stop something going to production (for whatever your definition of “important” might be). But here I was saying I was automating this task because it actually wasn’t the most important thing.
There were two competing heuristics at play in my answer:
- Time spent on test automation should be spent on automating the tests/checks that are most important to minimize the chance of error.
- Unimportant things should be automated so the humans can spent their time on more important things that computers can’t understand, like critiquing the product and understanding the user experience.
There are at least a few ways to resolve this. In part, “important” is ambiguous. In the first bullet, it refers to risk. In the second, it’s about menial labour versus complex thought. There’s also a question of automate-ability that will change the calculus. And, of course, they can both be true simultaneously; I have mission-critical test scenarios in my build pipeline, but I also have a linter. Each serves a different purpose.
I think the reason that this passing exchange stuck with me all day is that it felt dismissive of the work I had been doing. Did I really just say that the real reason I did it was to just get it out of our way so we could spent our time elsewhere? If that were true, why bother doing it at all? But no, it wasn’t that this task was unimportant. It was important, but the amount of time it took up was disproportionate to its actual importance. That’s the problem I was working to correct.
Lately I’ve been thinking quite a bit that from some angles quality assurance and testing is really about productivity. This was just the latest example of that.