Automated tests should run on their own in the background and we only care about them if they help us find bugs we missed. But not maintaining them is the same as having no test automation. In this article, I show you how you keep your tests running with as little effort as possible.
Sooner or later one of your automated test cases will fail. And the fault will not be because it found a bug, but because the test case is no longer in line with your application. Reasons for that can be as simple as a UI change or as complex as a complete rework of your business processes. Now there are three ways to handle this: Ignore it, delete the test case or adjust the test case.
Ignoring it is of course the worst option to choose. It leads to more and more failing test cases over time, no one will care about them anymore and the amount of work required to update them just piles up. So, ignoring should best not even be an option and most organizations handle this by implementing a CI/CD pipeline that simply does not allow it.
Deleting the test case can be an option if the test case is no longer required. But if you delete test cases every other change you implement you will end up with ineffective test automation. The safety net it should provide you will keep getting more and more holes.
So, in the long run, adjusting your test cases and maintaining them is the only option you have. In software development teams this is most likely not the most popular task and even if you are a full-time test engineer you want to keep the maintenance low.
The effort required to keep your test cases up to date really depends on how you build them in the first place. And it’s not even a matter of investing a lot of time and resources in the implementation.
By simply following these 5 key points in the test case design you can reduce your maintenance costs:
It is easier to maintain 50 test cases with five test steps each than one test case with 250 steps.
If you only care about the output of a given input, follow the principles of black-box testing. How your application handles the input and its steps change a lot over time. We constantly improve our logic and performance and make it more robust to errors. So in black-box testing we do not care about these changes and there is no need to adjust our test cases with every change.
What is the right amount is not that easy to say, but often corporate policies are in place like “we require five automated test cases for every feature in our software”. This leads to over-engineering in simple cases and not covering all the relevant cases in big features. A good start to define your needed test cases is to take a look at the feature requirements. Or start brainstorming about “what could go wrong” and then filter out what you find necessary.
As everywhere in software engineering follow the principle of don’t repeat yourself (DRY). But in testing, you must repeat yourselve. Because you need to test the happy case, the negative control case, and all the edge cases.
So when writing your test case keep in mind that you need to repeat yourself, but don’t want to. Your test automation framework should support you here. And if not, then you are using the wrong one.
The easiest and most important way to keep your test maintenance low is to use a test automation tool, that’s built with low maintenance in mind, like TestResults.io.
Through the intelligent architecture and end-to-end testing approach with features like codified expertise, graphical object model, and other intelligent algorithms, the maintenance is as low as possible. So don’t waste your time with unnecessary test maintenance that would be avoided with the right automated testing tool.
Now as you understand the importance of updating your test cases as well as how to design them to keep the maintenance as low as possible, the next step is to keep testing in mind in our daily work.
To make testing as effective as possible, it must be fully integrated in your development cycles. And you must take the tests serious.
Every failed test needs to be analysed and handled. It is ok if you find a minor bug and schedule it for a later release, but do not release a new version before you tested everything, looked at all the results and handled all failed test runs.
If you test every day and every feature, you will profit the most from your test automation. And you can no longer ignore failed test cases as testing is now part of your daily routine.
The most excessive method to do so is to adjust your whole project management according to the V-Model. Here every requirement and design is followed by a test and verification during the project phases. Other than in a waterfall model you will be confronted with testing every day and not just after the implementation is done and you start testing before the rollout.
In case you follow a more agile approach and no longer require specifications, the best way is to implement the testing in your CI/CD pipeline. With defined unit and integration tests for every change, testing will be part of your routine. Changes with failed tests no longer get deployed and for that do not affect other developers or even the production.
So in summary if you want to benefit from test automation and not just let it be additional work, maintaining your test cases is required. But if you design your test cases the right way and keep testing always in mind you will keep the maintenance effort and costs as low as possible.
Fun end-to-end testing across all browsers, operating systems, and devices. Super-stable, low-maintenance, and intelligent front-end testing tool that leverages your software quality, test automation efficiency and reduces costs. Over 3’000 Integrations. CI/CD. Codified Expertise. Remote testing across the office or continent.
