There are multiple articles on the web explaining how to write the test automation code, what are the best practices, what you should avoid etc. The key question that our customers often ask is:
Where shall we start with test automation?
Indeed, it is a wonderful and a valid question. Let me give a small example. Imagine you are going on holidays. The first you do is carefully select the destination, hotel and, naturally, a date. You have a fixed amount of time and also a budget for your long awaited time off. You don’t just simply jump on the first random plane to the middle of nowhere and say “Lets see how it goes”.
The same as you plan your holidays, you also need to plan your test automation journey. From a high-level perspective you need to address the following points which we will discuss within this article:
- Test Automation Goals and Scope (for your proof of concept)
- Budget and Staff
- Implementation and Evaluation
Test Automation Goals and Scope
The first step you need to take is to identify where test automation would be most beneficial for your organization. It doesn’t make sense to automate test cases which you execute once a year, whereas it does make sense to automate a set of regression test cases you need to perform for each new build. Set a clear goal of what test cases you will automate.
There may be several projects in your company where test automation could bring a lot of benefits. In this case you should really be careful when selecting the project with which you will start. The best practice is not use the biggest and most complex in-house software as the significance of the changes you bring might not be noticeable. On the flip side, you shouldn’t take a trivial project as one could state that your improvements “doesn’t prove anything”. Therefore, start with a something of a medium-size where you can show measurable results after introducing test automation. However keep in mind, this should be just a proof of concept and not a full blown test automation project. Once you are done with it, you will be able to decide if it is beneficial to continue with test automation within your organization.
Once you know with which project you will start and what test cases you want to automate, you will need to perform a set of activities as if you would start your own small software project. Treat it as a real software project.
It turned out to be helpful to have experienced experts on your team while setting up everything for the first time. Not for long, just long enough to have everything in place, working and accepted by everyone.
External experts are useful not only with their expertise but also by being what they are, external. They do not have to follow any internal political discussion, etc. Their advice is seen as what it really is: Not based on opinions but experience. And everybody is happy to accept or at least consider such advises.
Talk to us today. Not only do we provide a complete new approach to test automation but we can help you getting everything setup.
Budget and Staff
The biggest mistake people make when approaching test automation is believing that it can be done within the current budget. That is a big misconception. First of all you need to consider that you will need to allocate either yourself or a team of people for your small trial project. I believe it’s needless to say that, people cost money ?. Second of all, you will need to invest in tooling which will support your test automation attempt. Third of all, most likely you will need to invest in new infrastructure or a cloud infrastructure service to host your test environments, reporting service etc. Naturally, there are more items you can think and therefore it is important that you don’t underestimate the budget. Even with your proof of concept it would be a big pity to burn all your money and not have any meaningful results to show to the higher management.
In addition, you definitely need to find the correct people to do the job. It would be stating the obvious to say that the best would be to take a test engineer with test automation background, or even better a test automation engineer. However, since you are at the beginning of the journey you most likely don’t have such people available for the project. There is a solution to that. Take a software engineer instead. Since the developers know the architecture of the Subject Under Test, they are also a great asset in determining the easiest way to automate it. Additionally, if you are really looking for a robust and reliable solution, coding skills will be required and therefore having a developer on board will make your life way easier.
Now that you have your budget and staff ensure that you define the timeline for your proof of concept. You need to specify the time needed for at least the following items :
- Initial kick-off
- Identifying the spikes
- Identifying the tools
- Tool evolution
- Setting-up the test environments (your infrastructure)
- Writing the first test model
- Writing the first test case
- Reporting the results
- CI/CD Integration considerations
Each of the points should have a dedicated and allocated time slot. The timeline needs to be available to all of the team members associated with the test automation project so that the entire team can see the progress and the ultimate goal.
Check out our checklist, that we use before we on-board new customers:
Test Automation Readiness Score Checklist
You have set-up the goal, scope, budget and the timeline for your project. But when do you call it a success? How shall you measure the effectiveness of your POC? You need to set the key performance indicators which will give you a clear indication that introducing test automation in the company will be beneficial. Once you have those measurement it will also be easier for you to organize a test automation budget when you will be presenting them to the higher management. Such KPIs can include (but are not limited to):
- How many test cases can be automated in a given time-frame?
- Failures detected by automated test cases
- Execution time of automated test cases
- Number of false-positive and false-negative results
- How many hours can be saved in comparison to manual test cases?
- Increase in test coverage
- Reliability defects found
It is important that you identify these indicators at the very beginning so that you can measure your success along the way. Set the KPIs you think make sense in your company and define the ones you can measure.
Implementation and Evaluation
There is no way around it. You need to implement the proof of concept on your own. You will definitely hit some obstacles along the way, but that is a part of the deal. You need to keep in mind that at the end of the POC you will have a valuable solution which solves some of the testing problems your organization is facing. Gather all of the learnings from the whole process: What were the pitfalls, where did it take more time than expected but also what went better than anticipated. Along with the KPIs that will be your input to the evaluation meeting.
All of the stakeholders which can potentially benefit from your test automation solution should be invited to an evolution meeting where you will present the results of your proof-of-concept. This meeting should not only summarize your achievements but also show which advantages does test automation bring to your organization. Starting with time and money saving (yes, numbers talk) and continuing with the impact on the customers (better quality, reliability) you should be able to convince each one that test automation is the correct way to go. By the end of the meeting you should also discuss the deployment strategy for the other projects, future road map for test automation. This makes sure that your idea will not die with the end of your mini-project.
Originally, I intended to wish you good luck! Instead I want to give you the option for a short testing assessment, just book an appointment right now