Eliminating Hard-Coded Waits In Your Test Model
Eliminating Hard-Coded Waits In Your Test Model
Have you ever read a code written by your fellow test automation engineer and you saw this:
Or perhaps you were so frustrated in finding why the objects won’t load and you said “I will just add a sleep and the problem is solved”. Well if you have, you probably know how this approach has ended. Your test case execution worked flawlessly. Yes, your heard me right. Flawlessly. The only problem was that your flawless execution was done locally and not where your test case will be actually executed. If you have ever fallen to this trap, continue reading. If you haven’t, also read this article to make sure you don’t.
The origin of the problem
We can differentiate between two main reasons why hard-coded waits appear in your test automation solution:
- You need to fix a problem fast – Test Case crashes once executed on the server/VM/Cloud env. – you create a fast solution to get rid of the problem. You will go back to that test case once you have some free time. The problem is: you don’t have free time.
- It doesn’t work locally – your test case/model only works with a wait. It’s a timing thing, it will definitely work on the target env. as it works locally. Deep inside you already know that is not true.
People tend to add the hard-coded waits as they think those actually solve problems. They don’t. There will always be a difference in your target environment in which your test case will be executed. The network might be slower on one day, the process will use more CPU, more VMs will be running on the server and taking the CPU/RAM usage, the GPU will be used by a different process. There are multiple reasons why you could wait for objects to load a bit shorter or longer.
Solution 1: Global object Waits to find Objects
The typical incorrect and magic Wait/Sleep statement looks as follows:
Do.Something(); Thread.Sleep(3000); var object = new Button(method to find the object);
This approach would fail each time the object is not available after 3 second after an action against the SUT has been taken. We already know we don’t want that. It makes way more sense to search for an object for X amount of time and only fail if this time is exceeded. Imagine that you have a common control in your Test Model which is a Button. Now assume that for each Button object you will wait 60 seconds. That doesn’t mean that you will have a Thread.Sleep(60000). Instead you will try to find the object multiple times until you reach a global timeout. In practice it will look as follows:
Console output: 20:01:01 Searching for button: Login 20:01:02 Searching for button: Login 20:01:03 Searching for button: Login .... 20:01:57 Searching for button: Login 20:01:58 Button Login found
Thanks to this approach you will be covered for each case when you have a variety of response times from your execution environments. As long as you don’t test the performance of your software and you don’t have specific time requirements in which a certain object needs to be loaded, this approach will work with you . On a fast target env. the object will just be identified sooner than on a slower one. Of course you should also consider the global timeout for such type of an object identification. In the case it’s exceeded that means something is really wrong with the SUT.
Sleep() is evil in coding as well as in testing. Everybody agrees on that.
If you want to know more about modern automated testing with your own dedicated testing robots, let’s do a short assessment.
Solution 2: Stable screen
In a perfect world it would look as follows:
This call could be done whenever you create a new screen/page as the first call you make. In addition it can also be used if interactions lead to animations, like for drop down boxes.
Solution 3: Interaction verification (object appears, object disappears, object is changed)
I have many times seen hard-coded waits added just after an interaction with the SUT. Usually the scenario is common and looks more or less as follows:
1. AddToCart.Click(); 2. Thread.Sleep(4000); 3. CartFullBasket.Click();
The sleep is added just because we need to wait for the next control to be available after the interaction with the previous control has been done. If we already know that the new control is a result of an activity with the SUT why not use our global wait approach in a more fancy way. We can check if the control is loaded as a result of an interaction. Consider the example below:
In this case we are enforcing a check of a successful interaction with the SUT and at the same time we are not binding ourselves to any hardcoded, control specific waits. This also gives you one more hidden advantage. It will be way easier for you to identify which interaction with the SUT has failed during execution. Think about it. Usually you click around the screen and need to perform several actions to check the end result. Often what happens is that something which has failed a couple of steps back is a trigger for the unexpected behavior of your automated test case. This approach allows you to remove this problem as you will fail directly at the moment you need to fail and not N steps later.
One small improvement: Global interaction delay
Last but not least option is to set a global interaction delay. You definitely know that your test case execution is usually faster than you would have executed manually. That is what is expected from an automated solution, but on the other side an end user will definitely not click around and type as fast as a lightning stroke. The solution to that problem is the global interaction delay. This delay allows you make sure that there will always be a delay between an interaction with the SUT. It’s not a 100% bullet proof solution, but at least it will make sure that your test case execution acts more like a human than like a robot.
The hardcoded waits in your code are just pure evil. No matter what you say they will always cause trouble and cause your test cases to be flaky. The examples above only show you some proposals on how this problem can be addressed. I invite you to share your experience about handling the Sleeps() in the comments section below.