The Ultimate Pro Tip for Overcoming Flakiness in UI Testing

The Ultimate Pro Tip for Overcoming Flakiness in UI Testing

May 14, 2024

Ever been in the middle of an automated UI test and, bam, a popup about pending updates appears?

Or even worse, Windows decides to restart because of an update?

These interruptions aren't just annoying; they risk breaking your test cases, and creating what we call 'flaky tests.'

Update notification pop-ups are especially problematic for visual systems and can also disrupt conventional test automation tools that rely on element IDs and properties.

As testers, we need control over our testing environments, right?

You deliberately test in a specific browser version and want to report this accordingly. Or you test your software in the last three versions for your users who haven't updated their browsers automatically, so that they also have a flawless experience.

Unexpected OS or browser updates make that challenging!

But don't worry; Pascal has some hacks for you on disabling the automatic updates.

Let's start with Firefox

First, in Firefox Settings, search for "Update" and select "Check for updates but let you choose to install them."

This stops automatic updates but may still show update popups.

To fully disable these notifications, create a "distribution" folder in the Firefox installation directory (where firefox.exe is located, e.g. C:\Program Files\Mozilla Firefox\).

In this folder, make a policies.json file with the following content:

{ "policies": { "ManualAppUpdateOnly": true } }

This tweak gives you complete control over Firefox updates.

You can find a pretty illustration for downloading this step-by-step guide on our LinkedIn.

Edge and Chrome Nuisances?

Take These Steps:

1. Open "Task Scheduler" from the Windows Start Menu.

2. Delete tasks named GoogleUpdate... and EdgeUpdate...

3. Next, go to "Services" from the Start Menu.

4. Find gupdate and edgeupdate services.

5. Change their Startup type to Manual in properties.

And Windows? Here's a Tip:

You can pause Windows automatic updates for up to 35 days. This is a temporary fix, but it spares you from constant update interruptions.

Remember, regularly updating your OS is still essential for security.

Disabling updates entirely is more complex and depends on your machine's usage. One option is to disable the Windows Update Service, but it might be enabled again upon system reboot.

Remember, while these solutions offer temporary relief, always balance the need for a stable testing environment with system security, performance and new features.

Happy testing!