DORA metrics in software testing: How to measure testing without slowing delivery

What are DORA metrics?

DORA metrics are a set of engineering metrics used to understand how effectively an organization delivers software. Rather than measuring how much code a development team writes or how many tickets it closes, they look at what happens to code changes as they move through the software delivery process and into production.

The metrics balance two sides of delivery: speed and stability. How quickly can teams move a change from development to a production environment, and how reliably can they do it without creating failures or additional work?

This makes DORA metrics useful beyond DevOps teams. Software development teams, QA teams, and operations teams can use them to identify bottlenecks, measure performance over time, and drive continuous improvement across the delivery process.

What does DORA stand for?

DORA stands for DevOps Research and Assessment.

The DORA metrics were developed through Google's DevOps Research and Assessment program to provide standardized indicators of software delivery and DevOps performance.

Rather than looking at development and operations teams separately, the metrics provide a way to evaluate the performance of the delivery system as a whole. This is particularly useful when a slow release cannot be traced to one obvious problem. The bottleneck could sit in development, code reviews, testing, deployment processes, or elsewhere in the CI/CD pipeline.

Why were DORA metrics created?

Traditional engineering metrics don't always tell you whether a team is actually good at delivering high-quality software.

A team can write more code, close more tickets, increase its number of automated tests, or find more bugs without necessarily improving its ability to get reliable changes into production.

DORA metrics provide a different perspective. They help organizations benchmark software delivery performance, identify bottlenecks in CI/CD pipelines, and support continuous improvement in DevOps practices.

For software testing, this distinction matters. The goal isn't simply to implement automated testing or maximize the number of test cases. Testing should help teams deliver changes efficiently while preventing failures from reaching users. DORA metrics can provide valuable insights into whether the wider software development process is achieving that balance.

The original four DORA metrics

The original DORA framework uses four key measurements covering software delivery speed and stability.

Deployment frequency measures how often an organization deploys code to production. A higher deployment frequency can indicate that teams are able to make smaller changes and move them through the software delivery process regularly.

Lead time for changes measures the time between a code commit and its deployment to production. Long lead times can point to friction somewhere in the delivery process, including code reviews, testing, approvals, or deployment.

Change failure rate measures the percentage of deployments that fail. It provides a stability measure alongside deployment frequency and lead time. A team deploying frequently isn't necessarily performing well if a large proportion of those production deployments create problems that require intervention.

Time to restore service measures how long it takes to recover after a deployment failure. It is also commonly referred to as mean time to recovery. A shorter time to restore suggests that teams can identify a problem, fix it, validate the fix, and restore service quickly.

Taken together, these four key metrics prevent teams from optimizing for speed alone. Deployment frequency and lead time show how quickly changes move, while change failure rate and time to restore service provide information about the stability of that delivery process.

The fifth DORA metric: deployment rework rate

The DORA model has evolved beyond the original four metrics. It now includes a fifth metric: deployment rework rate.

Deployment rework rate looks at unplanned work that results from problems discovered after deployment. Instead of focusing only on whether a deployment failed outright, it helps capture the additional engineering work created when something reaching production needs to be corrected.

This is particularly relevant to software testing. A deployment may technically succeed but still introduce a defect that requires developers to stop planned work, fix bugs, test the correction, and deploy again.

Looking at all five DORA metrics therefore gives teams a broader picture of software delivery performance: how frequently they deploy, how quickly changes reach production, how often those changes fail, how quickly they restore service when something goes wrong, and how much unplanned rework production changes create.

Why DORA metrics matter for software testing

DORA metrics give QA teams a way to look beyond testing activity and ask a more useful question: is testing actually helping us deliver better software?

Metrics such as the number of test cases executed, bugs found, pass rate, or percentage of tests automated can tell you something about the testing process. But they don't necessarily tell you whether that process helps the organization deliver high-quality software quickly and reliably.

DORA metrics provide that wider perspective. They connect testing to software delivery performance, making it easier to see where QA helps the delivery process and where testing itself has become a bottleneck.

Testing sits directly between development and deployment

In many software development teams, testing is one of the final steps a change must pass before a production deployment.

A developer finishes a change, code reviews take place, automated tests run, QA performs any necessary manual or exploratory testing, and the change moves towards production. Testing is therefore closely connected to both sides of DORA performance: how quickly software can be delivered and how stable it is once deployed.

This also means problems in testing can show up in several DORA metrics at once.

A three-day regression cycle can increase lead time for changes and limit deployment frequency. Missing a critical failure scenario can increase change failure rate. If a production incident then requires another lengthy validation cycle before a fix can be released, testing can also affect time to restore service.

Looking at the entire software delivery process makes those relationships easier to see.

Faster testing can improve delivery speed

Testing doesn't need to disappear for software delivery to become faster. The feedback from testing needs to arrive sooner.

If automated tests run continuously as part of CI/CD, development teams can identify problems shortly after code changes are introduced. Developers can fix bugs while the change is still fresh rather than discovering them days later during a large regression cycle.

The same applies to test execution itself. Running suitable tests in parallel, preparing reusable test data, reducing environment setup time, and automating repetitive regression testing can all shorten the time between development and deployment.

Faster feedback can therefore contribute to shorter lead time and support a higher deployment frequency without simply removing quality checks from the delivery process.

Poor test coverage can increase change failures

A test suite can contain thousands of automated tests and still miss the scenario that matters most.

Test coverage is useful, but quantity alone doesn't show whether the highest-risk workflows are protected. A banking application, for example, might have extensive tests around its interface while missing an edge case that allows a duplicate payment or incorrectly handles an authorization failure.

If those problems aren't discovered until the production environment, they can contribute to failed deployments, emergency fixes, and deployment rework.

This is why test coverage needs to be considered alongside risk coverage. QA teams need to understand not only what has been tested, but which business-critical scenarios could fail and what the consequences would be.

For DORA metrics, that distinction matters. Testing another low-risk scenario may have little effect on delivery performance. Catching a critical defect before production can prevent an entire chain of recovery and rework.

Unreliable automation can slow the entire delivery pipeline

Automated testing is often introduced to make the development process faster, but unreliable automation can have the opposite effect.

A flaky test fails. Someone reruns it. It fails again. A developer investigates, only to discover that the application works correctly and the test itself is the problem.

Multiply that across a large test suite and automated testing can become a significant source of waiting time.

The same problem appears with high-maintenance UI automation. If small UI changes constantly break test scripts, teams spend time repairing automation instead of testing new functionality. Eventually, teams may stop trusting failed tests altogether.

That affects more than a QA metric. It can increase lead time, reduce the team's ability to make frequent deployments, and make incident validation slower.

Reliable automation therefore matters more than simply having a high automation percentage. The goal is not to implement automated testing everywhere. It is to automate where doing so creates faster, trustworthy feedback.

DORA metrics connect QA work to business and engineering outcomes

This is where DORA metrics become particularly useful for QA teams.

Consider two teams. One increases its automated test coverage from 60% to 80%. The other keeps roughly the same automation coverage but cuts its regression cycle from six hours to two, catches more critical failures before production, and reduces the amount of emergency rework after releases.

Looking only at automation percentage could make the first team appear more successful. Looking at software delivery outcomes tells a different story.

DORA metrics help shift the conversation from:

How many tests did we automate?

to:

Did testing help us deliver changes faster and more reliably?

That doesn't mean QA metrics should disappear. Test coverage, execution time, flaky test rates, escaped defects, and other engineering metrics can help explain why DORA performance changes.

The difference is that DORA metrics provide the wider outcome. They help QA teams connect their work to deployment speed, production stability, recovery, and continuous improvement rather than treating testing as an isolated stage of software development.

How software testing affects each DORA metric

Software testing is not measured directly by DORA metrics, but it can influence every one of them. Testing sits throughout the software delivery process, from the first code changes and code reviews to the final production deployment and, when something goes wrong, recovery.

This means testing can work in both directions. Reliable, well-placed tests give development teams faster feedback and more confidence to release. Slow test suites, lengthy manual processes, flaky automation, and poorly prioritized test coverage can become bottlenecks themselves.

Looking at each of the five DORA metrics makes that relationship clearer.

Deployment frequency

Deployment frequency measures how often code is deployed to production.

Testing affects deployment frequency because every release needs an appropriate level of confidence before it reaches users. If every production deployment requires several days of manual regression testing, increasing the number of deployments becomes difficult without increasing the workload of the QA team.

Automated testing can change that equation. Stable regression tests can run whenever code changes, while CI/CD integration allows teams to receive feedback without waiting for a separate testing phase. Tests can also run in parallel to reduce overall execution time.

For example, imagine a banking team releasing updates to its online payment flow. If checking login, beneficiary creation, payment authorization, transaction limits, and confirmation requires a two-day manual regression cycle, multiple deployments per day aren't realistic. If the repeatable parts of that regression testing are automated, testers can spend more time on new functionality, exploratory testing, and higher-risk scenarios.

But higher deployment frequency shouldn't simply mean removing testing gates. The aim is to make useful feedback available sooner so teams can deploy more frequently without sacrificing release confidence.

Lead time for changes

Lead time for changes measures how long it takes for a code commit to reach production. It reflects the efficiency of the wider CI/CD pipeline, not just how quickly developers write code.

Testing can account for a significant part of that lead time.

A change might be ready from a development perspective but spend hours or days waiting for a test environment, test data, a manual regression cycle, approval from QA, or the completion of a slow automated test suite. Flaky tests can make the problem worse by forcing teams to rerun tests or investigate failures that aren't caused by actual defects.

Consider a change that takes two hours to develop but three days to reach production. Looking only at development time would hide most of the delivery process. Measuring lead time encourages teams to look at everything between the initial code change and production, including:

• waiting for code reviews
• environment setup
• test data preparation
• automated test execution
• manual testing
• flaky test investigation
• approval and review processes
• deployment itself

Reducing lead time therefore doesn't necessarily mean testing less. It can mean implementing automated testing for repetitive checks, running appropriate tests earlier, improving test reliability, and removing unnecessary waiting between development and QA.

Change failure rate

Change failure rate measures the proportion of production deployments that fail or require intervention.

This is where the relationship between testing and DORA metrics is perhaps most obvious. Testing is one of the mechanisms software development teams use to identify problems before users encounter them.

But simply having a large number of test cases doesn't guarantee a low failure rate.

Coverage needs to reflect how the application actually works and where failures would have the greatest impact. Unit tests may confirm that individual components behave correctly, while integration testing checks how those components communicate. End-to-end testing can then validate whether complete user journeys still work when everything is connected.

Realistic test data and test scenarios matter too. A payment system might successfully process a standard €50 transfer while failing when a user reaches a transaction limit, when authorization expires halfway through the process, or when one downstream service becomes unavailable.

This is where risk-based testing becomes particularly relevant. Instead of treating every feature and test case as equally important, teams can prioritize the workflows where a production failure would cause the greatest damage.

A lower failure rate isn't necessarily the result of running more tests. Often, it comes from testing the right things before production.

Time to restore service

Time to restore service measures how long it takes teams to recover after a deployment failure. It is also commonly described as mean time to recovery.

Testing doesn't stop being useful once software reaches the production environment.

When an incident occurs, the engineering team first needs to understand what went wrong. Reproducing the failure in a controlled environment can help isolate the conditions that caused it. Once developers fix the problem, the change still needs to be validated before service can safely be restored.

A mature testing process can make this faster.

Reusable automated tests can verify that the original problem has been fixed and run regression checks against related functionality. Good test data can make production incidents easier to reproduce. Robust monitoring and incident management tools can provide additional information about where and when the failure occurred.

Imagine a banking release causes some international payments to fail after authentication. The team could patch the immediate error and deploy again, but that introduces the risk of creating another problem. A reusable automated test covering the affected payment journey allows the team to validate the fix and check related functionality before restoring service.

The goal isn't to turn an emergency fix into a full release cycle. It's to have enough reliable testing infrastructure already in place that validating a fix doesn't become another bottleneck during an incident.

Deployment rework rate

Deployment rework rate is the newer fifth DORA metric. It looks at unplanned work resulting from problems discovered after deployment.

This adds an important dimension to measuring DevOps performance because not every problematic deployment causes an obvious outage.

A release might technically succeed while introducing a bug that needs to be fixed the following morning. Another might require a patch, rollback, configuration adjustment, or follow-up production deployment. All of that work takes development and operations teams away from planned work.

Testing can reduce this rework by catching relevant problems earlier in the development process.

Suppose a new payment feature passes its basic functional tests and reaches production successfully. Later, customer feedback reveals that the flow fails for users with a particular account configuration. Developers now have to investigate the bug, reproduce it, write a fix, test that fix, complete another review, and deploy again.

The first deployment may have counted as successful, but the software delivery process still created avoidable work.

This is why deployment rework rate is particularly useful alongside change failure rate. One captures changes that result in failures requiring intervention, while the other helps teams see how much unplanned work production problems generate.

For QA teams, both point back to the same question: are we testing enough of the right risks before they become production problems?

DORA metrics vs traditional software testing metrics

Traditional software testing metrics tell QA teams what is happening inside the testing process. DORA metrics look further downstream at what happens to software delivery as a whole.

Both are useful, but they answer different questions.

A QA dashboard might show that 98% of tests passed and 80% of the regression suite is automated. Those numbers can help the team understand its testing process, but they don't tell you whether releases are becoming faster, whether production failures are decreasing, or whether developers are spending less time fixing problems after deployment.

DORA metrics provide that broader context. The most useful approach is not replacing QA metrics with DORA metrics, but connecting the two.

Why more tests don't necessarily mean better delivery

More test cases increase test quantity. They don't automatically increase test effectiveness.

Imagine a test suite grows from 2,000 to 3,000 tests. If those additional tests repeatedly check low-risk functionality while an important payment authorization scenario remains uncovered, the larger suite hasn't necessarily made the release safer.

It may even make delivery slower if those additional tests increase execution time and maintenance effort without providing useful new information.

This is why test counts are difficult to interpret in isolation. What matters is what those tests cover, which failures they can detect, how reliably they run, and whether the feedback arrives early enough for development teams to act on it.

For DORA performance, a smaller and faster test suite focused on important risks can sometimes contribute more than a huge suite built primarily to maximize coverage numbers.

Why 100% automation shouldn't be the goal

The same principle applies to automation.

A team could technically automate almost every test and still have poor software delivery performance. Automated tests can be slow, flaky, redundant, or expensive to maintain. Some testing activities also benefit from human judgement, particularly exploratory testing, usability testing, and investigating unfamiliar behaviour.

The goal should therefore be to automate testing where automation provides useful, repeatable feedback.

Regression testing is an obvious candidate. Stable checks around critical functionality can run continuously through CI/CD, reducing the amount of repetitive manual work required before each release.

A new feature with uncertain behaviour may initially benefit more from exploratory testing. Automating it immediately could mean spending time maintaining test scripts before the functionality itself has stabilised.

An automation percentage tells you how much is automated. DORA metrics can help you investigate whether that automation is actually supporting faster and more reliable delivery.

Test coverage vs risk coverage

Test coverage asks how much of the software has been tested. Risk coverage asks whether the things that could hurt the business most are being tested.

The distinction becomes important when testing time is limited.

Two releases might both report 90% test coverage. In the first, the missing 10% consists primarily of low-impact functionality. In the second, it includes a critical payment approval workflow. The percentage is identical, but the production risk isn't.

This is why teams should consider risk coverage alongside traditional test coverage.

Risk-based testing helps teams prioritize test cases according to factors such as the likelihood and potential impact of failure. Instead of treating every untested scenario as equally important, QA teams can direct more effort towards workflows where a defect could lead to production incidents, customer impact, regulatory problems, or costly rework.

That perspective also fits naturally with DORA metrics. The goal isn't simply to make the test suite larger. It's to catch the failures most likely to disrupt successful deployments.

How testing can improve DORA metrics

Improving DORA metrics through testing doesn't necessarily require more testing. Often, it means getting better feedback earlier and with less friction.

That can involve automating repetitive work, changing when tests run, reducing flaky tests, or prioritizing the scenarios that carry the greatest business risk.

Automate repetitive regression testing

Regression testing is necessary, but repeating the same stable checks manually before every release can add considerable time to the delivery process.

Automating appropriate regression tests allows them to run repeatedly without requiring the same manual effort each time. For frequently deployed applications, this can make a significant difference to how quickly a change can move towards production.

Start with workflows that are stable, frequently executed, and important enough to validate on most releases.

The objective isn't to automate the entire regression suite. It's to remove repetitive work that delays useful feedback.

Move appropriate tests earlier in development

A defect discovered shortly after a code change is usually easier to deal with than the same defect discovered at the end of a release cycle.

Teams can move appropriate unit, API, integration, and automated functional tests earlier in the development process so feedback reaches developers before a change has travelled far through the delivery pipeline.

That doesn't mean every type of testing should happen as early as possible. End-to-end validation, exploratory testing, and checks requiring complete environments may still happen later.

The aim is to avoid waiting until the final QA stage to discover problems that could reasonably have been caught earlier.

Run tests continuously through CI/CD

Automated tests become particularly useful when they're connected to the software delivery process rather than run separately from it.

CI/CD integration can trigger appropriate tests when code changes are committed, merged, or prepared for deployment. Instead of waiting for someone to manually start a regression suite, teams receive feedback as part of the normal development workflow.

This can help identify failures before they accumulate into a large batch of changes.

It also creates a clearer relationship between code changes and test results, which can make failures easier to investigate.

Use parallel test execution

A test suite containing hundreds or thousands of tests doesn't necessarily need to execute sequentially.

Where tests can run independently, parallel test execution can reduce the amount of time teams wait for results. A suite that takes several hours sequentially may provide feedback considerably sooner when workloads are distributed.

But parallelisation shouldn't be used to hide an unnecessarily large or inefficient suite. Teams should still remove redundant tests and make sure expensive end-to-end tests are providing enough value to justify their execution time.

Reduce flaky tests

A flaky test can pass and fail without a meaningful change to the functionality it is supposed to validate.

That creates noise in the delivery pipeline.

Developers rerun tests, QA investigates failures, deployments wait, and eventually teams may begin assuming that certain failures can simply be ignored. At that point, automation stops providing trustworthy feedback.

Reducing flaky tests can therefore help with more than test reliability. It removes unnecessary investigation and waiting from the software delivery process.

A failed test should mean something.

Test critical user journeys end to end

Lower-level tests are valuable, but they can't always tell you whether a complete user journey works when all the pieces come together.

End-to-end testing is particularly useful for critical workflows involving multiple systems or dependencies.

In banking, that could mean testing a payment from login through beneficiary selection, authorization, processing, and confirmation. Each individual service might behave correctly while the complete journey still fails because of an integration problem.

The goal isn't to turn every possible scenario into an end-to-end test. These tests tend to be slower and more complex than lower-level tests. Reserve them for journeys where validating the complete flow provides meaningful additional confidence.

Prioritize testing based on business risk

When teams have limited time before a release, test prioritization matters.

Risk-based testing helps determine which workflows deserve the most attention based on the likelihood and potential impact of failure.

For example, a minor layout problem and a broken payment authorization flow shouldn't automatically receive the same testing priority simply because both changed in the latest release.

This is also where Leonidas can help. TestResults' risk assessment tool analyzes software specifications to identify and score risks, helping teams understand which workflows deserve more testing attention and where important gaps in risk coverage remain.

That allows testing effort to focus on what could have the greatest effect on production stability rather than simply executing as many tests as possible.

Keep automated tests maintainable

Automation only saves time while the cost of maintaining it remains reasonable.

This becomes particularly important with GUI testing. Traditional UI automation can depend heavily on technical selectors. When an element ID, page structure, or other implementation detail changes, the test can fail even though the workflow still works perfectly for the user.

Every unnecessary failure adds investigation and maintenance to the delivery process. If it happens frequently enough, automation itself starts increasing lead time.

TestResults' Visual Sense technology approaches UI automation differently. It allows tests to recognize and interact with applications more like a user would, rather than relying solely on brittle selectors.

That means a routine UI change doesn't necessarily require a test repair when the element is still visible and the user journey still behaves correctly.

For teams trying to improve DORA metrics, maintainability is therefore part of automation performance. The best automated test isn't simply one that runs. It's one that continues providing trustworthy feedback without becoming another bottleneck in the software delivery process.

How testing can improve DORA metrics

Improving DORA metrics through testing doesn't necessarily mean adding more tests. In many cases, the bigger opportunity is making testing faster, more reliable, and better focused on the changes that could actually cause problems in production.

The aim is to shorten feedback loops without sacrificing confidence. That can help improve deployment frequency and lead time while keeping change failure rate and deployment rework under control.

Automate repetitive regression testing

Regression testing is one of the clearest opportunities for automation because the same checks often need to run after every significant code change.

If QA teams manually repeat login, navigation, payments, permissions, and other established workflows before every release, testing can quickly become one of the longest stages in the software delivery process.

Automating stable, repetitive regression tests means they can run without requiring the same manual effort for every release. QA engineers can then spend more time on new functionality, exploratory testing, and scenarios that require human judgement.

The goal isn't to automate everything. Start with tests that are repeated frequently, provide useful release confidence, and are stable enough to justify automation.

Move appropriate tests earlier in development

The earlier a useful test provides feedback, the sooner a problem can be addressed.

Unit tests, API tests, integration tests, and appropriate functional tests can run while code changes are still being developed or reviewed. This reduces the chance that relatively simple defects travel through the entire development process before being discovered during final regression testing.

It can also shorten lead time. Instead of handing a large batch of changes to QA and discovering several problems at once, development teams receive feedback continuously and can fix bugs while the relevant code is still fresh.

Not every test needs to move earlier. Some end-to-end, exploratory, or environment-specific testing naturally happens later. The point is to avoid postponing tests that could provide useful feedback sooner.

Run tests continuously through CI/CD

Automated testing becomes more useful when it's part of the delivery pipeline rather than a separate step someone has to remember to start.

Teams can trigger different tests when code is committed, merged, built, or prepared for a production deployment. Faster checks can run frequently, while larger regression suites can run at appropriate stages of the pipeline.

This gives developers feedback closer to the code changes that caused a failure and reduces the amount of manual coordination required between development and QA.

For teams trying to improve deployment frequency, this is particularly important. Frequent deployments become difficult if every release still depends on a separate, lengthy testing process.

Use parallel test execution

Even automated testing can become a bottleneck when tests take too long to run.

Where tests are independent, running them in parallel can reduce overall execution time. Instead of waiting for one test to finish before the next begins, teams can execute multiple parts of the test suite simultaneously and receive results sooner.

That can make a significant difference for large regression suites, particularly when testing needs to happen several times throughout CI/CD.

Parallel execution isn't a substitute for maintaining the test suite, though. If thousands of redundant or low-value tests are slowing the pipeline down, simply adding more execution capacity doesn't solve the underlying problem.

Reduce flaky tests

A test that sometimes passes and sometimes fails under the same conditions creates work without providing reliable information.

Every flaky failure has a cost. Someone needs to rerun the test, investigate the result, decide whether the application actually has a defect, and potentially delay a deployment while doing so.

Over time, another problem appears: teams stop trusting the test suite.

Reducing flaky tests therefore matters for both software quality and delivery speed. Automated tests should give development teams a clear signal. If a test fails, there should be a good reason to investigate it.

Test critical user journeys end to end

Not every problem can be found by testing individual components.

A payment application might pass its unit and API tests while the complete user journey still fails because of an issue between authentication, payment processing, and the confirmation interface.

End-to-end tests can validate these critical journeys from the user's perspective and catch problems that only appear when several systems interact.

But this doesn't mean every scenario needs an end-to-end test. These tests can be slower and more complex to maintain, so they are most useful for workflows where a failure would have a meaningful impact.

For a bank, that might include logging in, making a payment, approving a transaction, changing account permissions, or completing another business-critical process.

Prioritize testing based on business risk

When release time is limited, treating every test case as equally important isn't particularly useful.

Risk-based testing helps teams focus first on workflows where a failure would have the greatest potential impact. A cosmetic issue on a low-traffic page and a defect that causes duplicate payments shouldn't receive the same testing priority.

This is where Leonidas can help. Teams can upload software specifications, identify and score risks, map test cases to those risks, and see where important gaps in coverage remain.

Instead of deciding what to test based primarily on the size of the existing test suite, teams can prioritize the risks that matter most before a production deployment.

That can support DORA performance from both directions: keeping testing focused enough that it doesn't unnecessarily delay delivery, while giving high-risk functionality enough attention to reduce the likelihood of production failures.

Keep automated tests maintainable

Automated testing only saves time if teams don't spend that time repairing the automation instead.

This is particularly relevant for GUI testing. Traditional UI automation often relies heavily on technical selectors. If a developer changes an element ID, reorganizes the DOM, or makes another routine interface change, an automated test can fail even though the application still works exactly as expected from the user's perspective.

Those failures create maintenance work and can increase lead time if they block the delivery pipeline.

TestResults addresses this with Visual Sense technology. Rather than relying solely on brittle selectors, Visual Sense allows automated tests to recognize and interact with applications more like a human user would.

If the underlying UI code changes but the interface still looks and behaves correctly for the user, the test is less likely to fail simply because a technical locator changed.

That distinction matters when thinking about DORA metrics. Automation shouldn't just increase the number of tests a team can run. It should provide fast, reliable feedback without becoming another source of delivery friction.

When testing starts hurting your DORA metrics

Testing is supposed to reduce uncertainty before software reaches production. But there is a point where the testing process itself can start creating delivery problems.

A test suite that takes hours to run, automation nobody trusts, or a release process that depends on days of manual validation can increase lead time and make frequent deployments difficult. Meanwhile, having lots of tests doesn't necessarily prevent change failures if those tests aren't covering the workflows that carry the most risk.

Here are some signs that testing may be working against your DORA metrics rather than supporting them.

Regression suites take hours to finish

A regression suite can grow quietly.

A new feature gets 20 tests. A bug gets another five. Old tests remain because nobody wants to be the person who deletes something important. A few years later, every release has to wait for thousands of tests to finish.

The problem isn't simply that the suite is large. It's whether the time spent running it produces useful information.

If a regression suite adds hours to every release, look at which tests actually need to run, which can run in parallel, which duplicate existing coverage, and which could run at a different point in the CI/CD pipeline.

More tests aren't automatically better if most releases spend hours waiting for them.

Every release requires a large manual testing cycle

Manual testing remains valuable, particularly for exploratory work and new or unpredictable functionality.

It becomes a delivery bottleneck when the same predictable checks have to be repeated manually before every release.

If every production deployment requires someone to spend a day clicking through the same 50 workflows, deployment frequency will inevitably depend on QA capacity. A release may be technically ready on Tuesday but sit waiting until Thursday because testers are still completing the checklist.

Stable, repetitive checks are usually the first place to consider automation. Human testing can then focus on the scenarios where judgement actually adds value.

Flaky tests repeatedly block pipelines

One flaky test is annoying. Hundreds of flaky tests can change how an engineering team works.

A pipeline fails, so somebody reruns it. It passes the second time, so everyone moves on. Eventually, rerunning failed tests becomes part of the unofficial deployment process.

That costs time, but the bigger problem is trust.

If developers become accustomed to automated tests failing for reasons unrelated to code quality, a failed test stops being a useful signal. Teams either investigate failures that aren't real or start ignoring failures that might be.

Neither is particularly helpful for fast, reliable software delivery.

UI changes constantly break automated tests

UI automation can become especially expensive when tests depend on implementation details that change frequently.

A developer updates a component, changes an element ID, or restructures part of the interface. The user journey still works, but dozens of automated tests fail because their selectors no longer match.

QA then has to determine whether each failure represents a real defect or simply broken automation.

If this happens after routine UI changes, the automation itself starts adding to lead time.

This is one reason TestResults uses Visual Sense. Rather than depending solely on technical selectors, tests can recognize and interact with the interface more like a user would. A technical UI change doesn't necessarily need to break a test when the application still looks and behaves correctly.

Teams automate low-risk scenarios while critical workflows remain uncovered

Automation can look impressive on a dashboard while providing surprisingly little protection against production failures.

A team might have hundreds of automated tests around simple forms and navigation but very little coverage of the workflow responsible for moving money, granting permissions, or processing sensitive customer information.

The automation percentage goes up. The important risk hasn't changed.

This is why test prioritization matters. Teams should understand which workflows carry the greatest business risk before deciding where additional automation will create the most value.

Leonidas can support that process by identifying and scoring risks from software specifications and mapping test cases against them. The aim is to see whether important risks are actually covered, rather than simply whether another test has been automated.

Test environments become a delivery bottleneck

Sometimes the tests aren't slow. Getting somewhere to run them is.

Teams may need to wait for a shared test environment, manually configure services, create test accounts, prepare data, or reset the environment after another team has used it.

All of that waiting contributes to the delivery process even though it may not appear in test execution time.

If a ten-minute automated test requires three hours of preparation and waiting, optimizing the test itself won't meaningfully reduce lead time.

When investigating testing bottlenecks, teams therefore need to look at the full path from code change to test result, including environments and test data.

Failed tests require too much investigation

A useful failed test should make it reasonably clear what went wrong.

If every failure requires someone to search through logs, reproduce the entire workflow manually, inspect screenshots, compare environments, and determine whether the problem came from the application, test data, infrastructure, or the test itself, the cost of automation can become surprisingly high.

This investigation time is easy to overlook because it isn't technically "test execution time." But from the perspective of the software delivery process, the distinction doesn't matter. The deployment is still waiting.

Tracking how often automated tests fail for non-product reasons, how long failures take to diagnose, and which tests repeatedly require investigation can reveal bottlenecks that a simple pass-rate metric won't show.

Ultimately, testing isn't automatically improving delivery simply because more of it is automated.

The useful question is whether testing gives the team trustworthy information quickly enough to make better release decisions. If automation adds hours of waiting, creates constant maintenance work, or concentrates coverage on low-risk functionality while critical workflows remain untested, a higher automation percentage can coexist with worse software delivery performance.

DORA metrics and risk-based testing

DORA metrics tell teams what is happening to software delivery. Risk-based testing helps decide where testing effort should go before those delivery problems happen.

The connection matters because improving software delivery isn't simply about running more tests. A team can have a huge test suite and still miss the one failure that causes a production incident, an emergency fix, and another deployment.

Not every change carries the same risk

A typo on a settings page and a change to payment authorization shouldn't receive the same level of testing.

The likelihood of failure may differ, but so does the potential impact. A defect in a low-traffic interface might inconvenience a handful of users. A defect that creates duplicate payments, exposes customer data, or incorrectly handles account permissions could have much more serious consequences.

Yet test suites don't always reflect that difference. Teams often accumulate test cases over time and run them because they exist, rather than because they represent the most important current risks.

That can create an odd situation: thousands of tests pass, but the team still isn't confident about the release.

Prioritize high-risk workflows before release

Risk-based testing starts by identifying what could go wrong and prioritizing testing accordingly.

For a banking release, that might mean giving more attention to authentication, payment processing, account permissions, transaction limits, approval workflows, or customer data than to a low-impact interface change.

This doesn't mean ignoring lower-risk functionality. It means making sure limited testing time is spent where a failure would matter most.

That can also support DORA performance. If teams identify critical defects before production, they can potentially avoid failed deployments, emergency recovery work, and additional production rework. At the same time, they don't need to make every release wait for an unnecessarily large regression suite covering every scenario with equal priority.

Connect software specifications to testing priorities

One challenge is identifying those risks systematically.

Requirements and software specifications already contain information about what an application is expected to do. They describe business rules, workflows, permissions, integrations, exceptions, and other behavior that can reveal where failures might occur.

Instead of moving directly from a specification to a list of test cases, teams can add another question:

What are the risks in this specification, and which ones matter most?

That gives QA teams a clearer basis for deciding what deserves deeper coverage before a production deployment.

This is where Leonidas, TestResults' risk assessment tool, comes in. Teams can upload their software specifications to Leonidas, identify and score potential risks, and map test cases against those risks.

The result isn't simply another list of tests. It gives teams a way to see which important risks are covered, where gaps remain, and which workflows deserve more testing attention before release.

Measure coverage based on risk, not test quantity

Traditional test coverage can answer useful questions, but it can also create false confidence when it's reduced to a percentage.

Having 95% coverage sounds reassuring until the missing 5% contains the workflow capable of causing the most damage.

Risk coverage adds context. Instead of asking only how many requirements, lines, branches, or scenarios are covered, teams can ask whether the risks with the greatest potential impact are actually being tested.

That changes the conversation from:

How many tests do we have?

to:

If this release fails, have we tested the things most likely to matter?

For teams using DORA metrics, that distinction is useful. DORA shows whether the software delivery process is moving quickly and reliably. Leonidas can help teams decide what deserves attention before a change reaches production in the first place.

DORA metrics and test automation

Risk prioritization answers what should we test? Automation addresses another question: how can we test it repeatedly without turning testing into the bottleneck?

Good test automation can provide fast feedback throughout software development. Bad automation can add another layer of waiting, investigation, and maintenance to the delivery process.

The difference matters when measuring DORA metrics.

Where automation improves delivery performance

Automation works particularly well for checks that are repetitive, predictable, and frequently needed.

Regression testing is an obvious example. Instead of manually repeating the same critical workflows before every release, automated tests can run when code changes and provide feedback earlier in the CI/CD pipeline.

That can reduce manual testing time, shorten feedback loops, and make continuous delivery more practical.

Automation can also support incident recovery. If a production defect needs an urgent fix, existing automated tests can validate the corrected workflow and check related functionality without rebuilding the validation process from scratch.

In both cases, the benefit isn't simply that a test is automated. It's that useful information reaches the team sooner.

Where automation creates maintenance overhead

Automation isn't free.

Test scripts need to be created, maintained, investigated when they fail, and updated as applications evolve. That becomes particularly noticeable with UI automation.

Traditional automated UI tests often depend on technical locators tied to the structure of the interface. A button changes location, an element ID changes, or the underlying UI code is reorganized, and a test can fail even though a user would still have no problem completing the task.

Each failure then needs investigation.

Was there a real regression? Did the interface intentionally change? Is the selector outdated? Does the test need to be rewritten?

When that happens frequently across a large test suite, automation starts creating exactly the kind of delivery friction it was supposed to remove.

Why reliable automation matters more than automation percentage

This is why automation percentage isn't particularly useful on its own.

A team with 90% automated testing isn't necessarily performing better than one with 60%. The first team might have hundreds of flaky, slow, high-maintenance tests. The second might have a smaller automated suite focused on critical workflows that provides trustworthy results in minutes.

From a DORA perspective, the second situation can be considerably more useful.

The question isn't:

How much have we automated?

It is:

Does our automation give us reliable feedback quickly enough to make better release decisions?

That means looking at test reliability, execution time, maintenance effort, risk coverage, and how often automation genuinely catches problems alongside the headline automation percentage.

Testing like a real user with Visual Sense

This is one of the problems TestResults' Visual Sense technology is designed to address.

Instead of relying solely on brittle selectors to locate elements, Visual Sense allows automated tests to recognize and interact with an application more like a human user would.

If the underlying UI code changes but a button is still visible and usable from the user's perspective, the test doesn't necessarily need to fail simply because a technical locator changed.

That can make a meaningful difference in applications where interfaces change frequently. Routine UI updates are less likely to create hours of test maintenance that delay the software delivery process.

Combined with risk-based testing, the idea is straightforward:

Leonidas helps identify what matters most. Visual Sense helps automate those critical user journeys in a way that reflects how users actually experience them.

For teams tracking DORA metrics, that's a more useful goal than simply adding more automated tests. The aim is to test the right risks while keeping that testing reliable enough that it supports delivery rather than slowing it down.

How to use DORA metrics with your QA team

Implementing DORA metrics doesn't mean giving QA another dashboard to monitor. The useful part is connecting changes in software delivery performance with what is actually happening during development, testing, and deployment.

For QA teams, that means looking at DORA alongside testing data. If lead time increases, testing data might help explain why. If change failure rate decreases, you can investigate whether better coverage, different release practices, smaller changes, or another improvement contributed.

The aim is to gain valuable insights into the delivery process, not to turn every movement in a DORA metric into a QA KPI.

Establish your current delivery baseline

Before changing the testing process, establish how software is currently being delivered.

Measure DORA metrics consistently across a meaningful period. Look at deployment frequency, lead time for changes, change failure rate, time to restore service, and deployment rework rate.

Data collection will usually involve information from several systems. Version control can show when code changes were committed, CI/CD systems can provide build and deployment information, and incident management tools can help track production failures and recovery.

The baseline doesn't need to be perfect. It needs to be consistent enough that the engineering team can see how its delivery process behaves over time.

Find where testing contributes to delays or failures

Once you have a baseline, look at what happens between a code change and a successful production deployment.

If lead time is high, break that time down. How much is spent waiting for a test environment? How long does automated test execution take? Does every release require hours of manual regression testing? How often do flaky tests cause reruns?

The same approach works for stability metrics.

If change failure rate is increasing, examine the failures. Were they scenarios that could reasonably have been caught before production? Were the affected workflows covered by tests? Did the tests use realistic data? Was the problem caused by an interaction between systems that wasn't included in the test plan?

This is much more useful than assuming that testing is either working or failing based on one number.

Connect QA metrics to DORA metrics

DORA metrics tell you about the outcome. QA metrics can help explain what is happening underneath it.

For example, you might track:

The important word here is investigating. If regression execution time falls and deployment frequency later increases, that doesn't automatically prove one caused the other.

DORA metrics reflect the entire software delivery system. Development practices, code reviews, infrastructure, deployment processes, team size, architecture, and many other factors can change at the same time.

Identify the highest-risk bottlenecks

Not every bottleneck deserves the same amount of attention.

A five-minute delay in a low-risk test suite might matter less than a payment workflow that regularly reaches production without adequate coverage. QA teams should consider both delivery friction and business risk when deciding what to improve.

This is also where risk-based testing becomes useful. Instead of asking only which tests take the longest?, teams can ask which testing problem creates the greatest risk for delivery?

That might be a slow regression suite. It could also be missing coverage around a critical workflow, unreliable UI automation, unrealistic test data, or an integration that repeatedly causes production incidents.

Make one testing change at a time

Once you've identified a problem, make the improvement specific enough to evaluate.

Instead of deciding to "automate more," for example, automate a repetitive regression flow that currently takes several hours manually. Instead of trying to improve the entire test suite, address the group of flaky tests that repeatedly blocks production deployments.

Smaller changes make it easier to understand what changed in the testing process and what happened afterwards.

They also prevent teams from spending months rebuilding their QA process before knowing whether they're solving the right problem.

Measure the effect over multiple releases

Don't judge the result from the next deployment.

One unusually fast release doesn't mean lead time has permanently improved, just as one failed deployment doesn't mean the testing change was unsuccessful.

Measure performance over multiple releases and look for patterns. Did test execution become consistently faster? Are fewer releases waiting for manual validation? Are production failures involving the targeted workflow becoming less common?

DORA metrics support this kind of continuous improvement because they provide a consistent view of delivery performance over time.

The conclusion should still be cautious. You can say that a testing improvement coincided with or contributed to better delivery performance when the evidence supports it. You usually can't say that changing one test process caused a DORA metric to improve when the entire software delivery system was changing around it.

Example: how testing affects DORA metrics in banking

Consider a bank making what appears to be a small change to its online banking application.

The change: the team updates the payment confirmation screen.

The backend payment logic hasn't intentionally changed. From a development perspective, this looks primarily like a UI update. But the confirmation screen sits at the end of a critical user journey involving account selection, beneficiary details, payment amounts, authentication, transaction limits, payment processing, and confirmation.

That makes the testing around the change much more important than its size might suggest.

Scenario 1: testing becomes the bottleneck

The change is finished quickly, but the bank's regression suite takes several hours to complete.

Some automated UI tests then fail because the confirmation screen has changed. The application still works, but the tests rely on selectors tied closely to the previous UI structure. QA has to investigate the failures, update the test scripts, and rerun parts of the suite.

Lead time increases before the change has even reached production.

There is another problem. The test suite contains plenty of test cases, but payment-risk scenarios haven't been prioritized. The standard payment journey works, so the release goes ahead.

After deployment, customers discover that payments above a particular threshold can be processed successfully but don't display the correct confirmation status.

Now the team has a production incident.

Developers investigate and prepare a hotfix. QA needs to reproduce the issue, validate the fix, run another regression cycle, and approve another production deployment.

One relatively small UI change has now affected several DORA metrics. Testing delays contributed to longer lead time. The production defect affects change failure rate. Validating and deploying the hotfix affects time to restore service. The additional work can also contribute to deployment rework rate.

Scenario 2: testing is prioritized around risk

Now take the same change with a different testing approach.

Before release, the team identifies the payment confirmation journey as high risk because a failure could leave customers unsure whether money has actually been transferred. Testing effort is prioritized accordingly, rather than treating every test case as equally important.

Automated regression tests cover the critical payment journey and run as part of CI/CD. The UI automation is also less dependent on brittle technical selectors, so changing the structure of the confirmation screen doesn't automatically break tests when the user journey still behaves correctly.

The team tests the normal payment flow as well as relevant risk scenarios, including transaction thresholds and different payment outcomes.

The threshold problem is found before production.

The developers fix it while the change is still being worked on, the relevant tests run again, and the release proceeds without requiring an emergency hotfix afterwards.

That doesn't mean better testing single-handedly "improved the DORA metrics." But it shows how risk-prioritized, maintainable testing can remove delivery friction while catching important failures earlier.

For a bank, that's ultimately more useful than simply being able to say that it ran 5,000 tests before release.

Common mistakes when using DORA metrics for testing

DORA metrics can help QA teams understand how testing contributes to software delivery, but they become much less useful when they're treated as a scoreboard.

The goal isn't to chase the highest deployment frequency or compare which QA team has the shortest lead time. DORA metrics support continuous improvement by helping teams understand their delivery process, identify bottlenecks, and investigate what might be slowing them down or creating instability.

Here are some of the most common ways teams get that wrong.

Treating DORA metrics as individual QA performance metrics

DORA metrics measure the performance of a software delivery system, not the performance of individual testers.

Lead time, for example, can be affected by development time, code reviews, testing, approvals, infrastructure, deployment processes, and dozens of other factors. Change failure rate is similarly influenced by decisions made across development and operations teams.

Holding an individual QA engineer responsible for either metric ignores how software development actually works.

Instead, teams should use DORA metrics to start conversations. If lead time increases, where is the additional time being spent? If the failure rate rises, what types of failures are reaching production? Testing might be part of the answer, but it shouldn't automatically be treated as the cause.

Optimizing deployment frequency at the expense of stability

Deployment frequency measures how often code reaches production, but more deployments aren't automatically better.

A team could improve deployment frequency while simultaneously increasing its change failure rate and deployment rework rate. In that situation, looking at deployment frequency alone would give a misleading picture of the team's performance.

The DORA metrics are designed to balance delivery speed with stability. A useful improvement allows teams to release more efficiently without creating an unacceptable increase in production failures.

That is also why removing tests simply because they slow down a pipeline is rarely a meaningful optimization. The better question is whether those tests provide useful information, and whether that information could be delivered faster.

Assuming more automation automatically improves DORA metrics

Automating 90% of a test suite doesn't automatically mean the software delivery process is more efficient than when 60% was automated.

Automation only helps when the tests provide reliable, timely feedback.

A large automated test suite that takes six hours to execute, regularly produces false failures, or requires constant maintenance can actually increase lead time. Teams may spend more time waiting for tests, rerunning them, and investigating automation failures than they previously spent on manual testing.

When implementing automated testing, focus on the effect it has on the delivery process rather than automation percentage alone.

Measuring test quantity instead of testing effectiveness

Running 10,000 tests sounds impressive. It doesn't tell you whether the tests cover the scenarios most likely to cause serious problems.

The same applies to metrics such as test case count, number of bugs found, or number of successful test executions. They measure activity, but activity isn't necessarily the same as effectiveness.

A smaller test suite covering authentication, payments, permissions, data integrity, and other critical user journeys may provide more useful protection than thousands of low-risk checks.

DORA metrics can help shift the focus from how much testing happened to what effect testing had on software delivery.

Ignoring flaky and high-maintenance tests

A flaky test isn't just a testing problem when it runs inside the delivery pipeline.

Every false failure can trigger another test run, an investigation, or a delay while someone determines whether the application or the test is responsible. High-maintenance tests create a similar problem when routine application changes require frequent updates to test scripts.

Over time, teams may also stop trusting failures from the automated test suite.

When trying to improve deployment frequency or reduce lead time, it is therefore worth looking at test reliability alongside test execution speed. A fast test suite isn't particularly useful if developers don't trust its results.

Looking at one metric in isolation

The DORA metrics make more sense together.

Higher deployment frequency might look positive until you discover that change failure rate has increased. A low failure rate might look excellent until you realize the organization deploys once every three months. A short time to restore service is useful, but repeatedly restoring service after preventable failures isn't the same as avoiding those failures.

The same applies when evaluating changes to testing.

If a team removes part of its regression suite and lead time falls, that's only part of the story. The other DORA metrics can help show whether the faster delivery process remained stable.

Rather than optimizing one number, teams should look for patterns across the metrics and use those patterns to identify bottlenecks and guide continuous improvement.

Comparing teams without considering context

DORA metrics can be useful for understanding how a team's performance changes over time. They are much less useful as a simple leaderboard between teams working in very different environments.

A team maintaining an internal application doesn't necessarily face the same testing requirements as one responsible for payment infrastructure. Different products have different architectures, release processes, regulatory requirements, risk profiles, and definitions of a successful deployment.

The same applies to industry benchmarks. References to elite performers or high-performing teams can provide context, but they shouldn't become arbitrary targets that every engineering team is expected to reach regardless of what it builds.

The most useful comparison is often with your own delivery process over time. Measure DORA metrics consistently, understand what sits behind them, and use changes in the data to ask better questions about where your software delivery process can improve.

DORA metrics don't tell you everything about software quality

DORA metrics are useful because they connect engineering work to real software delivery outcomes. But they aren't a complete definition of software quality.

A team can deploy frequently, maintain a short lead time, recover quickly from failures, and still ship software that is difficult to use, inaccessible, insecure, or poorly aligned with what customers actually need.

DORA metrics should therefore be treated as one part of the picture, not the final score for whether your software is "good."

They measure software delivery performance, not complete product quality

The five DORA metrics focus on what happens as software moves towards and into production: how frequently teams deploy, how quickly changes get there, how often deployments cause problems, how quickly service is restored, and how much rework those deployments create.

Those are important outcomes. But a successful deployment isn't necessarily a successful product change.

A feature can reach production without triggering an incident and still confuse users. A new workflow can work exactly as specified while creating unnecessary friction. A page can function correctly while being inaccessible to part of its audience.

DORA metrics won't necessarily reveal those problems because that's not what they were designed to measure.

Accessibility, usability and security need additional measures

Software quality has dimensions that require their own testing practices and metrics.

Accessibility testing can identify barriers for users with disabilities. Usability testing can show whether people can actually understand and complete a workflow. Security testing can uncover vulnerabilities that might never appear in deployment frequency or lead time.

The same applies to performance, reliability, customer experience, and other quality characteristics.

DORA metrics can tell you a lot about the health of the software delivery process. They can't replace the broader set of signals QA and engineering teams need to understand the quality of the product being delivered.

Test coverage still needs context

Test coverage can add another layer to the picture, but it also needs context.

A high coverage percentage doesn't tell you whether the most important functionality is protected. Two teams can report similar coverage while carrying completely different levels of production risk.

This becomes especially important for complex applications. Covering another low-impact settings page doesn't necessarily provide the same value as covering a payment authorization flow, access-control rule, or integration responsible for sensitive customer data.

Instead of asking only how much is covered, teams should also ask what is covered and what happens if the uncovered parts fail.

Business risk needs to be measured separately

DORA metrics can show that a deployment failed. They don't inherently tell you whether that failure affected an internal reporting page or prevented thousands of customers from making payments.

Those incidents have very different business consequences.

Risk therefore needs its own place in testing decisions. Teams need to consider the likelihood of a failure alongside its potential impact on customers, revenue, data, compliance, and critical business processes.

This is where risk-based testing complements DORA particularly well. DORA helps teams understand the performance of the software delivery system. Risk assessment helps them decide where failures would matter most and where testing effort should be concentrated.

From "how many tests did we run?" to "did testing help us ship safely?"

It's easy to measure testing by counting things.

How many test cases did we run? How many passed? How many bugs did we find? What percentage of the regression suite is automated?

Those numbers aren't useless, but none of them answers the bigger question:

Did testing help us ship safely?

That's where DORA metrics give QA teams a different perspective. Testing isn't successful simply because the test suite gets larger or the automation percentage moves closer to 100%. It is useful when it helps teams get trustworthy feedback sooner, catch important problems before production, recover quickly when failures happen, and avoid unnecessary rework afterwards.

That doesn't mean optimizing every release for maximum speed. The goal is to release changes efficiently without increasing production risk.

And that requires knowing what deserves your attention in the first place.

Leonidas helps with the "what". Upload your software specifications, identify and score risks, map test cases against them, and see which critical workflows still need coverage.

Visual Sense helps with the "how". Once you know which user journeys matter, TestResults can automate them in a way that interacts with the application more like a real user, without relying solely on brittle selectors that turn routine UI changes into test maintenance.

Together, the idea is pretty simple: understand what carries the most risk, then test those journeys without making testing another delivery bottleneck.

Because the goal was never to have the biggest test suite.

It's to ship software that works when it matters.