A payment leaves the customer’s account. The processing provider loses its connection before confirming the transaction. Your banking application shows “pending,” the ledger shows a debit, and customer support cannot tell whether the customer should try again.
Each application may have passed its release tests. The recovery plan may be documented. What the bank still needs to establish is whether the complete payment process can recover without losing money, creating duplicate transactions or leaving customers with contradictory information.
DORA resilience testing gives financial institutions a framework for examining these weaknesses. For QA teams, the practical work starts with identifying critical services, tracing their dependencies and testing what happens when those dependencies fail.
Short summary
- DORA has applied since 17 January 2025 and covers EU financial entities, with separate oversight for designated critical ICT providers.
- Under Article 24, entities other than microenterprises need a risk-based testing programme, independent testers and a documented process for fixing and validating findings.
- At least yearly, those entities must test ICT systems and applications that support critical or important functions.
- Article 25 methods include vulnerability scans, performance and end-to-end testing, scenario-based exercises, penetration testing and, where feasible, source code reviews.
- A useful payment scenario tests an ambiguous provider timeout: prevent unsafe retries, establish the transaction’s final status, reconcile provider records with the ledger and show customers accurate information.
- Recovery testing should trace dependencies across internal systems and external providers, including delayed confirmations, duplicate messages and queued transactions.
- TLPT has a narrower scope: selected entities must conduct it at least every three years on live production systems, covering critical functions and relevant outsourced dependencies.
What is DORA resilience testing?
The Digital Operational Resilience Act (DORA) is the EU regulation addressing digital operational resilience across the financial sector. Applicable since 17 January 2025, it sets requirements for ICT risk management, incident reporting, digital operational resilience testing and ICT third party risk.
Banks, insurers, investment firms, payment institutions and other financial entities fall within its scope, subject to the regulation’s definitions and exemptions. DORA also establishes an oversight framework for designated critical ICT third-party service providers.
DORA testing examines an organization’s preparedness for ICT related incidents and helps identify weaknesses in technology, processes and controls. The findings must feed into remediation and validation. For financial organizations, this makes testing an ongoing part of the ICT risk management framework.
What are the DORA testing requirements?
Article 24 of DORA requires financial entities other than microenterprises to establish, maintain and review a comprehensive testing programme. The risk based approach considers the organization’s ICT risk profile, changes in cyber threats and the criticality of information assets and services.
Financial entities must use independent internal or external testers. Internal testing requires sufficient resources and safeguards against conflicts of interest. The resilience testing requirements also include procedures to classify, prioritize and remedy findings, followed by internal validation that identified gaps have been addressed.
This connects DORA compliance evidence to a working process. A useful record links the risk assessment, testing methodology, observed test results, corrective action and validation outcome.
How often must financial institutions test critical systems?
Under Article 24(6), financial entities other than microenterprises must ensure that appropriate tests are conducted at least yearly on all ICT systems and applications supporting critical or important functions.
Annual testing is therefore a minimum requirement for that scope. Testing plans should also reflect changing operational risks, system changes and emerging threats. Regular resilience testing helps teams revisit assumptions as the ICT infrastructure changes.
Which types of digital resilience testing belong in the programme?
Article 25 lists methods for selecting appropriate tests in proportion to the entity’s circumstances. Testing programs can combine several methods to examine different failure modes.
| Test method | Practical question it can answer |
|---|---|
| Vulnerability assessments and scans | Which known weaknesses exist in our systems or configurations? |
| Network security assessments | Where could weaknesses in network controls expose critical systems? |
| Gap analyses | Where do existing arrangements fall short of the requirements being assessed? |
| Physical security reviews | Could physical access or environmental conditions compromise ICT infrastructure? |
| Performance testing | Can the service handle expected demand or a backlog after an outage? |
| End to end testing | Does the complete business process produce the expected outcome across applications? |
| Penetration testing | Can a tester exploit weaknesses and bypass security controls? |
| Scenario-based resilience tests | How do people, systems and recovery processes respond to a defined disruption? |
Article 25 also includes methods such as compatibility testing and source code reviews where feasible. Comprehensive testing requires choosing methods that fit the risks being examined.
Start with critical operations and their dependencies
A risk based testing program starts with the service the organization needs to deliver. For a bank, that could be executing payments, providing access to account balances or processing securities transactions.
Identify the critical ICT systems, applications, data and people that support the service. Include cloud services, external payment processors and other ICT service providers. Document the handoffs between them, particularly where responsibility or transaction status can become unclear.
Risk assessments should connect each dependency to a possible business consequence. A provider timeout could leave a payment with an unknown status. A failed recovery job could replay a transaction twice. A queue that restarts successfully could still process events in the wrong order.
These risks can appear before anyone writes a test. A specification may describe how to submit a payment while leaving retry behavior, duplicate detection or changing permissions unclear. TestResults’ article on 10 risks hiding in an online banking specification explores how these gaps affect test priorities.
Those details help teams select operational resilience testing scenarios and define the evidence needed to assess recovery capabilities.
What an end-to-end payment recovery test looks like
Consider an illustrative test in which a payment provider accepts a transaction, but the connection drops before confirmation reaches the bank.
The expected outcome is that the payment reaches a known final state, the ledger reconciles and the customer receives accurate information.
| Stage | What the test should verify |
|---|---|
| Detect | Continuous monitoring identifies the failure and alerts the responsible team. |
| Contain | Retries and manual interventions follow the controls for transactions with an unknown status. |
| Recover | Recovery processes restore processing within the bank’s agreed recovery targets. |
| Reconcile | Provider records, account balances and ledger entries agree. |
| Communicate | Customer and operational interfaces show an accurate transaction status. |
| Validate remediation | The team repeats the scenario after a fix and checks whether the weakness remains. |
Operational resilience depends on how these stages work together. A service can restart while records remain inconsistent. Recovery plans should therefore define how teams verify the business outcome after restoring technical availability.
This also affects how teams approach API testing within complete business workflows. A successful response from one endpoint leaves further questions about whether the transaction reached the right application, triggered the expected action and produced the correct customer outcome.
Teams can extend the scenario to other ICT disruptions, including system failures, unavailable cloud services, delayed messages and interrupted access to a critical application. Controlled exercises can reveal dependencies that ordinary release tests do not exercise.
Where threat led penetration testing fits
Threat led penetration testing (TLPT) is advanced testing that uses threat intelligence to simulate realistic attacks on an identified financial entity. Threat led testing examines how an organization detects and responds to sophisticated cyber threats.
Under Article 26, selected financial entities must conduct TLPT at least every three years, with competent authorities able to adjust the frequency. Selection follows regulatory criteria. The term “significant institutions” alone does not establish the complete scope of this obligation.
TLPT covers several or all critical or important functions and runs on the live production systems supporting them. Its scope includes relevant underlying systems, processes, technologies and outsourced services.
The ECB updated TIBER-EU to align with DORA’s TLPT technical standards. The framework provides guidance for conducting controlled tests. Applicable DORA requirements and regulatory technical standards govern the legal obligation.
Include ICT third party dependencies in testing plans
Outsourcing introduces dependencies that financial institutions need to understand and test. An ICT third party may support a critical service even when customers never interact with it directly.
For TLPT, DORA requires financial entities to identify relevant outsourced dependencies and ensure the participation of ICT third-party service providers included in scope, with appropriate safeguards. The financial entity retains responsibility for compliance.
In broader recovery exercises, useful questions include how the bank detects a provider failure, what information it can obtain during an incident and how it reconciles records after service resumes.
The designation of critical ICT providers for regulatory oversight is distinct from the bank’s assessment of which supplier dependencies matter to its own critical services. A supplier can be essential to your payment process without being designated a critical provider under DORA’s oversight framework.
Prepare test data that exercises recovery
Test data management supports digital resilience testing by giving teams repeatable transaction histories, realistic relationships and defined failure states.
For the payment scenario, prepare transactions that completed before the outage, transactions with an unknown status and transactions that should remain queued. Include duplicate messages and delayed confirmations to check how recovery handles them.
Data masking and synthetic data generation can reduce exposure of personal information in non-production environments. Their effectiveness depends on implementation, access controls and the possibility of reidentification. Sensitive data protection also requires attention during production exercises.
The data strategy needs to fit the exercise. A controlled recovery test in a test environment can use synthetic transactions, while DORA requires TLPT to run on live production systems, with safeguards to manage the risks involved.
Turn findings into evidence of improvement
Testing should produce a record that allows another team to understand what happened and whether the weakness has been addressed.
For each scenario, retain the service and important functions covered, relevant dependencies, methodology, expected outcome, observed behavior, findings, remediation owner and validation result. Linking those records helps demonstrate how testing informs risk management and continuous improvement.
Findings may also inform recovery plans and incident response procedures. DORA’s incident reporting obligations, including those concerning major ICT related incidents, form a separate part of the operational resilience framework.
Regular testing can expose weaknesses before a disruption occurs. Actual improvements in downtime or recovery depend on the corrective actions taken and whether subsequent exercises confirm that those actions work.
Frequently asked questions
How can TestResults support DORA resilience testing?
TestResults can help QA teams repeatedly verify banking workflows across applications, such as checking whether a payment’s status and customer-facing information remain consistent after a change or fix. This supports testing evidence, but does not replace a bank’s full DORA testing programme.
Can test automation check whether a payment recovery fix worked?
Yes. Teams can rerun configured workflow checks after remediation to verify transaction status, ledger-related outcomes and customer interfaces. The exact checks depend on the systems and test environment available.
Can TestResults simulate a payment provider outage?
Test automation can verify workflow behavior under defined conditions, but an outage simulation also requires the bank to control or reproduce the failure condition and involve the relevant systems or provider. The article’s timeout scenario is an example of what the wider exercise should examine.
Does automated end-to-end testing meet DORA requirements on its own?
No. It can contribute repeatable evidence about business workflows, while a DORA testing programme also needs risk-based planning, suitable testing methods, remediation, validation and operational participation.
Can TestResults replace threat-led penetration testing (TLPT)?
No. TLPT is a specialist, threat intelligence-led exercise for selected entities, conducted on live production systems under DORA requirements. Workflow automation can support routine verification, but it is not a substitute for TLPT.
Where test automation supports resilience testing
For QA teams, the relevant use case is repeatable verification of business workflows across applications. In banking, those workflows can span customer interfaces, core banking systems, desktop applications and external services. TestResults’ banking test automation addresses workflows across this mix of technologies.
In a payment recovery exercise, automation can support checks of transaction status, operational interfaces and customer-visible outcomes. Those checks can be repeated after releases, integration changes and remediation, within the capabilities of the configured test environment.
Maintaining those checks matters too. A Swiss bank’s experience after a Pega update broke its test automation illustrates the maintenance challenges that can interrupt repeatable verification.
The broader resilience exercise also needs defined failure conditions, operational participation and evidence from the systems involved. A complete testing programme brings those elements together with security assessments, recovery exercises and specialist testing.
Start with one critical service. Trace it across the applications and providers involved. Then define what evidence would establish that the service recovered correctly, including the transactions already in progress when the failure began.
