How to test non-deterministic behaviour in banking AI agents

Banking AI agent non-deterministic behaviour testing checks whether an agent stays within defined safety, policy and workflow limits when the same request can produce different reasoning, responses or tool calls. It evaluates acceptable behaviour across repeated runs and changing conditions, rather than requiring identical wording or execution traces. The aim is to detect unsafe variation, missed controls and inconsistent outcomes before release.

Identify what can vary

An agent’s behaviour can change because of model sampling, model or prompt versions, retrieval results, tool responses, session memory, concurrent updates or timing. Environment drift can add further variation, such as changed account state, permissions, reference data or downstream service behaviour.

Start by mapping the agent’s inputs, tools, permissions and consequential actions. Record which components are fixed for a test and which are deliberately varied. Pin model, prompt, retrieval index and tool versions where possible. Use controlled test accounts and a non-production environment for actions such as payments or beneficiary updates.

Define expected behaviour as a contract. Specify permitted tools, required confirmation or approval steps, transaction limits, conditions for refusal or escalation, and acceptable final states. Separate these requirements from response wording, which may vary while still being correct.

Design repeated-run tests

Build a test set around customer intents, boundary conditions and known failure risks. Include ambiguous instructions, missing information, conflicting retrieved content, tool timeouts, stale account data and attempts to exceed the agent’s authority. Keep test cases independent where session memory could carry information between runs.

Run the same case repeatedly under controlled conditions to observe variation. Then vary one factor at a time, such as retrieval results, tool latency or initial account state. Add combined-condition tests after individual dependencies are understood. Repetition can reveal intermittent failures, while targeted adversarial cases help exercise specific safety boundaries.

Avoid using exact response matching as the main assertion. Instead, check hard invariants and use a documented rubric for qualities such as factual accuracy, relevance and appropriate escalation. Set acceptance criteria in advance. A high average quality score must not conceal a single prohibited transaction or permission breach.

Check actions and final state

For every run, validate the agent’s tool selection and arguments, the authorisation context, any approval gate, and the resulting state in the relevant system. Where an action can be retried, check idempotency and confirm that retries do not create duplicate effects. Check transaction records and audit events as well as the agent’s summary.

A response that says a payment was made is not evidence of settlement. Assert against the payment service or ledger state that the test is designed to verify. For a failed or interrupted workflow, check that the agent reports the actual status and follows the specified recovery or escalation path.

Practical example: a payment instruction

Suppose a customer asks an agent to pay a new beneficiary. Run the same request with varied tool latency and beneficiary data states. For each run, verify that the agent uses the approved beneficiary and payment tools, validates the amount and destination, and requests the required customer confirmation. If policy requires a second approver, assert that the agent waits for that approval before submission.

After submission, check the payment record and account state. Repeat or interrupt the tool response to test retry handling. The case passes only when every run respects the approval rules and produces a permitted final state. Wording may differ; an unauthorised submission or duplicate payment is a failure.

Capture evidence and interpret results

Keep a trace for each run, including the case and test data identifiers, model and prompt versions, relevant configuration, retrieval references, tool calls and responses, timestamps, final system state and assertion results. Apply the bank’s data handling controls to logs, since prompts and traces may contain sensitive information.

A reproducible seed can help investigate a failure where the model or test harness supports it. It does not establish that behaviour is safe across other sampling settings, inputs or dependency changes. Track failures by category and severity, and rerun them with the original trace before deciding whether a change has fixed the cause. Treat unexplained variation in a safety-critical action as a release risk that requires investigation.