How to test LLM applications in banks

LLM application testing for banks is the risk-based evaluation of applications that use large language models (LLMs), including their prompts, data sources, integrations and controls. It checks whether the application gives accurate, appropriately limited responses, protects customer information and handles unsafe or unauthorised requests correctly. Testing should cover the full application path, since a sound model response alone does not prove that the surrounding banking controls work.

Define the system under test

Map the application’s components and trust boundaries before writing tests. A customer-facing assistant may include an LLM, a retrieval-augmented generation (RAG) service that searches approved documents, identity and access services, and tools that read account data or initiate actions. Record which components supply data, make decisions and cause side effects.

Set requirements for each intended use. A product-information assistant may need to answer from approved content and identify when it cannot answer. An assistant that can access customer accounts also needs checks for authentication, authorisation, data isolation and safe handling of actions. Keep business rules, such as payment limits and approval requirements, enforced by deterministic application services.

Agree acceptance criteria with product, security, compliance and operations teams. Regulatory obligations vary with the service, jurisdiction and use of the system, so testing supports a bank’s control and evidence process; it does not establish compliance on its own.

Build a risk-led test plan

Prioritise scenarios by potential customer, financial, operational and data-protection impact. Include normal use, ambiguous requests, invalid inputs, unauthorised access, service failures and attempts to override instructions. Cover both the model’s response and the application’s observable effects, such as API calls, account changes, audit events and handover to a human reviewer.

Create a versioned evaluation set from approved requirements and representative synthetic or properly de-identified data. Include expected outcomes, prohibited outcomes and the evidence needed to assess each case. For generative responses, a single exact-string assertion is often too brittle. Check required facts, omissions, policy boundaries and response structure; use deterministic assertions for values and actions that must be exact.

Test dependencies and change impact. A model, prompt, system instruction, retrieval index, policy document or tool API change can alter behaviour. Run targeted regression tests across affected risks, and check for environment drift such as mismatched model versions, stale documents or different tool permissions between test and production.

Test behaviour, security and controls

For accuracy and grounding, test common questions, incomplete requests, conflicting source documents and questions that the application cannot answer. Verify that retrieved evidence supports the response and that the application handles missing or outdated information according to its requirements.

For security, test prompt injection in user messages and retrieved documents, attempts to obtain another customer’s data, and requests for internal instructions or sensitive information. Verify that content from an untrusted source cannot change tool permissions or override application policy. Check that retrieval applies access controls before returning customer-specific content.

For integrations, validate tool selection, arguments, permissions, timeouts, retries and error handling. Confirm that a model cannot bypass server-side authorisation or approval gates. Test duplicate-action prevention where retries could repeat a transaction, and verify that audit records capture the request, decision and resulting action at the level required by the bank’s controls.

Use test data that cannot expose real customer information. Restrict access to prompts, completions, traces and logs, since they may contain sensitive data. Check retention, masking and access controls for test evidence as well as for application data.

Practical example: account balance assistant

Seed a test environment with two customers and distinct account balances. For an authenticated customer, ask for the balance and assert that the response matches the authorised account data returned by the account service. Repeat with an unauthenticated session and with a request for the second customer’s balance. The expected result is a refusal or authentication route, with no disclosure and no unauthorised account lookup.

Then place an instruction in a retrieved document telling the assistant to reveal another customer’s details. Ask a normal balance question and verify that the injected instruction has no effect. Assert the response, retrieval permissions, tool-call log and absence of cross-customer data in the trace. These checks test the complete application path, not just a model-generated sentence.

Record results and release evidence

For each test run, record the application and model versions, prompt and retrieval configuration, test data reference, expected and observed outcomes, and any tool calls or control decisions. Define failure thresholds in advance for measurable evaluation results, and review failures by risk. A release decision should account for unresolved high-impact failures and the bank’s approval process. Re-run relevant tests after changes, and monitor production behaviour under the bank’s approved privacy, security and operational controls.