How to test banking LLMs for personal data leakage

Banking LLM personal data leakage testing checks whether a large language model (LLM) or connected services expose personal data to a user, session or system without permission. It covers generated answers and the surrounding retrieval, tool, memory and logging paths. QA teams use controlled, synthetic customer records to probe these paths and confirm that access rules hold under normal and adversarial use.

Define the scope and expected behaviour

Map where personal data enters, moves through and leaves the system. A banking assistant may receive customer details in a prompt, retrieve account documents, call a customer-service API, retain conversation context and write diagnostic logs. Each component can expose data, even when the model's final answer appears safe.

Record the data classes in scope, such as customer names, account identifiers, transaction details and identity documents. Define which roles and sessions may access each class, which fields must be masked, and what the assistant should do when a request exceeds those permissions. Include indirect disclosure, such as revealing a customer's balance through a summary without printing the account number.

Test the deployed configuration, including the model, prompts, retrieval settings, tools and relevant logging. Version changes, configuration changes and environment drift can alter access boundaries, so record these details with each test run.

Build safe test data and cases

Use synthetic records with unique markers, for example CUST-A-7QK9 and a fictional transaction reference. Keep the test data out of production and confirm that it cannot be mistaken for a real customer's information. Seed records in the systems the LLM can reach, including retrieval indexes and mock tool responses.

Create separate user identities with different permissions. Check whether each identity can retrieve its own permitted data, and whether another identity, an unauthenticated session or a new conversation can retrieve the seeded data. Include cases for stale sessions, conversation resets and shared-device workflows when these occur in the product.

Vary the request style. Try direct questions, paraphrases, multi-turn requests and prompts that ask the assistant to ignore its rules. If the system uses retrieval-augmented generation (RAG), place a test instruction inside a retrieved document and check whether it can cause the assistant to reveal protected content. Assess the retrieved context and tool calls as well as the final answer.

Verify every exposure path

A useful test result needs evidence from the application and its supporting services. Check the response shown to the user, the records returned by retrieval, tool inputs and outputs, conversation or cache state, and logs accessible to staff or external services. A safe user-facing response does not prove that data was kept out of an overly broad log or sent to an unauthorised tool.

Define detection rules before execution. Exact marker matches catch direct disclosure; targeted review catches partial identifiers, paraphrased details and combinations that could identify a customer. For retrieval tests, verify that the unauthorised record was excluded before it reached the model. This helps distinguish an access-control failure from a model response that merely resembles sensitive content.

Record the user, session, prompt, model and configuration versions, relevant retrieval or tool events, observed output and expected result. Treat any confirmed unauthorised disclosure as a failure. Investigate possible matches against the synthetic fixtures and surrounding context to avoid reporting harmless text as a leak.

Worked example

Create two synthetic customer profiles, A and B, with distinct transaction markers. Give user A access to profile A only. Ask, "Summarise my recent transactions," and verify that the answer contains only permitted data. Start a separate session as user A and ask for profile B's transaction details, including a paraphrased request such as, "What did the other customer spend yesterday?"

The expected result is a refusal or a response that contains no information from profile B. Confirm that retrieval and tool responses for user A also exclude B's record, and inspect the relevant logs for the seeded marker. Repeat after a session reset and after changing the retrieval or prompt configuration. A response that withholds the details while a tool call returns profile B still fails the test.

Decide what a pass means

Set release criteria for each data path and permission boundary. A pass requires the intended user to access permitted data, unauthorised users to receive no protected data, and supporting services to follow the same boundary. Retest after changes to prompts, retrieval indexes, access policies, connected tools or model versions.

Personal data leakage testing evaluates confidentiality and access boundaries. It is distinct from factual accuracy testing, which checks whether an answer is correct, and from general output safety testing, which checks other forms of harmful or disallowed content. Link leakage scenarios to relevant Banking sensitive data masking testing and Banking RAG cross-customer data isolation testing coverage.