How to test access permissions in banking RAG systems

Banking RAG access permission testing checks that a retrieval-augmented generation system retrieves and uses only information the current user is authorised to access. It verifies permissions across document search, generated answers and supporting citations, including after roles or access rights change. The aim is to prevent one customer, employee or business unit from learning another party's confidential information through the system.

Map the permission path

A retrieval-augmented generation (RAG) system finds relevant content and supplies it to a language model to answer a query. Access control can fail at several points: identity may be mapped to the wrong user, retrieval may ignore document permissions, cached results may cross user boundaries, or an answer may reveal information that was present in unauthorised context.

Document permissions should come from an authoritative source, such as the bank's identity and access management service or document repository. A document's access metadata must remain associated with its chunks during ingestion and indexing. If a document inherits permissions from a folder or customer record, test that inheritance as well as direct grants.

Check that the system resolves the authenticated user's identity, roles, groups, customer or tenant scope, and any applicable entitlements before retrieval. Filtering should apply to the content supplied to the model, not only to what the interface displays. Test citations and snippets too: hiding an answer while exposing a restricted document title or excerpt still discloses information.

Build a permission test matrix

Use synthetic or appropriately protected test data with clear ownership and access rules. Include users with different scopes, such as a retail customer, a relationship manager assigned to one customer, and a colleague without that assignment. Set up documents containing distinctive markers so a leak can be detected in retrieved context, answers, citations, logs and cached responses.

For each role and document class, cover these cases:

• An authorised user asks a question requiring an accessible document. Confirm that retrieval includes the permitted content and the answer uses it accurately.
• An unauthorised user asks the same question. Confirm that the restricted content is absent from retrieved context, the answer does not disclose it, and no citation or snippet identifies it.
• A user has access to one customer record but not another. Ask questions that name the other customer and questions that try to infer their details indirectly.
• A user's access is removed or their group membership changes. Repeat the request and check that old permissions do not persist through an index, session or cache.
• Documents contain instructions aimed at overriding system controls. Confirm that document text cannot grant access or change the user's entitlements.

A refusal or a general answer can be acceptable for an unauthorised query. The important assertion is that restricted information never enters the response path or leaks through related outputs. Avoid relying on wording checks alone; inspect retrieval results and, where the test environment permits, the context passed to the model.

Verify the end-to-end behaviour

Test the full path from sign-in through document ingestion, indexing, retrieval, generation and response delivery. Include API requests as well as the user interface where both are used. Recheck access after re-indexing, permission synchronisation, model or prompt changes, and deployment to environments with different identity mappings. These checks help expose environment drift and stale metadata.

Also test cache boundaries. Run the same query first as an authorised user and then as an unauthorised user, and reverse the order. Confirm that cached answers, retrieved passages and citations are either permission-scoped or re-authorised before reuse. Include concurrent requests if the application can reuse shared session or cache state.

Record the test identity, entitlement state, document identifiers, expected access decision, actual retrieval outcome and response evidence. Keep evidence free of real customer data. A passing answer assertion alone is insufficient if the system logged restricted content or sent it to an external model service contrary to the test boundary.

Worked example

Create a synthetic statement for customer C-104 containing the marker TEST-ALPHA-731. Grant access to relationship manager RM-17 and deny it to RM-22. Ask both users, "What is the statement marker for C-104?" For RM-17, assert that the permitted document is retrieved and the answer matches its marker. For RM-22, assert that the marker is absent from retrieved context, answer text, citations and any user-visible error details. Then revoke RM-17's access, repeat the query, and verify the denial takes effect without waiting for a stale cache to expire.

Treat a failure at any stage as an authorization defect, even when the generated answer appears harmless. Prioritise tests around customer separation, employee entitlements and permission changes, then extend coverage to the bank's document types and workflows.