How to test account information accuracy in banking chatbots

Banking chatbot account information accuracy testing checks that a chatbot returns account facts that match authorised, current banking records. It covers the values retrieved from banking services and how the chatbot presents them, including balances, transactions, account status and dates. Tests compare responses with an independent source of truth while checking permissions, currency, timing and wording.

Map each answer to its source

Start by listing the account information the chatbot can provide and the system that owns each field. A balance may come from a core banking service, while a card status or transaction description may come from a separate service. Record the authoritative field, its expected format, the customer permissions required and any freshness rule.

Prepare controlled test accounts with known data. Include ordinary cases and records that exercise relevant rules, such as pending transactions, holds, joint-account access, dormant status and multiple currencies. Use synthetic or appropriately protected test data. Keep the chatbot, application programming interface (API), test account and source system in the same controlled environment or document any differences that could affect results.

For each test, capture the question, authenticated user, account identifier, chatbot response and source-system value. Compare structured fields exactly where possible. Check amounts as decimal values with an explicit currency, rather than comparing loosely formatted strings. For natural-language answers, assert the required facts and meaning, allowing harmless wording variation.

Test the full retrieval and response path

Verify the account selection and authorisation before checking the answer. A user with two accounts should receive information for the account they selected or clearly identified. A user without access to another customer's account must receive no details from it, even if the conversation includes that account number. Include expired sessions and revoked permissions where the workflow supports them.

Then test the path from the chatbot through its retrieval or tool call to the banking service and back. Check that the request uses the right customer and account identifiers, that errors are handled safely, and that the response does not substitute a cached or previous customer's data. If the chatbot uses a large language model (LLM), test that it grounds account facts in returned service data. It should not invent a value when a field is missing or a service call fails.

Check time-sensitive information against the source record at the same point in the test. Where updates propagate asynchronously, define the accepted freshness condition and test both within and beyond it. A response may be factually correct for an earlier snapshot yet misleading if it omits when the balance was retrieved. Include timestamps or clear status wording when the product requires them.

Worked example: checking a balance

Set up an account with a ledger balance of £500.00 and a pending card payment of £40.00. The expected available balance is £460.00, according to the account's configured balance rules. Sign in as the account holder, ask, "What is my available balance?", and compare the answer with the account service response.

The test should confirm the amount, pound sterling currency, account selected and any required as-of time. It should also confirm that the chatbot does not label the ledger balance as available balance. Repeat with the pending payment removed, then with a service timeout. On timeout, the chatbot should report that it cannot retrieve the current figure or follow the approved fallback; it should not reuse £460.00 as though it were current.

Check failures and record evidence

Include cases for missing fields, delayed updates, malformed service responses, rounding, negative balances, multiple accounts and ambiguous questions. A failure can arise from the banking service, integration mapping, cache, dialogue state or generated wording, so retain enough evidence to identify the layer. Compare the chatbot's answer with the source record and service response, and record the request identifiers, timestamps, test data reference and relevant logs with sensitive values masked.

Run these checks after changes to prompts, retrieval logic, APIs, account rules or test environments. Environment drift can change fixtures or service behaviour and make comparisons unreliable, so verify test data and configuration before execution. Treat a mismatch in amount, currency, account identity or customer access as a defect requiring investigation, even when the surrounding response reads naturally.