How to test complaint routing in banking chatbots

Banking chatbot complaint routing testing checks whether a chatbot recognises a customer complaint, applies the bank’s routing rules and transfers the case to the correct team with the required context. It also verifies that the handover succeeds, sensitive information is handled safely and the customer receives an accurate confirmation. The expected route must come from documented bank policy, since complaint definitions and handling procedures vary.

Define the expected routing behaviour

Start by mapping complaint categories to their destinations and required actions. A complaint about a disputed card payment might go to a disputes queue, while a complaint about service from a branch might go to a customer relations team. Include any priority rules, exclusions and escalation conditions in the test requirements.

Separate the customer’s issue from the chatbot’s route decision. A customer may describe a payment problem without making a complaint; another may express dissatisfaction without using the word “complaint”. Test both cases against the bank’s definition and routing policy. Also include messages that mention several issues, such as a disputed payment and an inaccessible account.

Record expected outcomes for each scenario: whether the chatbot should classify the conversation as a complaint, which queue it should select, what information it must collect, and what confirmation it should give. Link these outcomes to the relevant acceptance criteria or business rules.

Build representative test conversations

Use realistic, approved test data and vary the wording, context and channel conditions. Include direct statements, indirect expressions of dissatisfaction, misspellings, short replies, multiple complaints and conversations where the customer changes their explanation. For a chatbot using a large language model (LLM), repeat representative cases and check that acceptable wording variations still produce the required route.

Test multi-turn context. For example, a customer might first report an incorrect charge and later say they want to make a formal complaint. Verify that the bot retains relevant details and updates the case classification without losing the original issue. Check that an ambiguous message triggers an appropriate clarification or safe escalation under the bank’s rules.

Include negative and boundary cases: a general product question, a complaint about a third party, a message with no usable details, and a request that falls outside the chatbot’s supported services. Test urgent reports, such as suspected fraud, against the bank’s separate escalation rules. Do not assume every urgent issue follows the standard complaints route.

Verify the decision and the handover

Assert the routing result at each relevant layer. Check the intent or category, any extracted entities, the selected destination, priority, and case status. Where the design uses confidence thresholds, verify the behaviour on either side of the configured threshold and confirm that low-confidence cases follow the documented fallback.

Then test the integration with the case-management or contact-centre system. Confirm that the destination accepts the case, the case identifier is returned where required, and the transcript and permitted metadata arrive intact. Check behaviour when the service is unavailable, rejects a request or responds slowly. The chatbot should follow its specified retry or fallback behaviour, avoid creating duplicate cases, and tell the customer what will happen next.

Review the handover for data minimisation. Confirm that fields sent to the receiving team are necessary for the complaint, access-controlled as intended and free of unrelated conversation content. Verify that authentication state and customer identity are handled according to the workflow. A complaint should not expose another customer’s data or rely on an unverified identity to disclose account information.

Practical test example

Suppose policy routes disputed card transactions to the card disputes queue and requires a human handover when the customer asks to complain. Use a controlled test account and send: “I don’t recognise this card payment, and I want to complain.” Check that the bot identifies both the disputed transaction and complaint request, selects the configured disputes route, creates one case, and passes the relevant transaction reference and conversation context. Verify the customer receives the expected confirmation.

Repeat with an unavailable case-management service. Check the configured fallback, such as a clear message and an alternative contact route, and confirm that the bot does not claim a case was created. The expected result should specify observable outcomes, rather than relying on a reviewer’s impression that the conversation sounded helpful.

Evaluate coverage and maintain evidence

Use a decision table or scenario matrix to connect complaint types, urgency, customer intent and integration outcomes to expected routes. Track misroutes, missed complaints, unnecessary escalations and failed handovers separately; each points to a different control or component. Review examples where the model’s language differs while the route stays correct, and examples where fluent wording hides an incorrect destination.

Retain the test conversation, expected route, actual route, case-system response and relevant configuration or model version. Re-run the cases after changes to prompts, routing rules, queue mappings, integrations or the chatbot model. This supports regression testing and helps distinguish a classification change from an integration failure. For broader context, see [Banking chatbot testing] and [Banking chatbot human handover testing].