How to test calculation tools in banking LLMs

Banking LLM calculation tool testing checks whether a large language model (LLM) selects an approved calculation tool, sends it valid financial inputs and communicates its result accurately. It covers the tool call and its controls as well as the arithmetic: a correct calculation can still produce a failed outcome if the model uses the wrong rate, changes the returned amount or presents an unauthorised action as complete.

A test approach for banking LLM calculation tool testing

Treat the LLM, tool interface and calculation service as separate components with observable boundaries. A useful test harness records the user request, selected tool, arguments, tool response and final answer. This lets testers locate failures in tool selection, argument construction, calculation or response handling.

Start with the business rule and define its assumptions: eligible balance, rate, currency, date range, day-count convention, rounding point and output format. Build expected results with an independent reference implementation or approved calculation examples. Avoid using the same calculation code as both the system under test and the test oracle, since a shared defect could make incorrect results appear correct.

Test the tool contract as well as the LLM. Check required fields, types, permitted ranges, currency codes and behaviour for missing, malformed or conflicting values. Verify that the tool rejects invalid requests safely. Keep calculation tools separate from tools that post transactions, change account details or make other state changes. If a workflow can trigger a financial action, test its authorisation and approval controls separately.

Test calculations and tool use

Create cases for ordinary inputs, boundary values and invalid conditions. Include zero and negative amounts where relevant, rate limits, date boundaries, leap years, currency precision and values that fall on rounding boundaries. Check the bank's specified rounding rule at the specified stage of the calculation; rounding intermediate daily amounts can produce a different result from rounding only the final amount.

Exercise tool selection with requests that need a calculation and requests that do not. For calculation requests, check that the chosen tool and arguments match the user’s intent and any trusted account or product data supplied to the system. Test ambiguous requests, such as a question that omits the currency or calculation period, against the expected clarification or safe failure behaviour.

Assert the final response against the tool result. The model should preserve the returned amount, currency and relevant assumptions. It should not claim that a calculation succeeded when the tool failed, or infer an unsupported amount from incomplete data. Where the response is intended for a customer, check that its wording distinguishes an estimate from a confirmed account figure.

Worked example: simple daily interest

Suppose an approved tool calculates simple interest using an actual/365 convention. The test fixture supplies a principal of GBP 10,000, an annual rate of 3.65% and a 30-day period. The expected result is GBP 30.00:

10,000 × 0.0365 × 30 ÷ 365 = 30.00

Assert that the LLM calls the interest tool with those values and the specified convention, that the tool returns GBP 30.00, and that the final answer reports the same amount. Add a paired case with a different day-count convention and verify that the model passes the requested convention instead of silently reusing the first case’s arguments. The test should also confirm that a calculation request does not create an account posting.

Control dependencies and diagnose failures

Results depend on the deployed tool schema, calculation rules, reference data and environment configuration. Record relevant versions in test evidence, including the prompt or model configuration, tool definition, rate table and rounding settings. Environment drift, such as a stale rate table or changed default convention, can cause failures even when the prompt is unchanged.

Classify failures by stage: incorrect tool choice, invalid or incomplete arguments, wrong tool output, inaccurate final response, or control failure. Re-run deterministic tool tests separately from LLM interaction tests. For LLM tests, use assertions on observable behaviour and retain the input, tool trace and output so that a changed response can be investigated. Include these cases in regression testing when a model, prompt, tool schema or calculation rule changes.

This work sits within [banking LLM application testing] and should connect to relevant [banking rounding and precision testing] for the underlying financial rules.