How to test AI payslip extraction for mortgage applications

AI payslip extraction testing for mortgage applications checks whether a system identifies and returns the correct information from payslips, then handles uncertainty and errors safely in the mortgage workflow. It covers document recognition, field-level accuracy, data validation and routing to manual review. The aim is to establish that extracted data is reliable enough for its intended use, with clear controls when it is not.

Define the expected output

Start with the fields the application needs and define how each should be represented. A typical schema might include applicant name, employer, pay date, pay period, gross pay, net pay, deductions, currency and payment frequency. The expected values and rules should come from approved requirements, including how the system handles missing or ambiguous information.

Keep extraction separate from downstream affordability calculations and lending decisions. For example, a payslip may show monthly gross pay, while an affordability service applies its own rules to income and outgoings. Test each component against its own requirements, then test the data passed between them through mortgage application workflow testing.

Build a labelled test set

Use payslips with verified expected values, sometimes called ground truth. Include samples from the document types the application accepts, such as digital PDFs, scanned pages and photographs. Cover different layouts, fonts, table structures and page quality, as well as multi-page documents where supported.

Add cases that expose likely errors: a cropped pay figure, a blurred date, a payslip with several deduction lines, a missing field, or two values that could be mistaken for one another. Include valid variations such as different date formats and pay frequencies. Use synthetic or properly anonymised documents where possible, and control access to any real applicant data used for testing.

Keep the labels independent of the extraction result. Record the source location for each expected value, such as the page and field area, so reviewers can check whether the system read the right part of the document. Version the test set and preserve it for regression runs.

Test extraction and validation

Compare each returned field with its expected value. Use exact matching for identifiers and dates after agreed normalisation. For monetary values, define how currency symbols, separators, decimal places and rounding should be handled. A test should distinguish a formatting difference from a materially incorrect amount.

Measure results at field level as well as per document. Field-level precision shows how often returned values are correct; recall shows how often expected values were found. Review error types alongside these measures. A system can return many fields while consistently confusing gross and net pay, which an overall document pass rate could conceal.

Check validation rules separately from recognition. If the extraction output contains a pay date outside the accepted range, a currency the workflow does not support, or a value in the wrong format, verify that the application rejects or flags it as specified. Do not silently replace missing values with assumptions.

Verify confidence and failure handling

Test the configured confidence behaviour using clear, ambiguous and unreadable documents. Confirm that low-confidence fields follow the required route, such as requesting a clearer upload or sending the case to a review queue. Test whether a reviewer can correct a value and whether the corrected value, source document and review action remain traceable.

Also test processing failures: corrupt files, unsupported formats, empty uploads, oversized documents and service timeouts. Confirm that the customer receives an appropriate status, the application does not use partial or stale extraction results, and retries do not create duplicate documents or review cases. Confidence thresholds are system-specific, so verify them against the approved workflow rather than assuming a universal cutoff. See also AI document confidence threshold testing.

Run an end-to-end example

Suppose a labelled payslip shows gross pay of £3,250.00, net pay of £2,480.00 and a pay date of 30 June. Submit the document through the mortgage application, then verify that the extraction response contains those values in the correct fields and currency. Check that the application stores the result against the correct applicant and displays it for review.

Repeat with a version where the gross-pay figure is blurred. The expected result should follow the configured uncertainty rule, such as a review task, rather than presenting an unverified amount as confirmed. Finally, check that downstream income calculations use only accepted data and that the audit record links the result to the uploaded document. This tests extraction, validation and workflow integration without treating a successful field read as a mortgage approval.