Reading an invoice is the easy part of accounts payable automation
For Finance controllers, CFOs and accounts teams · Compare approaches
TL;DR — what you’ll learn
- Evaluate critical fields on representative invoices that were not used to configure the system.
- Measure field errors and review effort instead of relying on a single confidence score.
- Validate supplier, item, unit and document identity before creating an ERP draft.
- Retain the original document and correction history so a later dispute can be reconstructed.
An invoice tool reads every digit in the grand total correctly. It also turns a supplier's 25-kilogram sack into 25 individual units. The demonstration looks impressive until someone tries to match the receipt. Extraction quality depends on which errors the business can afford to miss.
How do you evaluate AI invoice extraction before connecting it to the ERP?
Test the fields that drive accounting and operational decisions on a held-out set of representative invoices. Measure corrections by field and document type. A readable total is not enough when the item, unit or invoice identity is wrong.
- 01Receive the documentSource identity and duplicate check
- 02Extract proposed fieldsKeep page evidence and uncertainty
- 03Validate the draftSupplier, units, totals and referencesAre critical fields verified?YesAccept for processingNoAsk a reviewer to correct or resolve the uncertain values.
- 04Accept for processingUse the authorised accounts workflow
Illustrative information flow. The article explains the evidence, exceptions and responsible decisions. Diagram: NextEdge Labs.
Build a test set that resembles the inbox
Include clean PDFs, scans, multi-page invoices, unfamiliar layouts, credit notes and documents with handwritten annotations if those occur in practice. Keep some documents out of configuration work so the evaluation includes genuinely unseen examples. Remove unnecessary personal or confidential data before sharing material with a service provider.
For each document, establish an independently checked answer for supplier, invoice reference, date, currency, line items, units, quantities and amounts. Do not use the model's first extraction as the answer key. Record missing fields as missing rather than rewarding a plausible guess.
Confidence is a review signal
Microsoft's Document Intelligence documentation distinguishes confidence at different extraction levels. A high score should not substitute for checking critical fields on your own documents. Decide review rules by error consequence and observed performance, not by choosing an attractive percentage in a demo.
| Possible error | Useful check |
|---|---|
| Supplier alias mapped incorrectly | Match an approved supplier record |
| Invoice uploaded twice | Check supplier, reference and supporting document identity |
| Pack unit misunderstood | Compare approved item and unit mappings |
| Line amount disagrees | Recalculate and route the difference |
Source: Microsoft Learn · Field confidence and evaluation. Reference details
A high document score can hide the wrong field
Imagine an illustrative test of 100 invoices. Ninety-five totals are correct, but twelve invoices contain at least one wrong unit or item mapping. Reporting only 95% total accuracy would conceal the work needed to match receipts. Record accuracy by field and the proportion of documents needing any correction. Also identify errors that a validation rule caught before a reviewer saw them.
Separate extraction from master-data matching. The model may read the supplier's wording correctly while the system maps that wording to the wrong internal ingredient. Conversely, an imperfect text extraction may still be resolved safely by a reviewer using the source image. Combining those stages into one score makes it difficult to know whether to improve the model, the mapping table or the review screen.
Include the documents that people avoid in a demo
Use a credit note with a similar layout to an invoice, a multi-page invoice with a carried-forward subtotal, a scan with an unreadable line and a familiar supplier using a new format. Check that the system marks missing information as unknown. A plausible completion can be more dangerous than a visibly empty field because the reviewer may not realise evidence is absent.
Keep each evaluated document's expected answer and observed errors. When the configuration changes, rerun the same sample and a fresh sample. This helps detect an improvement on one supplier that creates regressions elsewhere. Avoid sending unnecessary employee or bank details to an external service; agree document handling, access and retention as part of the supplier assessment.
Count the full cost of the review
Consider a hypothetical manual process taking six minutes per invoice. An extraction tool takes seconds, but a reviewer still spends four minutes finding corrections and entering missing information. The useful comparison is the complete elapsed work per invoice, including exception handling and ERP preparation. It is not six minutes versus the model's response time.
Watch a finance user review the output. Can they jump directly to the source line, correct the mapped unit and see the affected totals recalculate? Can they reject a duplicate without creating another task? Those interactions determine whether the tool saves time in practice. Use observed timings from the pilot for the business case and keep assumed future savings separate.
Measure the work left for the reviewer
Track how many documents need a correction, how long review takes and which suppliers create repeated exceptions. A system that extracts more fields but makes errors difficult to locate may increase the total workload. Show the source page beside the proposed value and keep corrections attributable.
Only after this stage should the pilot draft an ERP transaction. Three-way matching, approval and payment are separate decisions. Preserve the original document and the extraction version so a later dispute can be reconstructed without relying on the current model output.
Summary
Invoice OCR succeeds when it reduces the work required to produce an accurate, reviewable accounting record. A strong evaluation includes difficult documents, independently checked answers and the cost of corrections. Extraction can prepare the evidence; it does not establish that the supplier's claim is commercially correct.
What to do next
- Build a representative redacted sample and reserve some documents for independent evaluation.
- Define critical fields and the consequence of each error with accounts payable.
- Measure reviewer time and corrections before connecting extraction to transaction posting.
Compare this workflow with your current setup
Share the step that takes the most manual work, the systems involved and the exceptions your team handles. That gives us a concrete starting point for comparing approaches.
Sources and further reading
Sources checked 15 October 2026. The worked examples and software evaluation suggestions are NextEdge Labs’ editorial analysis.
- Accuracy and confidence scoresMicrosoft Learn · Azure AI Document Intelligence documentation, accessed October 2026
Background on document extraction confidence. Field accuracy and review thresholds must be evaluated on the intended documents.
Visual credits
AI-generated illustration created for NextEdge Labs; it does not depict a client facility. Workflow diagram by NextEdge Labs. Icons from Lucide (ISC licence).
