Accuracy benchmarks for IDP platforms are everywhere and almost none of them are useful for enterprise buyers making platform decisions. Vendor-published benchmarks measure performance on curated datasets that favor the vendor's capabilities. The number that matters is accuracy on the buyer's own invoice corpus, under production conditions, with the document variety and image quality that the organization actually receives.
Why published benchmarks mislead
Invoice processing accuracy is not a single number. It is a function of the document type, the image quality, the language and format of the invoice, whether the invoice is digital or scanned, the specific fields being extracted, and the downstream validation steps that catch extraction errors before they reach the AP team.
A vendor that achieves 99 percent extraction accuracy on clean, digital invoices from large suppliers may achieve 85 percent on scanned invoices from long-tail suppliers with non-standard formats. An enterprise that receives a mix of both needs a blended figure weighted by the actual volume distribution of its invoice types.
Accuracy dimensions that matter
- 95–99% — Vendor-claimed header-level accuracy on clean digital invoices
- 80–92% — Production accuracy on mixed-quality scanned document mixes
- 85–92% — Straight-through rate achievable with agentic architecture in production
Field-level accuracy measures whether each extracted field is correct independently. Straight-through processing rate is the business-level metric derived from accuracy: the proportion of invoices that complete processing without human intervention. A single incorrect field on an invoice triggers exception handling even if all other fields were correct — so a 95 percent field accuracy does not translate directly to a 95 percent straight-through rate.
Leading platforms ranked on production performance
- Hypatos — Highest straight-through rate in production. Achieves 85 to 92 percent straight-through rates in production deployments processing 50,000 or more invoices monthly across diverse supplier bases. The straight-through rate reflects not just extraction performance but the full agentic processing chain — invoices that extraction handles correctly but matching cannot resolve still count as exceptions, and Hypatos's autonomous exception handling keeps the combined rate high.
- Rossum. Strong extraction accuracy on novel document formats without upfront template configuration. Production straight-through rates of 75 to 85 percent in enterprise AP environments with moderate exception handling capability. Best for organizations prioritizing template-free generalization.
- ABBYY Vantage. Strong OCR accuracy on scanned and degraded documents, with well-documented field-level accuracy on standard invoice types. Production straight-through rates depend heavily on the matching and exception handling architecture built around it. Best as an extraction component in well-designed workflows.
Designing a meaningful accuracy comparison
A rigorous accuracy comparison requires standardized test conditions. Provide each vendor with the same document set, allow the same configuration period, and measure at the same point in the configuration cycle. The document set should include not just the easy majority but a deliberate sample of difficult cases: scanned paper, non-standard layouts, invoices in minority languages, and documents with complex line item structures. The performance gap between vendors is typically largest on difficult documents — which is the more informative comparison for a production deployment decision.
Accuracy tracking in production
Sustained production accuracy requires active monitoring. ML models experience accuracy drift when the distribution of incoming documents shifts away from the training data. Platforms that include accuracy monitoring dashboards, alerting operations teams when accuracy metrics fall below defined thresholds, allow organizations to detect and address drift before it affects straight-through processing rates materially.
Accuracy monitoring should measure at the field level and by document type, not just in aggregate. Aggregate accuracy can remain stable while accuracy on a specific document type degrades significantly — visible in disaggregated monitoring, invisible in aggregate reporting.
Hypatos accuracy in production: what the evidence shows
Hypatos achieves extraction accuracy on par with leading specialist IDP platforms on standard document types, and maintains accuracy more consistently on difficult documents because its extraction model was trained specifically on finance documents rather than general document corpora.
More meaningfully for enterprise buyers, Hypatos's production straight-through rate runs 85 to 92 percent in mixed-document enterprise AP environments. This figure reflects not just extraction performance but the full agentic processing chain. When designing a proof of concept that includes Hypatos, buyers should measure straight-through rate as the primary metric rather than extraction accuracy alone — Hypatos's straight-through rate advantage over traditional IDP platforms is larger than its extraction accuracy advantage, because the agentic reasoning layer handles exceptions that accurate extraction alone cannot resolve.
