SourceVane · Practical AI
Measure AI document-processing cost per accepted document
Budget document extraction using required-field checks, billed retries and accepted original documents. Keep OCR and review costs separate.
Direct answer
The decision in brief
Use one original document as the task. Count every billed extraction attempt, then divide token spend by documents that pass a written field-level acceptance rule. A parseable JSON response alone is not a successful extraction.
How should you measure document extraction?
- Build a representative document set with known answers. Include layout changes, missing fields, long documents and poor scans; keep those cases labelled so an easy subset cannot conceal failures.
- Define required fields, permitted normalizations and numeric tolerances before the run. Verify extracted amounts, dates and identifiers against the document; record omitted fields separately from incorrect values.
- Log input and output tokens for every billed call, including rejected JSON, corrections and timeouts that were charged. Group calls by original document so one successful retry does not create a second completed task.
- Report accepted documents, rejected documents and documents sent to human review. Re-run the same set after changing a prompt or model and preserve the acceptance rule.
Write the acceptance rule first
For example, a document can require all invoice identifiers and currency amounts to match a reference record, with no invented fields. This is an acceptance-rule example, not a measured accuracy claim.
Worked illustrative example
These numbers are assumptions chosen to demonstrate the calculation. They are not provider quotes, measured success rates or model benchmarks. Both options use $1 per million input tokens, $4 per million output tokens and an assumed 90% final success rate.
The example has 5000 original tasks, 6000 input tokens and 400 output tokens per model call. Option A uses 1.05 average calls per task; Option B uses 1.55. Everything else is held equal to isolate the cost of additional calls.
| Measure | Option A | Option B |
|---|---|---|
| Monthly token spend | $39.90 | $58.90 |
| Accepted original tasks | 4500 | 4500 |
| Cost per accepted task | $0.0089 | $0.0131 |
Monthly token spend = tasks × calls per task × (input tokens × input rate + output tokens × output rate) ÷ 1,000,000. Divide by accepted original tasks for cost per accepted result.
What the estimate leaves out
OCR, file conversion, storage, document retrieval, human verification and taxes are outside this token-only estimate. A document accepted after human correction should be reported separately from an automatically accepted extraction.
Replace the assumptions with measured usage and the current rate for the exact provider route and conditions. Review source-linked API prices · Check retry accounting
What happened next · API prices · Decision guides · Latest AI coverage · Follow updates