SourceVane · Practical AI

Reduce AI API costs without losing useful results

Measure workload, retries and accepted outputs before comparing models, batching or reusable prompts.

Direct answer

The decision in brief

Start with cost per accepted result, not the advertised token price. Measure every billed attempt, remove accidental retries and oversized context, then test model changes, batching and prompt caching against the same workload and acceptance rule.

Illustrative retry example

Conclusion: Under these assumptions, 1.2 billed attempts raise token spend from $9.00 to $10.80 per 1,000 original tasks. That is a 20% increase with token use and accepted tasks held fixed.

These are illustrative assumptions, not provider quotes or measured success rates: 2,000 input and 500 output tokens per attempt, $2 per million input tokens, $10 per million output tokens, and 800 accepted final tasks. Cost per accepted result is $0.01125 at one attempt and $0.01350 at 1.2 attempts.

Real retries may use different token counts or improve acceptance. Sum actual billed tokens and measure accepted original tasks after the retry policy; never count attempts as successful tasks.

Calculate your own scenario · Inspect the scenario study and downloadable inputs.

What should you change first?

Measure one representative period: tasks, billed input and output tokens, elapsed time, failures and results that pass your acceptance rule.
Fix accidental retries and oversized context before changing providers. Failed charged attempts belong in the total cost.
Compare models with the same task set and acceptance rule. Use cost per successful task rather than token price alone.
Choose batching when work can wait; test prompt caching only when an eligible prefix actually repeats. Record cache writes, reads and misses separately.

Use these as evaluation steps for your own workload. Record the evidence and limits before acting on the result.

Review the reproducible scenario and dataset · Calculate with your own workload