How token pricing works

Many text-generation APIs quote prices per million tokens, with separate rates for input tokens (your prompt, context, and tool definitions) and output tokens (the model's response). Units and rates can differ by model and service tier; check the provider's current price table before budgeting.

Cost = requests × (input tokens × input price + output tokens × output price) ÷ 1,000,000

Worked example (illustrative).

Illustrative: 10,000 requests × (2,000 input tokens × $0.50/M + 500 output tokens × $2.00/M) ÷ 1M = $10 input + $10 output = $20 total. Replace every rate with your provider's current published price.

What inflates the input side

  • Long context. RAG pipelines and conversation histories grow prompts silently; cap and summarize history.
  • Tool definitions. Tool definitions included in a request can contribute to input usage; remove definitions you do not need and measure the result.
  • System prompts. Long instructions add input usage; supported prompt caching can change the effective rate for reused content.

What inflates the output side

  • Verbosity. Ask for concise formats (bullets, JSON) when prose isn't needed.
  • Reasoning tokens. Some models bill internal reasoning as output tokens — check your provider's docs.
  • Retries. A retry that the provider processes can add usage; inspect response usage and billing records.

Estimate before you ship

Measure real token counts from a representative sample of your traffic — tokenizers vary by model and content, so estimates from one model don't transfer. Then model your expected request volume, add a buffer for retries and growth, and set a budget alert at 50% and 80% of it.

Assumptions and limits

  • No provider prices are quoted; examples use labeled illustrative rates.
  • Token counts vary by model and content — measure your own usage.
  • Cached tokens, image/audio inputs, and batch discounts are excluded unless you model them.

For one provider's current rules, compare OpenAI API pricing with its prompt-caching guide. Other providers and models can use different meters and rates.

Common questions

What is a token in AI pricing?

A token is a unit of text processed by a model. Many text-generation APIs report input and output token counts separately. Check your provider's current price table for the applicable units, rates, and any other charges.

Why can input and output rates differ?

Providers may set different input, output, cached-input, and service-tier rates. Check the current price table for the model and service you use.

How do I estimate tokens before calling the API?

Use the provider's tokenizer on representative prompts to count input tokens, and set a max-output cap to bound the output side. Then apply the formula above with current published rates. Re-measure after any prompt or model change.

Last reviewed: . Developer guides for estimating AI API costs. Provider pricing changes often — always verify current rates with the provider.