What counts as input

  • The system prompt and all instructions.
  • Conversation history you include for context.
  • Retrieved documents (RAG) pasted into the prompt.
  • Tool/function definitions and their schemas.
  • Images or files, converted to tokens per the provider's scheme.

What counts as output

  • The generated response text.
  • Structured output (JSON, code) — formatting tokens count too.
  • Reasoning or 'thinking' tokens on models that bill them.
  • Tool calls the model emits before the final answer.

Trimming each side

Trim input by summarizing history instead of appending it, retrieving fewer but better documents, and removing unused tool definitions. Trim output by requesting concise formats, setting max token caps, and avoiding 'explain your reasoning' unless you need it — some models count reasoning tokens toward billed output; verify the model's current rules.

Assumptions and limits

  • Billing details (reasoning tokens, caching) vary by provider — verify in current docs.
  • No provider prices quoted.

Common questions

Do system prompts count as input tokens?

Instructions included with a request contribute to input usage. Supported caching can change the rate for reused input, so inspect usage and your provider's current billing rules.

Are reasoning tokens billed?

On some providers and models, yes — internal 'thinking' tokens count as output tokens. Check your provider's current documentation, because this is exactly the kind of detail that changes.

Which side should I optimize first?

Measure first, then decide. Your input/output mix depends on the model, prompt, output, and caching. Optimize whichever billed category dominates your measured workload.

Last reviewed: . Developer guides for estimating AI API costs. Provider pricing changes often — always verify current rates with the provider.