LLM API pricing explained: tokens, caching, batch and the cliffs

Every major model vendor bills the same way: a price per million tokens sent in (input) and a separate, higher price per million tokens the model writes back (output). A token is roughly three-quarters of an English word, so a million tokens is about 750,000 words.

The basic formula

Cost per request = (input tokens × input price + output tokens × output price) ÷ 1,000,000.

A support bot that reads a 2,000-token conversation and writes a 300-token reply on a $2 / $10 model costs about $0.007 per reply. At 100,000 replies a month that is $700. The same bot on a $10 / $50 flagship costs $3,500. The model choice is the biggest lever you have.

Why output costs 5x input

Reading a prompt is processed in parallel; writing a reply happens one token at a time and ties up the hardware for the whole generation. Vendors price output at roughly five times input to reflect that. Practical consequence: asking for shorter answers saves more money than trimming prompts.

The four discounts

• Prompt caching. If many requests share the same long prefix (a system prompt, a policy document, a codebase), vendors charge a fraction of the input price for the repeated part, typically 2.5% to 10%. Anthropic charges extra to write the cache the first time; OpenAI charges 1.25x input for cache writes on its flagship. Structure prompts so the stable part comes first.
• Batch. Jobs that can wait minutes or hours get about 50% off at all three big vendors. Nightly enrichment, classification backfills and evaluation runs belong here.
• Smaller tiers. Flash, Haiku, mini and nano tiers are 5x to 50x cheaper than flagships and handle most classification, extraction and routing tasks well. Route by task difficulty.
• Shorter outputs. Cap max tokens and ask for structured output. Verbose prose is the most expensive thing a model produces.

The cliffs

Some models charge more for long requests, and the surcharge applies to the whole request: OpenAI's GPT-6 Astra doubles input above 272K tokens, Gemini Pro steps up above 200K, Claude Haiku 5.5 steps up above 100K. Introductory pricing also expires on published dates; Google's Gemini 4 Argon doubles on December 1, 2026. Check the pricing tracker before you forecast.

Cost per task, not cost per token

Tokenizers differ between models and even between generations of the same family, so the same text can be 30% more tokens on one model than another. Measure what a finished task costs end to end on each candidate model; the per-token rate card alone will mislead you.