Glossary
Prompt caching
Charging a reduced rate for the repeated prefix of a prompt, such as a system prompt or a document reused across requests.
Cache reads cost 2.5% to 10% of the input price at the major vendors. Put stable content first in the prompt to benefit.
Related terms
Active parametersAPIAttentionLarge language model (LLM)Latency
Get the brief in your inbox
One email every morning: the stories that mattered in AI, each with a one-line "why it matters" for your role. Free, no ads, unsubscribe any time.
Published daily by AI Briefing. We never share your address.