Glossary
Latency
How long a model takes to respond, usually measured as time to first token and tokens per second.
Smaller tiers and shorter outputs are faster. Reasoning budgets and long contexts add latency.
Related terms
GPUInferenceMixture of experts (MoE)
Get the brief in your inbox
One email every morning: the stories that mattered in AI, each with a one-line "why it matters" for your role. Free, no ads, unsubscribe any time.
Published daily by AI Briefing. We never share your address.