Expert parallelism
Splitting a mixture-of-experts model across GPUs by expert rather than by layer, so each GPU holds a subset of the experts and tokens are routed to the right device.
Large MoE models such as Inkling 2 Small (8 of 384 experts active per token) are served with expert parallelism, often combined with tensor parallelism. Some model cards require the two degrees to be equal, which constrains which GPU counts and topologies work. If a serving framework has not merged support for a model's parallelism scheme, the weights are downloadable but not yet deployable.
Related terms
ComputeGPUInferenceMixture of experts (MoE)NVFP4
Get the brief in your inbox
One email every morning: the stories that mattered in AI, each with a one-line "why it matters" for your role. Free, no ads, unsubscribe any time.
Published daily by AI Briefing. We never share your address.