The terms that keep showing up in the cards, and what changes once you know them.
28 terms
A neural network built from stacked layers in which every token weighs every other token; the base design under almost all current LLMs.
The skeleton has barely changed since 2017, so what a new release actually changes is usually just the attention scheme and the routing.
See also Attention · Mixture of Experts · Decoder-only
The operation that scores how much each token should read from every other token, then blends their information in those proportions.
Its cost grows with the square of the input length, which is why every long-context or inference-cost argument ends up back at this one step.
See also Transformer · KV Cache · FlashAttention · Grouped-query attentionShowed up in Papers
A layout that holds many expert sub-networks and routes each token through only a few of them instead of all.
It backs claims of a huge model that runs fast, but all experts still sit in memory, so VRAM demand does not drop.
See also Active parameters · Dense model · Parameter count
The property of one model handling inputs of different kinds, such as text, images, audio and video, within a single representation.
Image support is not one thing: check whether a model only reads images or also produces them before trusting the label.
See also Vision-language model · Encoder · Diffusion model
A layout with a single stack and no separate encoder, where each position is masked to see only earlier tokens and predict the next one.
GPT, Qwen and DeepSeek all use it, so one-token-at-a-time generation is the root of how latency, streaming and per-token billing behave.
See also Transformer · KV Cache · Encoder
The part of a model that reads an input whole and turns it into meaning-bearing vectors, without generating anything itself.
A model built only from these, like BERT, cannot chat: it is meant for classification and search, and gets wired into a system differently.
See also Embedding · Vision-language model · Multimodal
An attention variant where several query heads share one set of keys and values, shrinking the cache instead of duplicating it per head.
It is close to standard now, so an older model without it burns several times the memory on a long conversation at the same parameter count.
See also KV Cache · Attention · FlashAttention
A way of encoding token order by rotating the vectors, now the default position scheme in most open models.
Most context-extension announcements are rescalings of this rotation, which is why quality slips beyond the length the model trained on.
See also Attention · Context window · Transformer
The share of a model's weights that actually take part in computing a single token, as opposed to the total it stores.
A name like 30B A3B means it stores 30B and runs 3B per token: read speed off the second number and memory off the first.
See also Mixture of Experts · Parameter count · Dense model
The classic layout in which every weight participates in every token, with no routing or selective activation.
A 27B of this kind computes all 27B per token: slower but more uniform, and with far more finetuning and quantization precedent.
See also Mixture of Experts · Active parameters · Parameter count
The total number of learned numbers a model holds; the B in a model name means billions of them.
It sets the memory bill: roughly parameter count times bytes per weight, so a 27B at 8 bits needs about 27GB before overhead.
See also Quantization · FP8 · Mixture of Experts
A class of language models sized to run on a laptop, phone or edge device rather than a server rack.
They get chosen for latency, cost and privacy, not raw quality, so compare them against response-time budgets, not frontier models.
See also Distillation · Quantization · MLX · Frontier model
An architecture that carries a fixed-size running state instead of re-reading all past tokens, so cost grows linearly with length.
It is the line of work avoiding attention's quadratic cost on long inputs, and current releases usually hybridize the two.
See also Attention · Context window · Transformer
A generative approach that starts from noise and removes it over many steps to produce an image, video or audio clip.
Unlike models that append one token at a time, step count is both quality and cost, which is why releases lead with how few steps they need.
See also Multimodal · Transformer · Vision-language model
A model class that turns images into vectors and feeds them into a language model so it can answer about what it sees.
Screenshot-driven QA rides on these, and small text or coordinates break first at the vision side's resolution limit.
See also Multimodal · Encoder · Embedding
Industry shorthand for the largest, most capable models currently at the performance edge, usually closed and served by API.
Policy and safety announcements use it to draw the line for what gets regulated; it marks a boundary more than a technical tier.
See also Small language model · Parameter count · Mixture of Experts
A release further trained to follow instructions, as opposed to the base version that only continues text.
Chatting with the base build of the same model gives strange replies; nothing is broken, you just downloaded the wrong variant.
See also Low-Rank Adaptation · Checkpoint · Distillation
A model trained to generate a long chain of intermediate work before answering, trading tokens and latency for accuracy on hard tasks.
A thinking budget on a card marks this class; the hidden reasoning tokens are billed and delay the first answer, so it pays off only on genuinely hard work.
See also Reinforcement Learning with Verifiable Rewards · Post-training · Token Pricing · Time to First Token
One stacked block of attention plus a feed-forward part, repeated dozens of times to form the model, with the repeat count being its depth.
A 48-layer spec is a depth claim, and since layers run strictly in sequence, a deeper model of the same size pays more latency per token.
See also Attention head · Feed-forward network · Transformer · Parameter countShowed up in Papers
Parallel slices within one attention block, each computing a different relation between tokens before their outputs are concatenated.
It explains specs that list query heads and KV heads separately: sharing one KV set across several query heads to shrink the cache is what GQA does.
See also Attention · Grouped-query attention · KV Cache · Layer
The two-layer transform that follows attention in every block; it holds most of a model's parameters.
MoE replicates this part and nothing else, which is why expert count can grow while active parameters stay flat, and a card's SwiGLU refers to it too.
See also Mixture of Experts · Active parameters · Layer · Expert routing
The small gate that picks which experts each token is sent to, and that choice is what keeps only a slice of the weights active.
MoE failures usually live here, not in the experts: routing collapses onto a few of them unless training adds a separate load-balancing loss.
See also Mixture of Experts · Active parameters · Feed-forward network · Model router
A stack that mixes layer types, running mostly cheap linear or state-space layers with a few full-attention layers kept for recall.
Releases claiming flat memory on long inputs usually mean this, and the cost is that serving stacks often need new kernels before they can run it at all.
See also State space model · Attention · KV Cache · Layer
Restricting each token to a fixed nearby span instead of the whole sequence, usually with a few full-attention layers left in the stack.
It is the cheap route to long context: information outside the window only travels layer by layer, so distant needle-in-a-haystack retrieval breaks first.
See also Attention · KV Cache · Context window · Hybrid architecture
The mechanism that tells an otherwise order-blind model where each token sits in the sequence.
It is the only channel carrying order, so a context window stretched several-fold usually means this scheme was rescaled, and accuracy leaks past trained distances.
See also Rotary position embedding · Context window · Attention · Continued Pretraining
The front end that turns an image into tokens the language part can read, usually trained separately and then attached.
It sets how many tokens one image costs, so raising resolution multiplies the bill, and small-text failures are usually its limit rather than the language model's.
See also Vision-language model · Multimodal · Encoder · Token Pricing
The raw per-token scores a model emits at each step, one per candidate, just before they become the probabilities sampling draws from.
Structured output, grammar constraints and speculative decoding all work by touching these, so an API that hides them keeps that control server-side only.
See also Structured output · Speculative Decoding · Distillation · Inference
An attention variant that caches keys and values compressed into a small latent vector and expands them back at use time.
It shrinks the KV cache several-fold at the same model size, which is why long-context releases advertise it in place of GQA.
See also Grouped-query attention · KV Cache · Attention head · Attention