The terms that keep showing up in the cards, and what changes once you know them.
30 terms
The smallest chunk of text a model reads or writes; a single word often splits into several of them.
Pricing, context limits, and speed are all counted in this unit, and Korean text splits into more of them than English for the same content.
See also Tokenizer · Context window · VocabularyShowed up in Discussion
The rule table and code that splits raw text into the units a model actually consumes.
Swap models without matching this and the output turns to garbage; the same sentence also costs a different number of tokens depending on which one is used.
See also Token · Vocabulary · Embedding
The full list of tokens a model can recognize, usually tens to hundreds of thousands of entries.
Characters missing from it get shredded into more tokens, which is where a multilingual model's handling of Korean is actually decided.
One of the numbers inside a model whose value is set by training; their count is what people mean by model size.
The 27B in a model name means 27 billion of them, which roughly fixes how much memory and what class of GPU you need to run it.
See also Weights · VRAM · Quantization
The trained parameter values themselves, saved as the files you download to run a model.
Releasing a model usually means uploading these files; the model is this pile of numbers, not the code that loads it.
See also Parameter · Open weights · Quantization
A list of numbers that places words or images so that similar meanings land close together.
Search, recommendation, and document retrieval all run on distances between these, and a paper saying representation learning usually means building better ones.
See also Retrieval-Augmented Generation · Token · MultimodalShowed up in Papers
The maximum number of input plus output tokens a model can hold in view at once.
Most cases of an agent forgetting come from this limit, and since raising it costs money, memory and summarization schemes show up right beside it.
Everything fed into the model for one turn: the question plus the rules, examples, and documents attached to it.
The same model performs very differently depending on it, and arguments over files like AGENTS.md are really arguments about what belongs in here.
See also Token · Context window · Agent
A very large neural network trained on huge amounts of text to predict the next token, and therefore to write.
Most models in the news are variants of one design, so however different the names sound, they compare on the same axes: parameters, context, quantization.
See also Parameter · Autoregressive generation · TransformerShowed up in Papers · Releases · Discussion
Generating one piece at a time by predicting what comes next, then feeding that prediction back in as input.
One wrong piece drags along everything after it, and both streaming output and tokens-per-second metrics fall out of this loop.
See also Token · Inference · Hallucination
Running an already-trained model to produce an answer, and the compute each of those runs burns.
Training happens once but this costs money on every request, which is why serving, edge, and quantization stories are all about shaving it.
See also Quantization · VRAM · KV CacheShowed up in Papers · Releases
A release where the trained parameter files are downloadable, so anyone can run the model on their own hardware.
It does not mean the code or the training data is open; the term gets used interchangeably with open source, but the license usually is not.
See also Weights · GGUF · Quantization
The phase where a model is shown data over and over and its internal numbers are nudged until the predictions get better.
It happens once and then freezes into files, so the thousands-of-GPUs figures in the news are almost always this, not the cost of using the model.
See also Inference · Pretraining · WeightsShowed up in Papers
A stack of layers that multiplies and adds numbers to turn an input into an output, with those numbers learned from data.
Everything in today's AI sits on this one structure, and a transformer is just one particular answer to how the layers get stacked.
See also Transformer · Parameter · Training
The umbrella term for models that produce new text, images, audio, or code instead of classifying or scoring what already exists.
When an announcement leans on this phrase it usually means a language or image model, and that quality gets judged by output rather than one accuracy score.
See also Large language model · Diffusion model · Multimodal
Steering a model by pasting a handful of worked examples into the input instead of changing the model itself.
Doing the same with no examples is called zero-shot, and this is the cheap move people tell you to try before reaching for fine-tuning.
See also Prompt · Fine-tuning · System prompt
A dial on how boldly the model picks each next piece: near zero it always takes the safest option, higher it wanders.
Low for code and extraction, higher for writing, and when someone reports that the output keeps changing this is the first setting to check.
See also Top-p · Nondeterminism · Autoregressive generation
A sampling cutoff that keeps the likeliest candidates until their probabilities add up to a set share, and drops everything below.
It travels next to temperature but cuts in a different way, which is why guides for running models locally tell you to tune it alongside top-k.
See also Temperature · Autoregressive generation · Nondeterminism
A cap on how many tokens one response may generate; hit it and the answer simply stops mid-sentence.
Most reports of an answer getting cut off trace back to this setting rather than the model, and it is a different number from the context window.
See also Context window · Token · Streaming
A string registered in advance so that generation halts the moment the model produces it.
It keeps the model from writing the other speaker's next line, and a badly chosen one chops valid answers off in the middle.
See also Max tokens · Chat template · Autoregressive generation
The instruction block placed ahead of the user's message that fixes the model's role, tone, and hard limits.
Most of a product's behavior lives here, which is why leaks make headlines, and it rides along on every request so you pay for it again unless it is cached.
See also Prompt · Prompt Injection · Prompt Caching
A conversation carried on by re-sending the whole previous exchange along with every new message.
The model remembers nothing between calls, so every extra turn costs more and the oldest part of the exchange is what gets dropped first.
See also Context window · Agent memory · Token
Sending each generated piece the moment it appears instead of waiting for the whole answer to finish.
It makes the wait feel shorter without making it shorter, and it is why checking or filtering the full answer before showing it gets awkward.
See also Time to First Token · Latency · Autoregressive generationShowed up in Models
The wall-clock time between sending a request and getting the answer back: what one user actually feels as slowness.
Throughput can improve while this gets worse: bigger bundles help the server and hurt the person waiting, so the two numbers have to be read together.
See also Throughput · Time to First Token · Batch size
How many requests or training examples get processed together in a single pass.
GPUs get more efficient the more you bundle, at the price of waiting, and the number in a training paper is not the number in a serving config.
See also Latency · Throughput · Continuous Batching
The property that the same input with the same settings can still come back different on each run.
It is why a test that passed yesterday fails today; temperature zero narrows it but never removes it, since GPU addition order shifts with batching.
See also Temperature · Eval · Reproducibility
The document that says how far you may go with weights you downloaded: commercial use, redistribution, derivatives.
Released does not mean unrestricted: user-count caps, non-commercial clauses, and naming requirements are all common.
See also Open weights · Open-source model · Weights
Strictly, a release that opens the training code and data as well as the weights, not just the downloadable files.
Cards use the phrase loosely and most of the time only the weights are actually open; the license is what settles which one you are looking at.
See also Open weights · Model license · Weights
The de facto public warehouse where model files and datasets are uploaded, versioned, and downloaded.
When you hear a model has been released, it usually means it showed up here, with the model card, license, and quantized builds on one page.
See also Open weights · Model Card · GGUFShowed up in Discussion
The format that stitches roles and messages into the single string a model was actually trained to read.
When a model you run yourself suddenly talks nonsense, a mismatched one is a common cause; it differs per model and ships with the weights.
See also System prompt · Multi-turn · GGUF