The terms that keep showing up in the cards, and what changes once you know them.
28 terms
The first and by far the most expensive stage, where a model learns general language ability from a huge unlabeled corpus.
Only a handful of labs can pay for this stage; nearly every other release is work layered on top of someone else's finished base.
See also Post-training · Base Model · Scaling Law · Continued Pretraining
Taking an already-trained model and continuing training on your own data to specialize it.
When a company says it trained a model on its data, it almost always means this, not training from scratch, and both cost and ceiling differ.
See also Parameter-Efficient Fine-Tuning · Low-Rank Adaptation · Supervised Fine-Tuning · Continued Pretraining
Training a small student model on a big teacher model's outputs to carry over as much behavior as possible.
When a small model punches above its size this is usually why, and the teacher's terms of service are the recurring dispute.
See also Quantization-Aware Training · Synthetic Data · Pretraining · Quantization
Simulating low-precision rounding during training so the model still holds up after it is compressed.
A release that says quantization-aware is claiming its 4-bit build holds quality, unlike a checkpoint converted after the fact.
See also Quantization · Distillation · FP8 · QLoRA
Freezes the original weights and trains a pair of small low-rank matrices beside them, which can be merged back in or swapped out.
The output is a file a fraction of the base model's size that you can ship, swap or stack, which is why most community tunes are shaped this way.
See also Parameter-Efficient Fine-Tuning · QLoRA · Fine-tuning · Model Merging
The full set of weights saved at one point in training, which is the thing you actually download and run.
If the weights never ship, nobody can build on the paper; labs that also publish intermediate saves are worth far more to researchers.
See also Pretraining · Base Model · Model Merging · Scaling Law
Everything done after the base model exists: instruction tuning, preference training, and safety work.
Two models built on the same base can behave nothing alike; the gap almost always comes from this stage, not from pretraining data.
See also Supervised Fine-Tuning · Reinforcement Learning from Human Feedback · Direct Preference Optimization · Pretraining
The first post-training step, where the model imitates curated prompt-and-answer pairs written or vetted by humans.
This is where instruction-following manners come from: the difference between a base checkpoint and its Instruct sibling.
See also Post-training · Base Model · Reinforcement Learning from Human Feedback · Synthetic Data
A family of methods that freeze the original weights and train only a small added set, cutting trainable parameters by orders of magnitude.
The acronym is a declaration that no full fine-tune happened, and the quality ceiling is not the same as one either.
See also Low-Rank Adaptation · QLoRA · Fine-tuning · Catastrophic Forgetting
Loads the base model in 4-bit and trains low-rank adapters on top, so a large model fits on a single consumer GPU.
Most claims of fine-tuning a 70B at home mean this; the trade is slower steps and some loss of precision.
See also Low-Rank Adaptation · Quantization · Parameter-Efficient Fine-Tuning · Fine-tuning
Humans rank model outputs, a reward model learns those preferences, and the model is optimized to score higher against it.
Tone, refusals and safety behavior are shaped here; when a model suddenly gets more cautious after an update, this stage is the usual suspect.
See also Reward Model · Direct Preference Optimization · Post-training · Group Relative Policy Optimization
A separate model trained on human preference data that scores candidate answers so RL can optimize against it.
If the scorer is biased, the trained model maximizes that bias: this is the usual road to sycophantic answers.
See also Reinforcement Learning from Human Feedback · Direct Preference Optimization · Reinforcement Learning with Verifiable Rewards · Reward Hacking
Samples several answers per prompt and trains on their relative scores, removing the separate value network that PPO needs.
It took PPO's place in reasoning-model training; seeing the acronym in a tech report tells you the model went through a reasoning RL stage.
See also Reinforcement Learning from Human Feedback · Reinforcement Learning with Verifiable Rewards · Reward Model · Post-training
Trains directly on chosen-versus-rejected answer pairs, skipping the reward model and the RL loop entirely.
Simple and cheap enough that most community alignment tunes use it instead of full RLHF.
See also Reinforcement Learning from Human Feedback · Reward Model · Post-training · Synthetic Data
RL on tasks where a checker such as unit tests or a compiler decides right or wrong, so the reward needs no human and no reward model.
Sudden jumps in coding and math scores usually come from this setup; in QA terms, the test suite itself becomes the training signal.
See also Group Relative Policy Optimization · Reward Model · Reward Hacking · Self-Improvement Loop
A model exploits gaps in the scoring rule, driving the number up while abandoning the behavior that number was meant to stand for.
Code that special-cases the test to make it pass is this; when benchmarks rise while real use gets worse, suspect the scorer first.
See also Reward Model · Reinforcement Learning from Human Feedback · Reinforcement Learning with Verifiable Rewards · Group Relative Policy Optimization
Training data generated by a model rather than collected from humans, then filtered before use.
Most post-training data is now made this way; the filter is the real product, and an unfiltered loop amplifies its own errors.
See also Distillation · Data Curation · Supervised Fine-Tuning · Self-Improvement Loop
Filtering duplicates, junk and harmful documents out of a raw corpus and deciding how much of each source to include.
It moves benchmark numbers more than architecture does, and it is the section tech reports most often leave blank.
See also Pretraining · Synthetic Data · Data contamination · Curriculum Learning
Ordering training data from easy to hard, or shifting the data mix in stages, instead of feeding it all at random.
Late-stage runs often switch to a high-quality mix; the same data in a different order lands the model somewhere else.
See also Pretraining · Data Curation · Continued Pretraining · Scaling Law
Empirical curves predicting how loss improves as parameters, data and compute grow, used to plan a run before spending on it.
It is the math behind budget-to-model claims; a release only becomes news when it lands off the curve.
See also Pretraining · Mixture of Experts · Distillation · Checkpoint
A model that finished pretraining but got no instruction or preference training, so it continues text rather than answering.
When a repo ships both, the plain one is not a chatbot: it is raw material for someone else's fine-tune.
See also Pretraining · Supervised Fine-Tuning · Post-training · Checkpoint
A model loses previously learned abilities while being trained on new data.
It is why a domain fine-tune often gets worse at everything else, and one reason people freeze the base and train adapters instead.
See also Fine-tuning · Continued Pretraining · Low-Rank Adaptation · Parameter-Efficient Fine-Tuning
Feeding a released model a large domain corpus in the same unlabeled way, before any instruction tuning.
This is how medical, legal or single-language models appear; it needs orders of magnitude more data than a fine-tune.
See also Pretraining · Fine-tuning · Catastrophic Forgetting · Data Curation
Combining the weights of several models arithmetically to make a new one, with no additional training.
It explains how new models appear on hubs overnight: nearly free in GPU time, and impossible to trace which parent gave which ability.
See also Low-Rank Adaptation · Checkpoint · Fine-tuning · Uncensored Model
A loop where a model generates attempts, a checker keeps the good ones, and those become training data for the next round.
Titles about self-evolving systems usually mean this loop; when the checker is weak, the loop just amplifies the model's own mistakes.
See also Reinforcement Learning with Verifiable Rewards · Synthetic Data · Distillation · Reward Hacking
The step size of each weight update, usually warmed up at the start of a run and then decayed along a fixed schedule.
It is why fine-tuning guides hardcode a number like 2e-5 with no explanation: too high and the model breaks what it knew, too low and the run changes nothing.
See also Fine-tuning · Optimizer · Training Loss · Catastrophic Forgetting
The rule that turns gradients into actual weight updates; AdamW has been the default for years, and Muon is the current challenger.
A release note that says 'trained with Muon' is a cost claim - same data, same size, fewer GPU-hours - and swapping optimizers means retuning the learning rate.
See also Learning Rate · Training Loss · Pretraining · GPU-hour
One number for how far the model's predictions sit from the training data; it slides down on a healthy run and spikes when it breaks.
It is the gauge behind lines like 'we rolled back to a checkpoint from three days ago' - a spike decides whether days of GPU time get thrown away.
See also Checkpoint · Learning Rate · Perplexity · Pretraining