The big picture
Fine-tuning does not build a model from scratch. It nudges an existing model with many examples of the behaviour you want.
- 01Sessionspick chats
- 02Datasetcollect examples
- 03Providerwhere it trains
- 04Data prepclean + convert
- 05Trainrun the job
- 06Evaluateprove it improved
- 07Test chattry it yourself
- 08Useput it to work
Tokens and context
Models read and write tokens, not characters or words. Almost every limit and every price in training is counted in them.
From chats to training examples
A chat becomes training data by deciding which of its replies the model should learn to write.
Datasets
A frozen, named collection of examples — from real chats, generated chats, public datasets or chat feedback — and the single biggest factor in how good the trained model turns out.
Data preparation
Data prep turns dataset examples into the exact training files a provider expects: cleaned, redacted, trimmed and split into training, validation and test sets.
What happens during training
Training is a loop: predict, measure the error, work out how each weight should change, change it a little — and repeat thousands of times.
Training methods
The methods differ in what signal the model learns from — and therefore in what data you need and what each can and cannot teach.
Preference tuning
Reinforcement fine-tuning
Distillation
Learning from documents
Evaluation
Loss curves show how well the model fits. Evaluation shows whether its answers are actually better.
Trying and serving a model
A trained model has to be deployed before it can answer — and a dedicated deployment costs money for every hour it exists.
Glossary
Every concept, A to Z. Each opens its full page.
- Annotation queues: checking the judge by hand
- Archiving runs
- Base vs tuned, side by side
- Beta (β) in DPO
- Checkpoints: when the base model is gone
- Choosing a training method
- Cleaning
- Comparing several models
- Context window
- Continued pre-training
- Continuing from a trained model
- Distillation
- Distillation settings
- Epoch, batch and step
- Full fine-tuning
- Graders: how answers are scored
- Gradient checkpointing
- GRPO, step by step
- Held-out checks in reinforcement fine-tuning
- How training is billed
- Hyperparameters
- Inference and deployment
- JSONL training files
- Learning from chat feedback
- Learning from documents: Q&A generation
- Learning rate
- LLM-as-a-judge
- LoRA and adapters (PEFT)
- Loss masking
- Message roles
- Multi-turn conversations and weights
- Off-policy distillation: the teacher writes answers
- On-policy distillation: the teacher grades the student
- Open-weight models
- Overfitting and underfitting
- Personal data and secrets (PII)
- Pre-training and the base model
- Preference accuracy
- Preference tuning (DPO)
- Prompting, retrieval or fine-tuning
- Public datasets from Hugging Face
- QLoRA (4-bit LoRA)
- Reading evaluation results
- Reading the DPO loss
- Reading the loss curves
- Reinforcement fine-tuning (RFT)
- Reward hacking
- RFT metrics to watch
- RFT settings
- Scoring a model on its own
- Serverless vs dedicated
- Supervised fine-tuning (SFT)
- Synthetic data and the Dataset Builder
- System prompt
- Targets (trainable replies)
- Test questions: why “held out” matters
- The standard recipe: SFT, then DPO
- The training loop
- The training pipeline
- Token limits and trimming
- Tokens
- Validation split
- What a dataset is
- What it costs: reading the numbers
- What makes a good dataset
- Where good preference pairs come from
- Why tokens drive cost
- Will the model fit? Memory for training