A good pretrained model can still be ruined in a few updates
Fine-tuning begins with useful weights, but that does not make optimization forgiving. A learning rate that jumps to full strength immediately can push those weights away from a good region before the task signal becomes reliable. An unusual batch can also produce a gradient large enough to destabilize the run.
Linear warmup avoids the first failure. The learning rate rises from zero to its peak over the first few percent of steps, then decays. BERT commonly uses linear decay; GPT-2, RoBERTa, and many modern models use a gentler cosine decay after warmup.
Gradient clipping addresses the second failure. When the gradient L2 norm exceeds a threshold, commonly 1.0, you scale it down before the optimizer step. This does not repair a bad objective, but it prevents one extreme batch from causing a catastrophic update.
Tradeoff: full fine-tuning of BERT-family models often uses learning rates from 1e-5 to 5e-5. LoRA adapters can use roughly 1e-4 to 5e-4 because the pretrained weights remain frozen. Larger models generally need the more conservative end of the range.
Fit the computation you need into the memory you have
Mixed precision cuts memory and increases throughput by performing most forward and backward computation in 16-bit formats while retaining master weights in FP32. FP16 offers the savings but can underflow on small gradients, so it needs loss scaling. BF16 keeps FP32’s exponent range and is preferred on A100 and H100 hardware because it avoids that underflow risk.
Automatic mixed precision in PyTorch or the HuggingFace Trainer handles those conversions. The source notes describe near-zero accuracy loss with about twice the speed and half the memory compared with pure FP32 training.
If the desired batch still does not fit, gradient accumulation simulates it. Run several forward and backward passes without stepping the optimizer, add their gradients, then update once. A per-device batch of 8 accumulated for 32 steps gives an effective batch of 256 on one device.
Tradeoff: accumulation reproduces the gradient of a larger batch within reason, but it takes multiple sequential passes. It solves memory pressure, not wall-clock throughput.
Distribute data, model state, or the model itself
The right distributed strategy depends on what does not fit. If the full model fits on each GPU, DistributedDataParallel is the standard choice: every GPU holds a replica, processes a different data shard, and averages gradients with AllReduce after each step.
If model parameters, gradients, and optimizer states exceed one GPU’s memory, Fully Sharded Data Parallel divides all three across devices. If individual weight matrices are too large, tensor parallelism splits those matrices within a layer. For very deep stacks, pipeline parallelism places different layer ranges on different GPUs.
These approaches solve different constraints. A 70-billion-parameter model may require tensor parallelism for its matrices, while a smaller model that nearly fits can use FSDP. Pipeline parallelism needs careful microbatching because one stage can sit idle while another works, creating pipeline bubbles.
Tradeoff: every form of parallelism buys memory or throughput with communication and coordination overhead. Start with DDP when the model fits; move to sharding or model parallelism only when the memory boundary requires it.
Choose how much of the model must move
Full fine-tuning updates every parameter and can deliver the best task quality when you have enough labeled data, compute, and memory. The cost is a full model copy per task, and narrow data can cause catastrophic forgetting. The source notes suggest it when you have more than about 10,000 task examples and maximum quality justifies the budget.
Feature extraction freezes the backbone and trains only a lightweight head on stored representations. It is fast, cheap, and cannot overwrite pretrained capabilities, but it is usually weaker because the representations never adapt to the task.
Prefix tuning and prompt tuning freeze the base model and learn continuous prompt vectors. Prefix tuning inserts them into keys and values across attention layers; prompt tuning adds them only at the input. They work best at large scale and often underperform on models below one billion parameters or under a large distribution shift.
Tradeoff: updating more weights increases task-specific capacity and storage cost. Freezing more weights makes each task cheap to store, but limits how deeply the representation can adapt.
Move the task through a low-rank path
LoRA freezes an existing weight matrix W and learns a low-rank update, written W plus B times A. If the rank is 8 for a 768-by-768 attention matrix, the original matrix has 589,824 parameters while the two LoRA matrices contain only 12,288. The adaptation path is about two percent of that matrix.
This works because useful fine-tuning updates often lie in a much smaller subspace than the full weight matrix. Ranks from 4 to 16 are sufficient for many tasks. Attention query and value projections are common targets, with key, output, and feed-forward projections added when the task needs more capacity.
At inference, you can merge the learned update into the base weights, so LoRA does not have to add latency. QLoRA pushes the memory reduction further: load the frozen base model in 4-bit NF4, compute LoRA gradients in BF16, and store only the adapters at high precision. The notes report that this makes a 70-billion-parameter model, normally about 140 gigabytes in FP16, fit on one 48-gigabyte GPU with minimal quality loss.
Tradeoff: QLoRA makes single-GPU adaptation practical, but the quantized base constrains the update path. Use full fine-tuning when the data and quality target justify it; use LoRA or QLoRA when memory, per-task storage, or catastrophic forgetting is the binding constraint.
Treat training as a chain of constrained choices
Stable adaptation is not one trick. Warmup and decay control the size of updates. Clipping limits rare extremes. Mixed precision and accumulation fit useful batches into memory. Distributed strategies divide the workload according to whether data, state, matrices, or layers are the bottleneck.
Then choose the smallest adaptation surface that can meet the quality target. Full fine-tuning moves every weight; feature extraction moves only the head; prompt methods move learned vectors; LoRA and QLoRA move compact low-rank adapters. In current single-GPU workflows, QLoRA is the practical default for adapting a large causal model.
The result still has to solve a concrete language problem. In the next lesson, we will map these architectures and adaptation choices onto classification, sequence labeling, parsing, and structured extraction.