Three tools that get reached for interchangeably
Somewhere in the last two years, "just fine-tune it" quietly became a reflex answer to almost any problem with an LLM's behavior — right alongside "just add it to the prompt" and "just RAG it." All three are real techniques. All three change what a model outputs. And that surface-level similarity is exactly why they get treated as interchangeable, when in practice reaching for the wrong one of the three is one of the most common and most expensive mistakes an AI engineering team can make.
Here's what actually distinguishes them, stated plainly before we go any further: prompt engineering changes behavior at the instruction level, per call, with no training involved. RAG changes what the model knows, by retrieving fresh information at query time. Fine-tuning changes the model itself — its weights, permanently — which makes it the only one of the three that can change how a model behaves in a way that survives without being re-specified every single time you call it.
That distinction — behavior versus knowledge, temporary versus permanent — is the entire spine of this lesson, and arguably of this course. Get it wrong and you'll spend a training budget solving a problem RAG would have fixed for the price of an API call, or spend months re-engineering prompts trying to force a model into a behavior only weight-level change was ever going to hold reliably.
What each tool actually touches
Start with prompt engineering, because it's the cheapest and the one you should almost always try first. It's instruction-level: you tell the model what to do, in the request itself, and it costs nothing beyond the tokens you send. Its ceiling is real, though — it's bounded by context window size, and by how reliably a model actually follows instructions once they're buried on page three of a long system prompt. A model that drifts off a formatting rule by the fortieth turn of a conversation isn't failing to understand the instruction; it's demonstrating the ceiling.
RAG solves a different problem entirely: it changes what the model *knows*, not how it behaves, by retrieving relevant material and injecting it into context at query time. Its defining advantage is that the knowledge base updates without touching the model at all — add a document, and the next query can use it immediately. That's precisely the right shape of solution when the problem is missing or stale information, and precisely the wrong shape when the problem is that the model's tone, judgment, or output format won't hold steady no matter what facts you feed it.
Fine-tuning is the odd one out, and it's odd in a specific, useful way: it's the only one of the three that changes the model's weights, which means it's the only one that can make a behavior change permanent — present by default, not something you have to re-assert in every prompt. That permanence is valuable exactly when it's needed and wasted exactly when it isn't, which is the whole judgment call this lesson is building toward.
The catch: All three techniques can often be made to produce a similar-looking output on a demo. The difference shows up under scale and drift — a prompted behavior degrades as context grows or instructions get buried; a fine-tuned behavior doesn't, because it isn't sitting in the prompt to begin with.
The distinction that actually decides it
If you remember one heuristic from this entire course, make it this one: fine-tuning is for teaching a model to behave a certain way; RAG is for giving a model facts it doesn't otherwise have. Everything else in this lesson is commentary on that sentence.
Take a concrete pair of examples. A support bot that needs to follow a very specific escalation format — always a summary line, always three suggested next steps, never a bare apology with no action attached — is a behavior problem. You can prompt for that format, and it'll mostly work, until the fortieth turn of a long conversation when the instruction has scrolled out of the model's effective attention and the format starts drifting. That's fine-tuning territory: teach the format directly into the weights, and it stops depending on the prompt holding it in place.
Now take a support bot that needs to answer questions about this week's product catalog. That's a knowledge problem, and it's RAG territory almost by definition — the catalog changes weekly, and "retrain the model every week to keep it current" is not a serious operational plan for almost any team. Confusing these two categories is the single most common misapplication in this space: teams fine-tune to inject knowledge that's going to be stale in a month, when RAG would have solved it more cheaply and stayed current automatically. Or they lean on RAG and elaborate prompting to force a behavior that never quite holds, when a fine-tune would have made the behavior the model's default instead of a rule it has to be reminded of every time.
Why this stays a specialist skill, not a default
It's worth being honest about where fine-tuning actually sits in the broader landscape before you invest in learning it deeply: it is genuinely niche relative to prompting and RAG in production systems today. Enterprise AI adoption surveys consistently place it well behind both, used primarily by teams with a specific, high-volume, well-defined task — not reached for as a general-purpose customization tool the way prompting is.
That's not a reason to skip this course. It's a reason to be precise about when fine-tuning earns its cost, which is a narrower question than "can it do the job." The clearest signal it's worth considering: a task run at high enough volume that a smaller, specialized model becomes cheaper per call than a frontier model with a long instruction prompt attached to every request — and the task's required behavior is stable enough that you're not retraining every few weeks chasing a moving target. Where either half of that signal is missing — low call volume, or frequently shifting requirements — fine-tuning is very often the wrong reach, even in cases where it's technically capable of doing the job well.
There's a second, more interesting wrinkle worth knowing about: the boundary between "optimize the prompt" and "fine-tune the model" isn't fixed, it's actively moving. Newer prompt-optimization techniques have shown they can match or beat some fine-tuning approaches on certain tasks, using a fraction of the compute — direct evidence that the line between these two tools is a live research question, not settled fact. Don't treat fine-tuning as a technique whose scope only ever grows; treat it as one whose boundary you'll need to keep re-checking.
Worth flagging: a specific research result (an ICLR 2026 oral) showed a prompt-evolution method beating a common reinforcement-learning fine-tuning approach by a wide margin, using far fewer training rollouts, on its benchmark tasks. Treat that as a snapshot of one paper's results on one set of tasks, not a permanent verdict on which approach wins — but do treat the direction as real: this boundary moves.
Ask the question separately, every time
In practice, production systems don't pick one of these three tools and retire the other two — they layer them. A fine-tuned model handles behavior and format discipline. RAG supplies it with current facts at query time. Prompt engineering handles per-request steering on top of both. None of the three makes the other two obsolete, and a system that only ever uses one of them is usually leaving something on the table.
The decision this lesson has actually been teaching isn't "which tool wins" — it's "which problem am I solving, for this specific capability gap," asked freshly each time rather than answered once for the whole product. A behavior problem gets fine-tuning. A knowledge problem gets RAG. A steering problem gets a better prompt. Mixing those up is where the expensive mistakes happen.
Once fine-tuning genuinely is the right call, the next question is how — and full fine-tuning, updating every parameter in the model, is rarely how it gets done anymore. The next lesson covers why: a technique called LoRA that updates a tiny fraction of a model's parameters and gets most of the benefit anyway, which is the idea that makes everything else in this course practical to actually run.