What is fine-tuning in AI? and why you almost certainly do not need it

TL;DR: Fine-tuning retrains a model on your examples so it internalises a style or format. It is the right tool for consistent output shape at high volume. It is the wrong tool for teaching a model facts — that is what retrieval is for — and it is almost never the first thing you should try.

The plain definition

A base model has already been trained on an enormous general corpus. Fine-tuning takes that finished model and continues training it on a much smaller set of your own examples — typically hundreds to thousands of input/output pairs. The result is a model that has absorbed a pattern: your tone, your classification labels, your output format.

What it is good at

• Consistent format. If you need the same JSON shape or the same report structure ten thousand times, fine-tuning enforces it more reliably than instructions.
• House style. A specific voice that is hard to describe but easy to demonstrate with examples.
• Narrow classification. Sorting support tickets into your seven internal categories, where your categories are idiosyncratic.
• Cost per call at scale. A fine-tuned smaller model can beat a large model with a huge prompt, both in price and latency.

What it is bad at

The big one: fine-tuning does not reliably teach facts. If you fine-tune on your company handbook, the model learns to sound like your handbook, not to answer accurately from it. Facts belong in retrieval — see the RAG explainer — where the model reads the document at question time instead of trying to remember it.

It also freezes you in time. Your fine-tune is built on today’s base model. When a better base model ships in four months, you either stay behind or redo the work.

The ladder you should actually climb

1. Better prompt. Specific role, explicit constraints, stated output format. Free, instant, fixes most problems.
2. Few-shot examples. Three to five examples of exactly what you want, in the prompt. This alone closes most of the gap fine-tuning would.
3. Retrieval. If the issue is the model not knowing your information, give it the information.
4. Fine-tuning. Only when 1-3 are exhausted, the task is stable, and the volume justifies the effort.

Almost every “we need a custom model” conversation I have seen ends at step 2.

What it actually costs

Not just the training run. The real costs are building a clean dataset of hundreds of high-quality examples, evaluating whether the fine-tune is genuinely better (harder than it sounds), hosting, and redoing it when the base model changes. Budget in weeks, not hours.

How to tell which problem you have

Ask: is the model getting the shape wrong, or the substance wrong? Wrong shape — inconsistent format, off-brand tone — is a fine-tuning-shaped problem, though better prompting usually solves it first. Wrong substance — inventing details, missing your specifics — is a retrieval problem, and fine-tuning will make it worse by making confident errors sound more fluent. That failure mode has a name: hallucination.