Training vs inference: the AI distinction that explains most confusing headlines

TL;DR: Training is the one-off, enormously expensive process of building a model. Inference is the cheap-per-use process of running it to answer your question. Your chats do not train the model in real time — and knowing that clears up most of the confusion.

Training: building the thing

Training is where a model learns statistical patterns from a vast corpus of text, images and code. It runs for weeks or months across thousands of specialised chips, costs somewhere between tens of millions and hundreds of millions of dollars for a frontier model, and produces one artefact: a giant file of numbers called weights.

Critically, training happens once (plus periodic updates). When it ends, the model is frozen. That freeze date is what people mean by a “knowledge cutoff.”

Inference: using the thing

Inference is what happens when you send a message. The frozen weights are loaded, your input runs through them, and text comes out. It costs fractions of a cent, takes seconds, and — this is the part people miss — changes nothing about the model. The weights are identical before and after your conversation.

What this explains

• “Why does it not know about last week?” Because the weights froze before then. Search-connected tools like Perplexity get around this by retrieving current pages at inference time, not by knowing more.
• “Does it learn from my corrections?” Not within the model. It remembers inside the conversation because the history is resent each turn — see how tokens work. Start a new chat and the correction is gone.
• “Is my data training it?” Only if the provider separately saves your conversations and uses them in a future training run — a policy choice, not an automatic consequence of you typing. Consumer tiers often do by default; business and API tiers usually do not. Check the setting rather than assuming either way.
• “Why did it get worse?” Usually inference-side changes — routing, quantisation, safety layers — not the model unlearning. It cannot unlearn; the weights are static.

Where the money goes

Training is the headline number, but inference is the ongoing business. Every query costs electricity and chip time, forever, at enormous scale. That is why providers push smaller and distilled models, why “thinking” modes cost more (more generation per answer), and why usage caps exist on flat-rate plans.

It is also why the price of a given capability keeps falling: the training bill is sunk, and inference efficiency improves relentlessly. What cost a fortune in 2023 is close to free now.

The practical version

1. If you need current information, you need a tool that retrieves at inference time — not a “newer” model.
2. If you need the model to know your documents, that is retrieval, also at inference time.
3. If you want privacy, the lever is the provider’s data-retention setting, not avoiding certain phrasings.
4. If you want to change how it behaves, use a system prompt — it is an inference-time instruction and takes effect instantly.