What Is In-Context Learning? How AI Learns Live

In-context learning is what happens when a large language model changes its output based only on the examples and instructions inside a single prompt, with no update to its underlying weights. Show it a pattern once or twice and it follows the pattern, and the instant the conversation ends the model has forgotten it, nothing was actually learned the way a person learns something. This is different from fine-tuning, which permanently changes a model's weights through an additional training run. It is also different from the context window, which is only storage capacity, the maximum number of tokens the model can read at once. In-context learning is what the model does with the space it has, not how much space it has.

Zero, One, and Three Examples: Watch the Output Change

The clearest way to see in-context learning happen is to run the same instruction three times and change nothing except how many examples come before it. Below is one task, a support team turning customer complaints into ticket titles, run with zero examples, one example, and three.

Instruction: Rewrite this complaint as a one-line support ticket title.

Complaint: "The app keeps logging me out every time I switch tabs on my phone, it's been happening since the last update and it's honestly making me want to switch apps."

Output: User Gets Logged Out Frequently When Switching Tabs on Mobile After Recent Update

Example: Complaint: "The export button does nothing when I click it on Safari." -> Title: [Export] Button unresponsive on Safari

New complaint: (the same complaint as above)

Output: [Login] Session drops when switching tabs on mobile after update

Example 1: Complaint: "The export button does nothing when I click it on Safari." -> Title: [Export] Button unresponsive on Safari

Example 2: Complaint: "The dashboard chart doesn't update after I change the date range." -> Title: [Dashboard] Chart not refreshing on date range change

Example 3: Complaint: "I can't invite teammates, the invite link says expired immediately." -> Title: [Invites] Link expires immediately on creation

New complaint: (the same complaint as above)

Output: [Auth] Session drops when switching tabs on mobile

The instruction never changed. What changed was how tightly the output matched an unstated convention. Zero examples produced a grammatically fine sentence, useful to a human, useless as a ticket title in a tracker that expects a bracketed component tag. One example fixed the format but guessed the wrong component, "Login" instead of the product's actual term for that subsystem. Three examples fixed both: the bracket format locked in, and the component name shifted to "Auth" once the pattern of naming a subsystem rather than a user action showed up twice in a row. Nothing was trained. The model simply had more data, inside the prompt, to infer the rule from.

How LLMs Learn From Examples in a Prompt

Mechanically, every token in a prompt, instructions and examples alike, gets attended to when the model generates its next token. Examples don't get stored anywhere; they shift the probability distribution over what a plausible output looks like, for this one generation, because the model is pattern-matching across everything currently sitting in its context. Add a fourth or fifth example and the pattern usually sharpens, up to a point of diminishing returns, after which more examples just cost tokens. That's a narrower question than how a language model processes a prompt from input to output more broadly, which covers the mechanics behind this behavior.

In-Context Learning vs Fine-Tuning

The two get confused because both make a model behave differently on a specific task. The difference is where the change actually lives.

• Fine-tuning changes the model's weights through what happens during a fine-tuning run, while in-context learning leaves every weight untouched.

• Fine-tuning needs a training pipeline, compute, and a labeled dataset; in-context learning needs nothing but better examples in the next prompt.

• A fine-tuned behavior persists across every future conversation until the model is retrained; an in-context pattern disappears the moment the examples are gone.

• Fine-tuning suits a behavior you want every single time regardless of prompt; in-context learning suits a behavior that should change by task, client, or use case.

In-Context Learning vs a Longer Context Window

A bigger context window and better in-context learning sound like the same upgrade, and they are not. The window is a limit, expressed in tokens, on how much text a model can read in one pass, including how a context window's size and cost get set. In-context learning is what the model does with whatever text sits inside that limit. A million-token window filled with irrelevant text produces nothing useful; three well-chosen, clearly patterned examples in a four-thousand-token prompt can outperform it. Room to add examples is necessary for few-shot prompting to be possible at all. It is not sufficient for the prompting to actually work.

Few-Shot Learning: Why Examples Can Beat a Longer Instruction

An instruction describes a rule in words. An example shows the rule as data, and data resolves ambiguity that words leave open. Tell a model to keep responses concise and it has to guess what concise means for your case. Show it three concise responses and it copies the actual length, tone, and structure instead of a paraphrase of the word concise. This is why few-shot learning in AI so often outperforms a longer, more detailed instruction: a paragraph of qualifiers is still a description, while two or three worked examples are the thing itself. The trade-off is prompt length and the cost of sending examples on every call, weighed against how much an instruction alone leaves to chance, a balance covered in more detail in the practical trade-offs between zero-shot and few-shot prompting.

Frequently Asked Questions

Does in-context learning change the model's weights?

No. Nothing about the model itself is altered. The effect lasts only as long as the examples stay inside the active prompt or conversation.

How many examples does few-shot learning need?

There is no fixed number. One example is often enough to fix a format; three to five usually locks in a subtler pattern like tone or edge-case handling. Past that point, additional examples tend to cost tokens without changing the output much.

Is in-context learning the same as retrieval-augmented generation?

No. Retrieval-augmented generation pulls relevant documents into a prompt before generation happens. In-context learning is what the model does with whatever text is in the prompt, including text that retrieval placed there. Retrieval is one way context arrives; in-context learning is the mechanism that uses it.

Does a larger context window improve in-context learning?

Not by itself. A larger window gives you room to include more examples, but those examples still have to be relevant and consistently patterned. A short prompt with three sharp examples generally beats a long one padded with unrelated text.

Can in-context learning fail?

Yes. Inconsistent examples, contradictory formatting, or an ordering where the wrong pattern gets picked up first can all push the output somewhere unintended. In-context learning, explained simply, is pattern matching, and pattern matching is only as reliable as the pattern actually present in the examples.