What Is Catastrophic Forgetting in AI?

What is catastrophic forgetting in AI? It is what happens when a neural network learns something new and, in the process, overwrites what it already knew. The classic case: you fine-tune a model on 800 support tickets, it becomes excellent at classifying support tickets, and it simultaneously gets noticeably worse at summarising, at following formatting instructions, and at speaking any language other than the one in your tickets. Nobody asked it to give those up. The training gave it no reason to keep them.

The name is not hyperbole. The effect was first documented in the late 1980s, and the word catastrophic was chosen because the loss is not gradual. Performance on old tasks can collapse within a few hundred gradient steps.

Why catastrophic forgetting in AI happens

A neural network stores everything it knows in one shared set of weights. There is no folder for French, no separate module for arithmetic. The same parameters carry all of it, distributed across the network in overlapping patterns.

Training adjusts those weights to reduce error on the data in front of it. When the data in front of it is 800 support tickets, gradient descent has exactly one objective: get better at support tickets. Any weight that can be moved to serve that goal will be moved, including weights that were doing important work for something else. The optimiser has no memory of what those weights were for and no penalty for damaging it, because none of the old data is in the batch to complain.

That is the whole mechanism. Forgetting is not a bug in the training code. It is the predictable result of optimising for one thing while measuring only that thing.

The symptoms you actually see

Catastrophic forgetting rarely announces itself. The target metric goes up, which is what everyone is watching. What you see instead:

• The model became noticeably more repetitive or more terse across the board.

• It stopped following formatting instructions it used to handle, like returning clean JSON or respecting a word limit.

• It answers in the register of your training data even when the question is unrelated. Fine-tune on legal memos and everything starts sounding like a legal memo.

• Multilingual ability dropped, if your training set was monolingual.

• It got worse at saying it does not know, and more willing to answer confidently outside its range.

The tell that separates forgetting from other problems is breadth. A prompt issue affects one kind of request. Forgetting affects a wide, unrelated spread of capabilities at once, all in the direction of your training data.

Three things it gets confused with

A prompting change. If you also changed the system prompt when you deployed the fine-tune, you have two variables. Revert the prompt, keep the model, and measure again before blaming the weights.

[Model collapse](/blog/what-is-model-collapse). That is a different failure, where a model trained on its own generated output degrades across generations. Related family, different cause. Forgetting is about one training run displacing old knowledge, not about a feedback loop over many.

Simply picking the wrong base model. If the capability was weak before you fine-tuned, you did not lose it. This is why the pre-training measurement below matters so much.

How to avoid it

The mitigations are well established and mostly boring.

Measure before you train. Build a small evaluation set covering the capabilities you care about keeping, not just the one you are improving, and run it against the base model first. Without that baseline you cannot tell loss from something that was never there. Ten prompts per capability is enough to catch a collapse.

Use adapters instead of full fine-tuning. Techniques like LoRA train a small set of additional parameters and leave the original weights frozen. Because the base weights never move, the base knowledge cannot be overwritten. This is the single most effective structural defence and it is cheaper than full fine-tuning as well.

Turn the learning rate down. Large updates cause large displacement. A learning rate an order of magnitude below your first instinct, run for more steps, usually reaches the same target performance with far less collateral damage.

Mix in general data. Blending a slice of general instruction-following examples into your training set, often called rehearsal or replay, keeps some pressure on the old behaviours during training. Even ten to twenty percent general data helps.

Stop earlier than feels right. Most forgetting happens after the target metric has already plateaued. If your task accuracy stopped improving at epoch two and you trained for six, epochs three to six bought nothing and cost you general capability.

There is also a body of research on penalising changes to weights that mattered for previous tasks. Elastic weight consolidation is the best-known example. It is elegant, and largely academic for anyone using a hosted model, since you rarely control the training loop at that level.

When forgetting is the point

Worth saying: sometimes you want it. If a model has a habit you are deliberately training out, that is forgetting working as intended. The problem is never forgetting itself, it is unmeasured forgetting.

The practical question is almost always whether you need to fine-tune at all. For most product work, retrieval or a better prompt gets you there without touching weights and without this risk. Our comparison of RAG, fine-tuning and long context walks through which one your problem actually calls for, and what fine-tuning is covers the mechanics if you decide it is the right tool.

FAQ

Does catastrophic forgetting affect large models less?

Somewhat. Bigger models have more capacity and more redundancy, so there is more room to absorb a new task without displacing an old one. It reduces the effect rather than removing it. Large models fine-tuned aggressively on narrow data still lose breadth.

Can I recover a model that has already forgotten?

If you kept the base weights and trained an adapter, yes, discard the adapter. If you did a full fine-tune and overwrote your only copy, no. Retrain from the base checkpoint with the mitigations above. Keep base checkpoints.

Does this happen with prompting or retrieval?

No. Both leave the weights untouched. That is a real argument for solving problems with context rather than training whenever the option exists.

How much data does it take to cause forgetting?

Less than most people expect. A few hundred examples at a high learning rate can measurably shift general behaviour. Volume matters less than learning rate and number of epochs.

Is this why a model gets worse after a vendor updates it?

Not usually, and you cannot diagnose it from outside. When a hosted model changes behaviour after an update, you are seeing the result of an entire training and evaluation pipeline, not one fine-tune. Pin model versions if that behaviour matters to you, as model version numbers explains.