What to Put in an AI Product Changelog

An AI product changelog needs one category that ordinary software changelogs do not have: behaviour changes that ship without a code change. A model swap, a prompt revision, a retrieval tweak, a temperature adjustment. None of them appear in a diff a customer could inspect, all of them can change the output they get tomorrow from the input that worked today.

That is the entire difference, and it is why copying a standard "Added / Changed / Fixed" template leaves the most important thing unlogged.

The five sections

Added. New features, new endpoints, new supported formats. Same as any product.

Changed. Interface and workflow changes. Same as any product.

Fixed. Bugs. Same as any product.

Model and behaviour. The one that is specific to you. Anything that alters what the system produces without altering what the customer does. Model version changes, system prompt revisions, changes to retrieval or ranking, changes to default parameters, changes to which requests route to which model.

Limits and policy. Rate limits, context caps, pricing-relevant thresholds, data retention changes, new content restrictions. These are commitments, and customers plan around them.

Five sections, and the fourth is the one that earns you trust.

What a behaviour entry should contain

Bad: "Improved response quality."

That tells a customer nothing and, worse, tells them nothing changed that they need to check. If they have prompts or downstream parsing tuned to your old behaviour, you have just broken them silently and told them it was an improvement.

Better, on four dimensions:

• What changed, in mechanism terms. "Switched the summarisation endpoint from the previous model to a newer one."

• What is likely to differ. "Summaries run roughly 15% shorter and use more structure. Bullet output is more common where the source has lists."

• Whether anything is now unsupported. Explicit yes or no. Most of the time it is no, and saying so is valuable.

• What to do if it broke something. A version pin, a rollback window, a support path.

A real entry, in about four lines:

2026-08-22 · Model and behaviour The document summarisation endpoint now runs on our newer model. Summaries are typically 10 to 20% shorter and use headings more often. No API or schema changes. If you parse summary output positionally, re-check it. The previous model stays available on ?model=legacy until 2026-10-01.

That is enough for someone with an integration to decide in ten seconds whether it affects them. Which is the only job the entry has.

The dates you owe people

Two dates matter more than the rest, and both need to be in the changelog rather than only in an email.

Deprecation dates. When you announce that a model or endpoint goes away, put the removal date in the entry and repeat it in every subsequent entry that touches the same surface. Customers find changelogs by searching, months after the email went to an address nobody reads. What to do when an AI model gets deprecated is the customer's side of this, and it is a good check on whether your notice is actually actionable.

Default-change dates. If a new model becomes the default on a date, say the date. "Now the default" without a date leaves everyone unsure whether their existing traffic already moved.

Where changelog and release notes part company

They are not the same document and conflating them is a common mistake.

The changelog is a record. The release note is a communication. If you only maintain one, maintain the changelog, because you can always write a note from it and you can never reconstruct it afterwards. When you do write the note, prompting AI to draft release notes from changelog entries is a genuinely good use of it, in that direction and not the reverse.

Three failure modes

Logging the model version and nothing else. "Updated to model v4.2" is a fact with no consequence attached. The customer cannot tell whether to care. Always pair the version with the observable difference.

Batching a quarter of behaviour changes into one entry. Nobody can work out which change broke their workflow in March. Behaviour entries should be dated to the day they shipped, even when they are small.

Quietly editing old entries. If an entry was wrong, add a correction with its own date. A changelog people suspect of being rewritten is worth less than no changelog, because it removes the one thing it was for.

The part that is really about trust

There is a temptation to leave silent behaviour changes out, on the grounds that most customers will not notice and the ones who do will be annoyed. That trade looks good for exactly as long as it takes for one customer to notice on their own.

At that point they do not have a quality issue, they have a disclosure issue, and those are much harder to recover from. Publishing the change first turns "the output got worse" into "the output changed and they told us how", which is a support ticket rather than a trust problem. The same logic behind telling users you changed the AI model applies to every silent change, not only the model swap.

For a small team the whole practice costs about ten minutes per release. That is a cheap way to be the vendor whose behaviour changes are predictable, which is worth more in this category than in most. Where it fits in the broader picture of running an AI product as a business, it belongs under retention rather than marketing.

FAQ

What should go in an AI product changelog that a normal changelog does not have?

A dedicated section for behaviour changes that ship without an interface change: model swaps, prompt revisions, retrieval or ranking changes, default parameter changes. These alter output without altering the API, so nothing else surfaces them.

How detailed should a model change entry be?

Enough for an integrator to decide in seconds whether it affects them. Name what changed, describe the observable difference, state clearly whether anything is unsupported, and give a path if it broke something.

Should I log prompt changes?

Yes, when they change output in a way a customer could notice. You do not need to publish the prompt itself. Publishing that the behaviour moved, and how, is the part that matters.

Do I need both a changelog and release notes?

Not necessarily, but they serve different readers. If you keep one, keep the changelog: it is a permanent factual record, and release notes can be written from it later.

How do I handle a change that made output worse for some users?

Log it with the same detail as any other change, say plainly what regressed, and give a version pin or rollback path if you have one. Discovering it themselves is worse for them and much worse for you.