How to Read an AI Model's System Card
A system card isn't marketing copy. Here's how to skip the framing and read the four sections that actually tell you whether a new model is safe to build on, using Claude Opus 5's real system card as an example.
A system card is the document an AI lab publishes alongside a new model release that explains what the model can do, how it was tested for safety, and where it should not be used. Reading one well means skipping the marketing framing at the top and going straight to four sections: intended use versus out-of-scope use, the safety evaluation results, the benchmark methodology notes, and the known limitations section. Those four sections tell you whether a model is safe to build on, not the headline benchmark score. This guide walks through what to look for, using Anthropic's Claude Opus 5 system card as a real example of what a thorough one includes.
What Is a Model System Card, Exactly?
A model system card, sometimes shortened to model card, is a technical report a lab releases alongside a new model. It isn't user documentation and it isn't a press release, though the opening pages of some cards read that way. Its job is to disclose training decisions, evaluation results, and deployment restrictions to researchers, regulators, and the businesses that will put the model into production.
Length and rigor vary a lot by lab. Some run past a hundred pages with red-team transcripts and quantified capability thresholds. Others are a handful of pages with benchmark tables and a short paragraph on limitations. Neither format is legally required everywhere yet, though disclosure rules are tightening, as our breakdown of EU AI Act transparency rules covers. Until enforcement catches up, quality is whatever a lab decides to publish, which is why knowing what to look for matters.
The Sections That Matter Most, and Why
Not every section in a system card carries equal weight for someone deciding whether to build on a model. These are the ones worth reading in full, even when you skim the rest.
Intended use vs out-of-scope use: What the lab actually tested the model for. Strong coding benchmarks with no medical or legal evaluation means those domains aren't vetted, no matter how well a demo performs.
Safety evaluation results: Red-teaming outcomes, refusal rates on harmful requests, and any risk-tier classification, such as Anthropic's ASL levels or OpenAI's preparedness framework tiers. These are measurements, not assurances.
Benchmark methodology notes: A score is close to meaningless without the sample size, whether the lab checked for training-data contamination, and whether the result is a single run or an average. Cards that bury this in a footnote are telling you something too.
Known limitations section: The part labs have the least incentive to write honestly, so when it's specific instead of generic, it's worth extra attention. Specific limitations mean the evaluation team found real problems.
Deployment surface differences: Safety numbers often differ between the raw API and the consumer product, because the product layer adds system prompts and filters the API doesn't include by default.
Comparison to the prior model: A good card reports regressions, not just gains. If a new model hallucinates more in some category than its predecessor, that belongs in the card.
A Worked Example: Claude Opus 5's System Card
Anthropic published the Claude Opus 5 system card on July 24, 2026. It's a useful worked example because it does most of the above instead of gesturing at it.
The card places Opus 5 at ASL-3, the same protection level as its predecessor, Opus 4.8, and explains why instead of stating the tier alone. The model is classified CB-1, meaning it can meaningfully assist with non-novel biological or chemical weapons information, but it did not cross the CB-2 threshold for novel weapons development. That's a specific, checkable claim, not a vague assurance.
On capability testing, the card reports five cyber evaluations, including OSS-Fuzz and a Firefox exploitation benchmark, plus external testing from the UK AI Security Institute. It also covers prompt injection resistance across coding and browser tools, and alignment testing, where Anthropic reports Opus 5 scored as its most-aligned model to date.
It's also specific about deployment-surface gaps: single-turn harmlessness scores move from roughly 96 percent on the bare API to about 99 percent on claude.ai, a difference the card attributes to the system prompt the consumer product adds by default. That's exactly the kind of number a rushed card would leave out.
Its known-limitations section names specific weaknesses instead of hedging: verbosity that over-shares operational detail, susceptibility to requests reframed as fiction, and a small increase in hallucination rate versus Opus 4.8. Reporting a regression against your own prior model is the detail that separates a real system card from a compliance formality. The full index of Anthropic's published cards, including this one, lives on its system card archive.
How to Evaluate a New AI Model Release Using Its System Card
Treat the system card as a first source when a new model drops, ahead of leaderboard screenshots and launch-day threads:
Read intended use first. If your use case isn't listed, treat the model as unvetted for it, not automatically safe.
Check the evaluation date and model version against what you're calling in production. Cards are versioned per release, and providers sometimes update a model quietly after launch.
Compare the benchmark methodology to how you'll actually use the model. A benchmark run in a single attempt with no tool access tells you little about an agent running multi-step tasks, which is worth understanding at the level of what an AI benchmark actually measures.
Read the limitations section for anything that maps to your workload, not for how many pages it spans.
Weigh the safety tier against comparable models if you're choosing between providers, including whether an open-weight or closed model changes what you can verify yourself.
If you're evaluating a vendor's model choice rather than a lab's own release, this same reading habit is one piece of a broader check, covered in our guide to vetting an AI vendor. And if the question is whether a release is worth the migration effort at all, start with the benchmark and limitations sections, alongside our post on deciding when to upgrade to a newer model.
Common Mistakes When Reading a System Card
Three mistakes show up constantly. First, trusting the headline benchmark chart without reading the methodology notes underneath it. Benchmarks aren't standardized across labs, and a small percentage difference can come from different scoring rules rather than a better model. Second, ignoring which deployment surface the safety numbers came from. A model safe behind a product's system prompt isn't automatically safe when called raw through an API. Third, reading only the executive summary and skipping the appendix, usually where labs put the red-team transcripts and the numbers that don't flatter the release.
None of this requires reading every card cover to cover. It requires reading the same four sections whenever a model you rely on gets updated, a habit worth building into however you already track AI releases.
FAQ
What is a model system card?
A model system card is a technical report an AI lab publishes with a new model release. It documents intended use, safety evaluation results, benchmark methodology, and known limitations, and it's distinct from marketing pages or a product changelog.
Do all AI labs publish system cards?
No. Practice varies by lab and release size. Some labs publish detailed cards with every major model, others publish shorter model cards, and smaller updates sometimes ship without one at all. Disclosure requirements are expanding through regulation, but until enforcement is universal, format and depth remain the lab's choice.
What's the difference between a system card and a benchmark leaderboard?
A benchmark leaderboard ranks models on a narrow set of standardized tests. A system card is the lab's own disclosure of what it tested, how it tested it, and where the model falls short, including things a leaderboard won't show, like safety evaluations and known limitations.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
