Dashboard

XSS in AI-Generated Code: Three Patterns to Fix

XSS in AI-generated code usually hides in three places: rendered markdown, chat output and user links. See each risky pattern and the safe fix to ship.

Carlo Zuercher
Carlo Zuercher
Staff Engineer, Platform
11 October 20261 min read

XSS in AI-generated code, short for cross-site scripting, almost always shows up in one of three places: markdown that gets rendered as raw HTML, text from an AI model that gets injected into the page, and links built from a web address a user typed in. In each, the code works perfectly in testing, which is why it ships. In each, a stranger can make the page run their script in another user's browser, steal a session or act as them.

React and similar frameworks escape text by default, so plain code is usually safe. The trouble starts when an agent needs "rich" output and reaches for the escape hatch.

What XSS in AI-generated code actually is

Cross-site scripting happens when an app puts untrusted data into a page in a way the browser treats as code. The data might be a comment, a profile name, a chat message or a URL. The OWASP Cross Site Scripting Prevention Cheat Sheet puts the main defense this way: when you need to display data exactly as a user typed it, output encoding is recommended. To place a variable in an HTML context, it says to use HTML entity encoding for that variable. It also advises refactoring away from unsafe sinks like innerHTML in favor of textContent or value.

The pattern to remember: a "sink" is any place data becomes markup. Agents add sinks whenever a feature needs formatting.

Pattern 1: markdown rendered as HTML

A notes app, a comment box or a help center needs bold text and links, so the agent adds a markdown library and writes the output into the page.

tsx
// risky
<div dangerouslySetInnerHTML={{ __html: marked.parse(comment.text) }} />

Markdown allows raw HTML, so a comment containing <img src=x onerror="..."> becomes live code. OWASP recommends DOMPurify for HTML sanitization, and the fix is to sanitize the HTML before it reaches the page:

tsx
import DOMPurify from "dompurify";

const html = DOMPurify.sanitize(marked.parse(comment.text) as string);
<div dangerouslySetInnerHTML={{ __html: html }} />

The OWASP guidance warns that framework escape hatches such as React's dangerouslySetInnerHTML can bypass built-in protections, so the name is a warning to take seriously, not a style quirk. Grep your codebase for it. Every hit should have a sanitizer right next to it.

Pattern 2: AI model output injected as HTML

This one is new to apps built around chat. The agent builds a chat feature, the model answers in markdown, and the answer is rendered with the same dangerouslySetInnerHTML line as above, without a second thought because "it comes from our own AI".

It does not come from your AI alone. The model's reply is shaped by whatever it read: a web page it summarized, a document a user uploaded, a message in a shared thread. A hidden instruction in that content can make the reply contain an image tag or script that a careless renderer will execute. This is prompt injection turning into XSS, and it is the reason model output must be treated as untrusted exactly like user input.

The fix is the same as before: render model output through a markdown renderer configured not to allow raw HTML, or sanitize its output, or both. Keep the model's text out of any sink that interprets markup.

A profile page lets people add their website. The agent writes:

tsx
<a href={user.website}>Website</a>

React escapes the attribute, but a javascript: address can still run script when clicked in many setups, and leaning on the framework to catch it is not a plan. Validate the protocol before using it:

ts
function safeHref(input: string): string {
  try {
    const u = new URL(input);
    return ["http:", "https:", "mailto:"].includes(u.protocol) ? u.href : "#";
  } catch {
    return "#";
  }
}

<a href={safeHref(user.website)} rel="noopener noreferrer">Website</a>

Use an allow list of protocols, not a block list of bad ones. Block lists forget something.

A ten-minute review for your own app

  1. Search the code for dangerouslySetInnerHTML, innerHTML and v-html (or the equivalent in your framework). List every hit.

  2. For each, trace where the data comes from. If any path starts with a user, a database field a user can write to, or model output, it needs a sanitizer or a different approach.

  3. Search for href={ and src={ fed by variables, and check each source.

  4. Paste a test string like <img src=x onerror=alert(1)> into every text field and see whether anything fires. If it does, you found one.

  5. Ask the agent to fix findings, then repeat the test yourself. Do not accept its report on its own.

Content security policy headers add a second layer by limiting what scripts a page may run, but they are a safety net for mistakes, not a replacement for encoding and sanitizing.

Where this fits with other agent mistakes

XSS sits next to other classics agents repeat. SQL injection in AI-generated code follows the same shape, untrusted data reaching an interpreter, and how to catch an AI coding agent introducing a vulnerability gives a general review routine. If your app has a rich text field, adding a rich text editor to an AI-built app is where this problem is most likely to arrive. For the wider risk picture, start with the AI risks overview. A Content Security Policy is the browser-side backstop for this class of bug, explained in what a content security policy is.

Frequently asked questions

Does React protect me from XSS automatically?

Mostly, for normal text. It escapes values placed in JSX. It does not protect you when you use dangerouslySetInnerHTML, when you build links from untrusted URLs, or when you write to the page directly with innerHTML.

Can AI find XSS in my code?

It can find obvious cases quickly, such as unsanitized dangerouslySetInnerHTML. It misses data-flow cases that span several files, so combine its review with your own search and a manual test.

Is markdown safe to render?

Markdown itself is a formatting language, but most renderers pass raw HTML through by default. Sanitize the output or configure the renderer to disallow HTML.

Why does model output count as untrusted?

Because the model's reply is influenced by content you do not control, including web pages and uploaded files. Anything an outsider can influence should be treated as untrusted.

How did this land?

About the author

Carlo Zuercher
Carlo Zuercher

Staff Engineer, Platform

Carlo works on the platform that turns prompts into running apps. He writes the engineering deep dives and the changelog notes worth reading.

Share

Get the next post in your inbox

One email a month. Product updates, engineering posts, and the best of Built with Swarmz.

I agree to receive emails about AI building tips and Swarmz product news. Unsubscribe any time.