Insecure Output Handling (LLM → XSS/Code Execution)

An LLM can be steered — via prompt injection or simply by asking it directly — into producing HTML, JavaScript, SQL, or shell commands in its response. If the application renders or executes that output without the same sanitization it would apply to raw user input, the model becomes a confused deputy for classic injection attacks.

The model's output is, from a security perspective, just more untrusted text — it deserves exactly the same output-encoding and sanitization as anything else you didn't write yourself

Rendering LLM output as raw HTML (dangerouslySetInnerHTML, innerHTML) reopens XSS even in an app that's otherwise careful about escaping user input

Feeding LLM output into an eval(), a shell command, or a SQL query re-opens the exact same injection classes those APIs are always vulnerable to — the model is just a new source of the malicious string

This is distinct from prompt injection: the injection point here is the OUTPUT reaching a dangerous sink, regardless of how the model was convinced to produce it

Want this flow explained in detail as it plays?

POST/api/chat

Headers

Host:app.flowground.dev

Content-Type:application/json

Body

Quick-Insert Payloads

Defense in depth

Render LLM output as plain text by default; only allow a strict, explicitly-allowlisted subset of markdown/HTML through a real sanitizer (e.g. DOMPurify) — never dangerouslySetInnerHTML on raw model output
Never pass LLM output directly into eval(), a shell command, or a raw SQL string — if the model's output needs to trigger an action, parse it into a constrained, typed format (e.g. a tool call with a validated schema) instead of free text
Apply the same output-encoding rules (HTML-entity encoding, parameterized queries) to model output that you'd apply to any other untrusted string reaching that same sink

Vulnerable vs. Fixed

function ChatMessage({ text }) {
  return <div dangerouslySetInnerHTML={{ __html: text }} />; // model output rendered as raw HTML
}