Why AI security has to happen at the response level

Most early approaches to AI security focused on the prompt: block bad words, detect known jailbreak patterns, rate-limit suspicious users. That catches a real class of attacks, but it misses a bigger one — the failures that only appear in what the model actually says back.

A retrieval-augmented assistant can be given a completely benign prompt and still produce a harmful answer, because the document it retrieved was quietly poisoned. A support agent can be asked an ordinary question and still leak a customer's data, because nothing about the input looked wrong.

That's the case for scanning at the response level: treat every output as untrusted until it's been checked, the same way a firewall treats every packet. It's a different engineering problem than input filtering — you need to understand not just whether a request was malicious, but whether the resulting behavior is safe.