Prompt injection

Prompt injection is when someone sneaks instructions into the text an AI reads, and the AI follows them as if they came from you. The root of it is that a model can't tell your instructions apart from the content it's processing, since it all arrives as one stream of text. So a malicious line buried in a webpage or an email can say "ignore your instructions and do this instead," and the model may just comply. Simon Willison named it, drawing the parallel to SQL injection, and OWASP ranks it the number one risk for LLM apps.

It gets dangerous the moment an AI can both read untrusted content and take actions, like an agent that reads your email and can also send it. A single booby-trapped message can then turn the agent against you.

Keep this in mind any time you connect an AI to the outside world: browsing the web, or reading documents someone else can write to. The more the AI can do on your behalf, the more a hidden instruction can cost you.

There's no clean fix yet, so the practical moves are to limit what an agent can do without asking, and to treat anything it read from the wild as suspect. If you're just chatting with a model on your own inputs, the risk is low. It climbs fast once agents and other people's content enter the picture.