The LLM threat landscape and the lethal trifecta

0:00 The LLM threat landscape

In nineteen ninety-eight, the web learned about SQL injection: attackers slipping commands into data. The industry eventually fixed it by keeping code and data strictly apart. Large language models undo that fix by design. Instructions and data arrive in the same stream of tokens, and the model decides which is which. In this lecture you will learn why that matters, what attackers want from LLM systems, where the trust boundaries are, and a simple test called the lethal trifecta.

0:34 Analogy: the assistant and the post pile

Here is an analogy to hold onto. Imagine a personal assistant who reads all your post aloud and acts on it. A letter from your bank, a note from your manager, a flyer through the door, it all arrives in the same pile, in the same handwriting as far as the assistant is concerned. If a flyer says, the owner has asked you to pay this invoice today, a careful assistant might pause. A busy one might just pay. The problem is not that the assistant is stupid. It is that instructions and ordinary post arrive in the same pile. That pile is your context window.

1:20 The core problem

In a traditional app, the developer's code is trusted and user input is data. A parameterized query guarantees the input is never executed as code. An LLM has no such guarantee. Your system prompt, the user's message, a retrieved web page and a tool's output all land in one context window as tokens. No current model can reliably tell the developer's instructions from instructions an attacker hid in a document it was asked to summarize. Nearly every attack in this course follows from that one fact.

1:57 Assets

Start any assessment by listing assets, the things an attacker wants. Data: customer personal information, internal documents in your retrieval index, conversation history, system prompts and API keys. Actions: anything your tools can do, like sending email, issuing refunds, executing code or making purchases. Integrity: the correctness of outputs people rely on. Availability and cost: your token budget and rate limits. And reputation, because a screenshot of your assistant saying something outrageous travels fast.

2:29 Untrusted inputs

Next, trust boundaries. Draw your system: the user, your app, the model, tools, retrieval and downstream systems. Then mark every place untrusted content enters. The user, obviously. But also retrieved documents, web pages, emails, files, images and audio, tool outputs, the descriptions of MCP tools, messages from other agents, and long-term memory the system wrote earlier. Every one of those is a potential instruction channel.

2:57 The lethal trifecta

Here is the quickest test for agent designs, from security researcher Simon Willison. Exfiltration becomes possible when a system has three things together. Access to private data. Exposure to untrusted content. And the ability to communicate externally, by sending email, fetching URLs, rendering images or calling webhooks. If all three are present, assume an attacker who controls some content can steer the agent into sending private data out. For high-risk flows, remove a leg or put strong controls on it.

3:32 Attacker profiles

Who are the attackers? Curious or malicious users in the chat, trying jailbreaks or extracting your system prompt. Remote content authors who control a web page, an email, a product review or an MCP server, and use indirect injection. Insiders who can edit your knowledge base. Supply-chain attackers who publish a poisoned model, dataset or plugin. And fraudsters running scripts to exhaust your budget or abuse free tiers.

4:02 Case: inbox assistant

A worked example. A small agency in London connects an assistant to its shared inbox to summarize emails and draft replies. Private data: the inbox. Untrusted content: every incoming email. External communication: it can send email. All three legs are present. An attacker emails hidden text asking the assistant to forward the finance thread to an outside address. Whether the model resists today is not a guarantee. The robust fix is architectural: drafts only, or sending restricted to known recipients with human approval.

4:38 Pitfalls

Three pitfalls. Believing our system prompt tells it not to; instructions are not security controls. Forgetting indirect channels, since retrieved documents and tool outputs are attacker-controllable far more often than teams assume. And assuming the model provider handles it. Providers add useful defenses, but your architecture decides the blast radius when those defenses fail.

5:01 Example: the recipe assistant

A simple example of the trifecta test in action. A recipe website adds an assistant that suggests meals from its public recipes. Private data? None. Untrusted content? Yes, user comments. External communication? No tools, and images come only from its own domain. One leg present, low exfiltration risk. Now the same company adds loyalty accounts with saved addresses, and a tool that emails shopping lists anywhere. Private data, untrusted comments, and an email channel. Same assistant, all three legs, and suddenly a hidden instruction in a comment could send a customer's address to a stranger. The feature change, not the model, created the risk.

5:46 Mistake: review once, grow forever

And a common mistake worth naming. Teams often run a security review once, at launch, when the assistant is simple. Six months later it has three new tools, a connection to the CRM and long-term memory, and nobody re-ran the review. Make the trifecta check part of your definition of done for any new tool or data source. It takes five minutes and it catches the most dangerous class of design error before it ships.

6:19 Deeper: which leg to remove?

One level deeper on the London inbox example. Which leg is easiest to remove? Private data is the point of the assistant, and untrusted email is its input. But autonomous sending is optional. Switch to drafts that a human approves, and the attacker's instruction can at most produce a draft that nobody sends.

6:42 Watch me do it: a system sketch

Watch me do it: the one-page system sketch for a real feature, a sales assistant for a digital agency in Lahore that answers client questions using the CRM and the shared drive. System name and purpose: helps account managers answer client status questions. Users: twelve account managers. Model: a hosted model through our provider's API. Now the most important line, untrusted inputs. I list them one by one: the account manager's own messages, client emails ingested into the CRM, documents on the shared drive that clients can upload to, notes typed by anyone in the CRM, and the output of the web search tool. Five sources, and only one of them is fully under our control. Sensitive data reachable: client contracts, pricing, contact details and invoices. Actions available: create CRM tasks, draft emails, and send emails to any address. External channels: sending email, the web search tool, which can fetch any URL, and the chat UI, which renders Markdown images. Trifecta present? Yes, in two flows. A client-uploaded document plus CRM data plus email sending is one. A web page plus pricing data plus image rendering is the other. Existing controls: login only. I've just found two high-risk flows in fifteen minutes, before writing any detailed threat model, and I know exactly where to start.

8:15 Recap

Recap. LLMs mix instructions and data, so any content can become an instruction. List assets, mark every untrusted input, and apply the lethal trifecta test. Your next step: complete the one-page system sketch in the lesson for an LLM feature you own, and mark every flow where all three legs of the trifecta are present.