Defense-in-depth architecture for LLM apps and agents

0:00 Defense in depth for LLM systems

If you remember one thing from this course, make it this: no single control stops prompt injection. Classifiers miss things. Models get manipulated. Filters get bypassed, sometimes all in the same attack, as EchoLeak showed. So professional teams build defense in depth, independent layers where an attacker must beat every one, and where the architecture limits the damage when they do. In this lecture you will get a layered reference architecture, the key patterns, and a way to switch things off safely when something goes wrong.

0:37 Analogy: airport security

Think of it like airport security. There is not one check but many: identity at booking, bag scanning, a metal detector, a boarding pass check, locked cockpit doors. Each has gaps. Together they make it very hard to cause harm. And the most important one, the locked cockpit door, does not care how convincing the passenger is. In LLM systems, your locked cockpit doors are the deterministic controls: scoped permissions, tool authorization, egress blocks. They hold no matter what the model decides.

1:13 Layers 0–4

Here is the reference architecture in the lesson, as nine layers. Layer zero, scope and design: minimal capabilities and the trifecta check. Layer one, identity and access: authenticated users, user-scoped credentials, tenant isolation. Layer two, input handling: size limits, invisible character scanning, injection classifiers and provenance tags. Layer three, context assembly: minimization, spotlighting and trust tiers. Layer four, the model itself, with provider safety features and structured outputs.

1:43 Layers 5–8

Layer five, action control: narrow tools, authorization inside tools, hard limits, human approval and sandboxes. Layer six, output handling: validation, encoding, link and image allowlists, content security policy and redaction. Layer seven, egress: network and recipient allowlists, and no generic fetch tools on agents with private data. Layer eight, detection and response: tracing, canaries, anomaly alerts, kill switches and runbooks. Notice which layers carry the most weight: zero, one, five and seven, because they hold regardless of what the model does.

2:18 Patterns

Now the patterns. Privilege separation: the component that reads untrusted content has no dangerous tools, and the component with tools never sees raw untrusted content. A twenty twenty-five research paper on design patterns for securing agents catalogs more: an action selector, where the model picks from fixed actions and tool output never comes back to it; plan-then-execute; map-reduce, processing untrusted items in isolation; and context minimization. CaMeL, from Google DeepMind researchers, tracks where every value came from and enforces policies on where it may flow.

2:55 Principles

Three more principles. Prefer allowlists over denylists, for tools, domains, recipients, file paths and commands, because denylists always miss something. Do human approval properly: show the actual parameters, rate-limit approvals and require re-authentication for the riskiest actions. And track provenance for every piece of context, showing sources in answers. For generated media, standards like C2PA Content Credentials can signal origin, but they complement access controls rather than replacing them.

3:25 Kill switches

Now the off switch, which you should design before you need it. Feature flags that disable a tool, a route or an entire agent instantly. Circuit breakers that disable actions automatically when anomaly thresholds are crossed, like a spike in outbound emails or refunds. A read-only mode for investigations. And credential revocation runbooks for every agent identity. The lesson includes a short circuit breaker around an email tool that pages on-call and pauses sending when volume looks abnormal.

3:59 Case: Dubai inbox agent

A realistic example. A consultancy in Dubai deployed an agent that triages a shared inbox, drafts replies and schedules meetings. They removed autonomous sending entirely; drafts only. OAuth tokens were scoped to the shared mailbox. Emails were spotlighted as untrusted with sender reputation as a provenance tag. The draft tool could not add recipients beyond the original thread. Outputs were rendered with link allowlists. The runtime had no general internet access. And traces carried canary tokens, with alerts on unusual draft volume and a one-click disable. Red-team tests that once produced exfiltration drafts now produced harmless drafts that humans simply rejected.

4:43 Common mistakes

Common mistakes. Stacking only probabilistic layers, a classifier plus a prompt plus a judge, with no deterministic boundary underneath. Approval fatigue, where human review becomes a rubber stamp. A kill switch that nobody has ever tested. And applying one pattern everywhere, instead of the strictest pattern to the highest-risk flows. Here is a question to test your own system: if the model were fully compromised right now, which deterministic control would stop data leaving and money moving?

5:16 Walkthrough: the phishing review

Let's walk a simple example through the layers. A user asks a shopping assistant to summarize reviews, and one review contains an injection telling the model to show a phishing link with a fake discount. Layer two, the input scanner, may or may not flag it. Layer three spotlights reviews as untrusted, which lowers the odds of success. Suppose the model is fooled anyway, at layer four. Layer six, the output link allowlist, strips the link because the domain is not yours. And layer eight logs the attempt and alerts, so the review is removed. The attack needed every layer to fail. It only took one to stop it.

6:03 Deeper: an action selector

One level deeper on the action-selector pattern. For an order assistant, the model chooses one of five fixed actions: show status, show delivery estimate, start a return, change address, or hand off to a human. The chosen action's parameters come from the verified session. Tool output is rendered by templates straight to the user and never returns to the model, so injected text in tool data cannot steer a next step.

6:34 Watch me do it: circuit-breaker drill

Watch me do it: testing the circuit breaker from the lesson in a staging drill. The guarded send email function does three things before sending. First, it checks a feature flag for the email tool; if the flag is off, it refuses. Second, it counts emails sent by the agent in the last ten minutes and compares with the configured limit, say thirty. Third, if the count is over, it turns the flag off itself, pages on-call with the reason, and refuses. Now the drill. I write a script that simulates an injected agent trying to send a report to many addresses in a loop. Emails one to thirty go out to our test inbox. Email thirty-one trips the breaker. The flag flips off, the pager fires with rate anomaly, and every later attempt fails with email sending paused. I time it: the breaker tripped within seconds of crossing the limit. Then the second half of the drill, the human part. On-call opens the runbook, checks the traces for the agent's recent inputs, finds the planted instruction, and confirms no real customer emails were involved. They switch the agent to read-only, and only after review turn the email flag back on. Total drill time: twenty-two minutes. We write down two improvements, including a pre-filled trace query, and schedule the next drill.

8:10 Recap

Recap. Assume every single control can fail. Build nine layers, and make sure the deterministic ones, scope, identity, action control and egress, hold even if the model is fully compromised. Use privilege separation and allowlists, do approvals properly, track provenance, and design your kill switches in advance. Try this now: for your highest-risk flow, write down the deterministic control that stops exfiltration and damaging actions, and schedule a drill to test your kill switch.