---
title: "AI Security: Prompt Injection, Data Leakage and Red Teaming"
description: "Threat-model, attack and harden LLM apps and agents using OWASP, MITRE ATLAS and NIST guidance, garak, PyRIT and promptfoo"
url: https://optimizeall.com/learn/ai-security-and-red-teaming
updated: 2026-10-05
---

AI · Advanced · 438 minutes · free · updated Sep 2026

# AI Security: Prompt Injection, Data Leakage and Red Teaming

Threat-model, attack and harden LLM apps and agents using OWASP, MITRE ATLAS and NIST guidance, garak, PyRIT and promptfoo

- **Lessons:** 17 in 7 modules
- **Video lectures:** 17 lectures, 143 minutes
- **Updated:** Sep 2026

## Tools you'll use

- OWASP Top 10 for LLM Applications
- OWASP Agentic Top 10
- MITRE ATLAS
- NIST AI RMF
- garak
- PyRIT
- promptfoo
- Inspect
- Microsoft Presidio
- Model Context Protocol
- OpenTelemetry
- Docker

[Start the course](https://optimizeall.com/learn/ai-security-and-red-teaming/llm-threat-landscape)

## About this course

LLM applications and agents read untrusted text, hold sensitive data and increasingly take real actions, which makes them a new and fast-moving attack surface. This advanced course teaches you to think like an attacker and build like a defender. You will threat-model LLM apps, map risks to the OWASP Top 10 for LLM Applications (2025 and the August 2026 update) and the OWASP Top 10 for Agentic Applications, and use MITRE ATLAS and NIST guidance. You will dissect direct and indirect prompt injection, jailbreaks, data exfiltration through tools, links and images, excessive agency, supply-chain risks in models, datasets and MCP servers, RAG poisoning, improper output handling and secret or PII leakage. Then you will design defense in depth, run structured red-team exercises with garak, PyRIT and promptfoo, and prepare incident response. The capstone: red-team and harden a tool-using agent.

## What you will learn

- Threat-model an LLM application or agent, identifying trust boundaries, assets and abuse cases
- Map risks to the OWASP LLM and Agentic Top 10 lists, MITRE ATLAS and NIST guidance
- Explain and demonstrate direct and indirect prompt injection, jailbreaks and exfiltration channels
- Secure tools, agents, RAG pipelines and supply chains including models, datasets and MCP servers
- Design defense-in-depth controls: privilege separation, allowlists, approvals, provenance and monitoring
- Plan and run red-team exercises using garak, PyRIT and promptfoo, and report findings
- Prepare and run incident response for AI-specific security events

## Before you start

- [Building Production AI Agents](https://optimizeall.com/learn/ai-agents-engineering)
- [Integrating AI Platforms: Claude, OpenAI, Gemini and Open Models via API](https://optimizeall.com/learn/ai-platform-apis-integration)

## Course content

### Threat modeling LLM applications and agents

Understand why LLMs break the code/data boundary, identify assets, trust boundaries and the lethal trifecta, and run a practical threat model with abuse cases.

- [The LLM threat landscape and the lethal trifecta](https://optimizeall.com/learn/ai-security-and-red-teaming/llm-threat-landscape): 13 min
- [A practical threat-modeling process for LLM systems](https://optimizeall.com/learn/ai-security-and-red-teaming/threat-modeling-process): 14 min

### Frameworks: OWASP, MITRE ATLAS and NIST

Use the OWASP Top 10 for LLM Applications (2025 and 2026 editions) and Agentic Applications, MITRE ATLAS and NIST AI guidance as a shared language for coverage, attacks and risk management.

- [OWASP Top 10 for LLM and Agentic Applications](https://optimizeall.com/learn/ai-security-and-red-teaming/owasp-top-10-llm-and-agentic): 14 min
- [MITRE ATLAS and NIST AI guidance](https://optimizeall.com/learn/ai-security-and-red-teaming/mitre-atlas-and-nist): 13 min

### Prompt injection, jailbreaks and data exfiltration

Recognize direct injection and jailbreak families, defend against indirect injection by separating reading from acting, and close exfiltration channels such as images, links and tools.

- [Direct prompt injection and jailbreaks](https://optimizeall.com/learn/ai-security-and-red-teaming/direct-injection-and-jailbreaks): 14 min
- [Indirect prompt injection and separation patterns](https://optimizeall.com/learn/ai-security-and-red-teaming/indirect-prompt-injection): 15 min
- [Data exfiltration: images, links, tools and EchoLeak](https://optimizeall.com/learn/ai-security-and-red-teaming/data-exfiltration-channels): 14 min

### Agents, tools, supply chain and RAG poisoning

Contain excessive agency with narrow, authorized tools and approvals; secure models, datasets, packages and MCP servers; and defend RAG indexes, vector stores and memory.

- [Excessive agency and secure tool design](https://optimizeall.com/learn/ai-security-and-red-teaming/excessive-agency-and-tool-security): 15 min
- [AI supply chain: models, datasets, packages and MCP servers](https://optimizeall.com/learn/ai-security-and-red-teaming/ai-supply-chain-and-mcp): 14 min
- [RAG poisoning, vector store security and memory](https://optimizeall.com/learn/ai-security-and-red-teaming/rag-poisoning-and-vector-security): 14 min

### Output handling, data protection and cost abuse

Treat model output as untrusted input to every downstream sink, and prevent sensitive information disclosure, hidden context exposure and unbounded consumption.

- [Improper output handling: XSS, SQL, code execution and SSRF](https://optimizeall.com/learn/ai-security-and-red-teaming/improper-output-handling): 14 min
- [Secrets, PII, hidden context and unbounded consumption](https://optimizeall.com/learn/ai-security-and-red-teaming/secrets-pii-hidden-context-and-cost): 14 min

### Defense in depth and red teaming

Design layered defenses with deterministic cores and kill switches, plan and run structured AI red-team engagements, and automate testing with garak, PyRIT, promptfoo and Inspect.

- [Defense-in-depth architecture for LLM apps and agents](https://optimizeall.com/learn/ai-security-and-red-teaming/defense-in-depth-architecture): 15 min
- [Planning and running AI red-team engagements](https://optimizeall.com/learn/ai-security-and-red-teaming/red-teaming-methods): 14 min
- [Red-team tooling: garak, PyRIT, promptfoo and Inspect](https://optimizeall.com/learn/ai-security-and-red-teaming/red-team-tools): 14 min

### Incident response and capstone

Prepare for and respond to AI security incidents, then red-team and harden a deliberately insecure tool-using agent end to end.

- [Incident response for AI systems](https://optimizeall.com/learn/ai-security-and-red-teaming/ai-incident-response): 14 min
- [Capstone: red-team and harden a tool-using agent](https://optimizeall.com/learn/ai-security-and-red-teaming/capstone-red-team-and-harden-an-agent): 25 min

## Certificate: Certified AI Security Practitioner

The holder can secure LLM applications and agents: threat modeling, mapping risks to OWASP, MITRE ATLAS and NIST guidance, defending against direct and indirect prompt injection, exfiltration, excessive agency, supply-chain and RAG poisoning attacks, designing defense in depth, running structured red-team exercises with open-source tools, and responding to AI security incidents.

- **Final assessment:** 30 questions, 45 minutes
- **Passing score:** 80%