Building Production AI Agents · Human oversight, guardrails and security · lesson 10 of 18 · 15 min

Human-in-the-loop: approvals, escalation and review

Humans are a design component, not a fallback

The most reliable production agents are designed around people: who approves what, who is notified, who can take over. Human-in-the-loop (HITL) is not an admission of failure; it is how you ship value early while trust is built, and how you keep high-stakes actions accountable.

Classify actions by risk

Start with an action inventory: every tool, tagged by reversibility and blast radius.

| Tier | Examples | Default policy | |---|---|---| | 0 Read-only | Search CRM, read analytics | Auto-approve, log | | 1 Reversible, internal | Create draft, add note, tag lead | Auto-approve, log, sampled review | | 2 External or costly but reversible | Schedule post, pause campaign, create invoice draft | Approval required, or auto within limits (e.g., under £200) | | 3 Irreversible or high impact | Send bulk email, issue refund, publish, delete, change prices | Always human approval, two-person rule for large amounts |

Write the inventory down and review it with the business owner. Regulated teams (finance, health, employment decisions) may need approvals by law or policy; check with compliance.

The approval mechanics

A good approval flow has five parts:

Interrupt: the agent proposes an action; the runtime pauses the run before execution and persists state (lesson 9).
Present: show the approver the action, its arguments, the agent's reasoning summary, the evidence and the expected impact, in plain language.
Decide: approve, reject with reason, or edit arguments.
Resume: the run continues with the decision recorded as a tool result the agent can see ("Rejected by Aisha: discount too high; max 10%").
Audit: log who approved what, when, with which arguments.

Frameworks provide these primitives: LangGraph's interrupt() with checkpointers; the OpenAI Agents SDK's needs_approval on tools with interruptions you approve or reject on the run state; the Claude Agent SDK's permission modes, can_use_tool callback and hooks (for example a PreToolUse hook that denies or asks); Google ADK's tool confirmation flow. Managed platforms offer permission policies such as always-allow, always-ask, or automatic evaluation.

Escalation and handover

Beyond approvals, define when the agent must stop and hand over: low confidence, repeated tool errors, user frustration, legal or medical questions, VIP customers, or anything outside policy. The handover should include a summary so the human does not start from zero.

Review UX that people will actually use

Batch approvals where safe (approve 20 low-risk drafts at once) to prevent fatigue.
Show diffs for edits ("price 149 → 129 AED").
Use mobile-friendly notifications (Slack, Teams, email) with deep links.
Track approval latency; if approvals take hours, the agent's speed advantage disappears.
Watch for rubber-stamping: if approvers accept 100% in seconds, either the tier is too strict or reviewers need better information.

Worked example: a Jeddah restaurant group's promotions agent

The agent drafts weekly promotions for 15 branches using sales data.

Tier 1: draft promo copy in Arabic and English (auto).
Tier 2: schedule social posts (approval by branch manager, batch view).
Tier 3: change menu prices in the POS (regional manager approval; changes over 10% require finance too).
Escalation: any promotion touching alcohol-free "health" claims or Ramadan timing goes to the marketing lead.

After a month, 90% of tier-2 items were approved unchanged, so the team raised the auto-approve threshold for low-budget boosts while keeping tier 3 manual. (Figures illustrative.)

Hands-on: an approval gate around risky tools

import json, uuid

RISK = {"search_sales": 0, "draft_promo": 1, "schedule_post": 2, "update_price": 3}
PENDING: dict[str, dict] = {}   # use a database table in production

def gate(tool_name: str, args: dict, run_id: str) -> dict | None:
    tier = RISK.get(tool_name, 3)                   # unknown tools default to highest risk
    if tier <= 1:
        return None                                  # auto-approve, still log
    if tool_name == "schedule_post" and args.get("boost_budget_sar", 0) <= 200:
        return None                                  # auto-approve within a limit
    approval_id = str(uuid.uuid4())
    PENDING[approval_id] = {"run_id": run_id, "tool": tool_name, "args": args, "tier": tier}
    notify_approver(approval_id, tool_name, args)    # Slack/Teams/email with a deep link
    return {"status": "pending_approval", "approval_id": approval_id}

def notify_approver(approval_id, tool_name, args):
    print(f"[APPROVAL NEEDED] {approval_id}: {tool_name} {json.dumps(args, ensure_ascii=False)}")

def decide(approval_id: str, approved: bool, reviewer: str, reason: str = "") -> dict:
    item = PENDING.pop(approval_id)
    item.update({"approved": approved, "reviewer": reviewer, "reason": reason})
    # Persist to an audit table, then resume the run (lesson 9) with this tool_result:
    return {"type": "decision", **item}

In your loop, call gate() before executing each tool. If it returns a pending status, persist the run and stop; resume when decide() is called, passing the decision back as the tool result so the agent can adapt (for example, revise a rejected discount).

Pitfalls

Approvals that show raw JSON only; reviewers cannot judge impact.
Defaulting unknown tools to auto-approve.
No expiry: stale approvals executed days later against changed data. Re-validate before execution.
Letting the agent "retry" a rejected action with trivially different arguments; count rejections and escalate.

Measuring success

Approval rate unchanged, approval latency, override/edit rate, incidents per 1,000 actions, and reviewer time per item. Use these to move actions between tiers deliberately.

Video lecture: Human-in-the-loop: approvals, escalation and review

Lecture coming soon · 15 chapters · about 9 minutes. Read the full transcript below.

Human-in-the-loop
Why it matters
Risk tiers
Five parts of an approval
Framework support
Simple example: discount codes
Review UX
Example: Jeddah promotions agent
Hands-on: approval gate
Pitfalls and metrics
Approvals out of hours
Deeper: the Jeddah promotions (illustrative)
Watch me do it: gate() and decide()
Try this now
Recap

Lecture transcript

Human-in-the-loop

The best production agents aren't the ones that never involve people. They're the ones designed around people: who approves what, who gets notified, and who can take over. In this lesson you'll learn to classify actions by risk, build approval flows that reviewers actually use, and define clean escalation paths.

Why it matters

Why does this matter? Because trust is earned in increments. Nobody hands a new employee the company card on day one. They start with small purchases, get reviewed, and gain limits as they prove themselves. Agents should work the same way. Well designed approvals let you ship useful automation today, while people stay in control of the risky moments. Badly designed approvals either block everything, so nobody uses the agent, or become a rubber stamp that gives false confidence.

Risk tiers

Start with an action inventory. List every tool and tag it by reversibility and blast radius. Tier zero is read only, like searching the CRM: auto approve and log. Tier one is reversible and internal, like drafts and notes: auto approve with sampled review. Tier two is external or costly but reversible, like scheduling a post or pausing a campaign: require approval, or auto approve within limits. Tier three is irreversible or high impact, like bulk emails, refunds, publishing, deleting or changing prices: always a human, and for big amounts, two people.

Five parts of an approval

An approval flow has five parts. Interrupt: pause the run before the action executes and save the state. Present: show the action, its arguments, a short summary of why, the evidence and the expected impact, in plain language. Decide: approve, reject with a reason, or edit. Resume: continue the run and feed the decision back as a tool result, so the agent can adapt. For example, rejected by Aisha, discount too high, max ten percent. And audit: log who approved what and when.

Framework support

Most frameworks give you these building blocks. LangGraph has interrupt with checkpointers. The OpenAI Agents SDK lets you mark tools as needing approval and then approve or reject on the run state. The Claude Agent SDK has permission modes, a can use tool callback and hooks, like a pre tool use hook that denies or asks. Google's ADK has a tool confirmation flow. Learn the concept once, and the APIs are just syntax.

Simple example: discount codes

A simple example. An agent manages a small online shop's discount codes. Creating a draft code is tier one, automatic. Publishing a code up to ten percent off is tier two, auto approved within the limit. A fifty percent code is tier three and pauses for the owner, who sees: create code SUMMER50, fifty percent off, all products, expires Sunday, estimated cost based on last month's sales. She edits it to twenty percent and approves. The agent sees the edit as its tool result and adjusts its announcement copy to match.

Review UX

Approvals only work if reviewers actually engage. Batch low risk items so people can approve twenty drafts at once. Show diffs, like price one hundred and forty nine to one hundred and twenty nine dirhams. Send mobile friendly notifications with deep links. Track approval latency, because if approvals take hours, the agent's speed advantage disappears. And watch for rubber stamping. If everything is approved in two seconds, either the tier is too strict, or reviewers don't have enough information to judge.

Example: Jeddah promotions agent

A worked example. A restaurant group in Jeddah uses an agent to draft weekly promotions for fifteen branches. Drafting copy in Arabic and English is auto approved. Scheduling posts needs the branch manager, in a batch view. Changing menu prices in the point of sale system needs the regional manager, and changes over ten percent also need finance. Anything involving health claims or Ramadan timing escalates to the marketing lead. After a month, most scheduled posts were approved unchanged, so they raised the auto approve limit for small boosts while keeping price changes manual.

Hands-on: approval gate

The hands on code wraps risky tools in a gate. Each tool has a risk tier, and unknown tools default to the highest tier. Low tiers pass straight through. Scheduling passes automatically only under a small budget. Everything else creates a pending approval, notifies someone, and pauses the run. When a reviewer decides, the decision goes back into the conversation as a tool result, and the run resumes from its checkpoint.

Pitfalls and metrics

Avoid these pitfalls. Approval screens that show only raw JSON. Unknown tools defaulting to auto approve. Stale approvals executed days later against changed data, so re validate before execution. And agents that retry a rejected action with trivially different arguments, so count rejections and escalate. Measure approval rate, approval latency, how often reviewers edit, incidents per thousand actions and reviewer time per item.

Approvals out of hours

What about approvals when nobody is around, like overnight or on weekends? Decide explicitly. Some actions can wait: the agent pauses, and the approval is waiting in the morning. Some are time sensitive, so route them to an on call person, or pre approve narrow classes of action with tight limits, like pause any ad set that spends more than twice its daily budget. What you should never do is let an unanswered approval silently turn into an automatic yes after a timeout.

Deeper: the Jeddah promotions (illustrative)

Let's deepen the Jeddah restaurant example. Fifteen branches, weekly promotions, and in week one every scheduled post needed a branch manager's approval. Managers approved about nine in ten unchanged, illustrative figures, but approvals often waited until late afternoon, so morning posts went out late. The team adjusted: boosts under two hundred riyals were auto approved, managers got a single batched approval card each morning on their phones, and anything touching Ramadan timing still went to the marketing lead. Approval latency dropped from hours to minutes, and nobody lost control of the risky decisions, because price changes stayed firmly in tier three with finance sign off above ten percent.

Watch me do it: gate() and decide()

Watch me do it. Let's walk through the approval gate code. The risk map gives each tool a tier: search sales zero, draft promo one, schedule post two, update price three. The gate function looks up the tier and defaults unknown tools to three. Tier zero and one return none, meaning go ahead. Schedule post is also allowed when the boost budget is two hundred riyals or less. Anything else creates an approval id, stores the run id, tool, arguments and tier in pending, sends a notification and returns a pending status. In the loop, when gate returns pending, I save the run and stop. Later, the manager's decision calls decide with approved true or false, their name and a reason. That returns a decision payload, which I append as the tool result before resuming the run. So if she rejects a discount as too high, the agent reads that reason and proposes something smaller.

Try this now

Try this now. List every tool your agent can call and give each a tier from zero to three. For your riskiest tool, write the exact approval message a reviewer would see on their phone: the action, the key arguments, why the agent wants it, the evidence, and the expected impact, in plain language. Show it to someone who'd actually approve it and ask, could you decide in thirty seconds? If not, rewrite it until they can.

Recap

To recap: tier every action by risk, build approvals with interrupt, present, decide, resume and audit, define escalation triggers, and design review screens people will use. Your next step: build the action inventory for your agent, and sketch the exact approval message a reviewer would see for your riskiest action.

Key takeaways

Design human involvement deliberately with a risk-tiered action inventory.
Approvals need interrupt, clear presentation, decision, resume and audit.
Feed decisions back to the agent as tool results so it can adapt.
Define escalation triggers and hand over with a summary.
Measure approval latency and edit rates to move actions between tiers.

Try it

Build an action inventory for your agent's tools with tiers and policies, and sketch the approval message a reviewer would see for your riskiest action.