Human-in-the-Loop: The Booking No One Approved

Most agent guardrails check whether an action is well-formed. They never ask whether anyone should have done it at all. Here is how to build the one seam where a person steps in before the irreversible.

Your agent will eventually do something it cannot undo, and the validators you already wrote will wave it right through. Most agent guardrails check whether an action is well-formed.

In this article: You will learn why gating every action is a mistake, and how to gate the small slice that actually matters. We cover the three conditions that route an action to a human, a routing rule the model cannot argue with, how to pause and resume a run in both the Claude Agent SDK and LangChain Deep Agents, and the timeout policy that decides what silence means. By the end you will be able to build a human-in-the-loop approval gate that treats a non-response as a denial, not a yes.

Sarah asked for "economy class, under $400, flexible dates." The agent found a business-class fare at $1,847, decided the price was acceptable given "premium availability," and called book_flight with a non-refundable ticket. No second model challenged the reasoning. No human approved the charge before it hit her card. By the time she saw the confirmation, the ticket was locked: no cancellation, no refund, and a two-hour support call that resolved nothing. One missed gate, one customer gone.

Ordinary input validation would not have caught this. The fare was a valid number, the date was real, and the booking was structurally clean. What failed was a judgment call about whether a $1,847 non-refundable purchase, made against a request for an economy fare under $400, should execute without a person in the loop. Some actions are not the harness's to make alone.

This article builds the gate that routes those actions to a human, and the policy for what happens when the human says nothing. We will build it twice: once in the Claude Agent SDK and once in LangChain Deep Agents. The shapes differ. The contract is identical.

Gate the 0.8%, not everything

The instinct after reading Sarah's story is to require approval for everything. Resist it. Anthropic's production measurement found that only about 0.8% of observed API tool calls are irreversible. Gating the other 99.2% buys you nothing and costs you a lot. Every approval prompt adds latency, trains users to click through without reading, and turns the human into a rubber stamp. An approval gate that fires constantly is worse than no gate, because it manufactures the exact reflexive approval it was meant to prevent.

So the gate is a decision before it is a mechanism. Route to a human only when one of three conditions holds:

1. Irreversibility. The action cannot be undone once it executes. book_flight with a non-refundable fare is irreversible. search_flights is not.
2. High cost. The blast radius of a bad call exceeds the cost of a human's attention. A $1,847 ticket clears that bar; a $32 flexible fare does not.
3. Trust-boundary crossing. The plan acts on content that the schema cannot validate, where a compromised input could be steering the action.

Everywhere else, the validators and contracts you already built are enough. Pay for human attention where it changes an outcome, and nowhere else.

The routing rule the model cannot argue with

When the gate does fire, the routing is mechanical, governed by two axes: reversibility and confidence. The one rule that matters most is a category-level constraint, not a tuning knob. An irreversible action always routes to a human, regardless of how confident the agent is. Models do not autonomously execute high-stakes irreversible calls on their own say-so. Reversible actions have more room: agreement proceeds, a correctable objection gets amended and retried, and low confidence escalates.

The critical design move is that the list of irreversible tools lives in harness code, not in the prompt. The model cannot talk the gate out of escalating book_flight, because the decision is not the model's to make.

from dataclasses import dataclass

IRREVERSIBLE = {"book_flight", "cancel_booking", "issue_refund"}  # ①
CONFIDENCE_FLOOR = 0.70

def route(tool_name: str, confidence: float) -> str:
    if tool_name in IRREVERSIBLE:                     # ② category constraint
        return "escalate" if confidence >= CONFIDENCE_FLOOR else "default_deny"
    if confidence < CONFIDENCE_FLOOR:                 # ③ reversible, low confidence
        return "escalate"
    return "proceed"                                  # ④ reversible, confident

① The registry of irreversible tools is harness-side data, not a prompt instruction, so the model cannot edit it or argue it away. ② An irreversible tool escalates to a human no matter what the confidence is, and default-denies outright when confidence is below the floor. ③ A shaky reversible call escalates too, because low confidence on a reversible action is still worth a person's glance. ④ A confident reversible call proceeds without bothering anyone; the model reads this result, never the rule that produced it.

Note: The full extracted listing at code/harness_engineering/part-7-human-in-the-loop/listings/01-route.py shows the runnable version with the markers removed.

Build the gate: pause for a person

Both frameworks let you suspend the run at a tool call and wait for a human decision. The shapes differ, but the contract is identical: stop before the irreversible action, surface the proposal to a person, and do not proceed until they decide.

In the Claude Agent SDK, a PreToolUse hook returns an ask decision for the tools that need approval. The run pauses; the harness fires a notification with the proposed action and waits.

from claude_agent_sdk import ClaudeAgentOptions, HookMatcher

IRREVERSIBLE = {"book_flight", "cancel_booking", "issue_refund"}

async def approval_gate(input_data, tool_use_id, context) -> dict:
    name = input_data.get("tool_name", "")
    if name not in IRREVERSIBLE:                      # ① 99.2% pass straight through
        return {}
    return {"hookSpecificOutput": {                  # ② route irreversible to a human
        "hookEventName": input_data["hook_event_name"],
        "permissionDecision": "ask",
        "permissionDecisionReason":
            f"{name} is irreversible and requires human approval.",
    }}

options = ClaudeAgentOptions(
    hooks={"PreToolUse": [HookMatcher(matcher="*", hooks=[approval_gate])]},  # ③
)
# For long reviews, return "defer" instead of "ask" to suspend the process without
# burning context, then resume the persisted session when the decision arrives.

① The gate checks the harness-side registry first, so reversible tools never interrupt anyone and the hook returns an empty dict that lets the call proceed. ② An ask decision suspends execution and routes the proposal, with its reason, to a person; for reviews that may run long, defer ends the process and persists the session so a cross-timezone approval does not hold a context window open for an hour. ③ The hook is wired into PreToolUse with a wildcard matcher, so every tool call passes through the gate before it fires, reusing the same durable-session property the harness already provides.

Note: The full extracted listing at code/harness_engineering/part-7-human-in-the-loop/listings/02-approval-gate-sdk.py shows the runnable version with the markers removed.

Deep Agents models the same risk tiers declaratively with interrupt_on, backed by a checkpointer so the paused run can be resumed.

from langgraph.checkpoint.memory import MemorySaver
from langgraph.types import Command
from deepagents import create_deep_agent

agent = create_deep_agent(
    model="anthropic:claude-sonnet-4-6",
    tools=[search_flights, book_flight, issue_refund],
    interrupt_on={
        "book_flight": {"allowed_decisions": ["approve", "edit", "reject"]},  # ① full control
        "issue_refund": {"allowed_decisions": ["approve", "reject"]},
        "search_flights": False,                     # ② read-only: never interrupt
    },
    checkpointer=MemorySaver(),                      # ③ required to pause and resume
)
config = {"configurable": {"thread_id": "sarah-1"}}

agent.invoke({"messages": [{"role": "user",
    "content": "Book the cheapest economy fare under $400."}]}, config)
# The run halts at book_flight. After a human decides, resume:
agent.invoke(Command(resume={"decisions": [{"type": "approve"}]}), config)  # ④

① High-risk tools get the full approve / edit / reject set, so a reviewer can correct a fare rather than only accept or refuse it. ② Read-only tools are exempt with False, the same 0.8% discipline expressed as configuration. ③ The checkpointer is mandatory; it is what holds the paused state between the halt and the resume. ④ The application supplies the human decision through Command(resume=...), and the run continues from exactly where it stopped.

Note: The full extracted listing at code/harness_engineering/part-7-human-in-the-loop/listings/03-interrupt-on-deepagents.py shows the runnable version with the markers removed.

Three questions every gate must answer

A pause is not a policy. Before you ship the gate, answer three things explicitly.

Who sees the request. The proposal, the objection, and the stakes go to an actual person through a real channel: a Slack message, an email, or a dashboard. A gate that notifies no one is a gate that never opens.

What counts as approval. It is risk-proportionate, and stricter than it feels like it needs to be. Opening a link is not approval. For a $1,847 non-refundable booking, a single button click is too weak; a typed confirmation code tied to the session is the bar. Define this before you need it.

What happens when no one responds. This is the question most teams skip, and it is the one that bit Sarah's successors. Silence is not consent.

Silence is not consent

The timeout policy is the heart of the gate, and it follows the same reversibility axis as the routing. If no human approves an irreversible action within the window, the action is denied and logged, not held open and not waved through. For reversible actions you can default to allow-with-audit. A decision that arrives after the window has closed is discarded, because a reviewer clicking approve five minutes late must not silently override a denial that was already recorded.

import asyncio, time

TIMEOUT_S = 300  # 5 minutes for irreversible actions

async def await_decision(req: dict, notify, poll, irreversible: bool) -> str:
    """Return 'allow' or 'deny'. Encodes the timeout policy, not just the wait."""
    await notify(req)                                # ① fire notification, never block
    deadline = time.monotonic() + TIMEOUT_S
    while time.monotonic() < deadline:              # ② poll within the window
        decision = await poll(req["session_id"])
        if decision and decision["decided_at"] <= deadline:   # ③ discard stale approvals
            return decision["choice"]
        await asyncio.sleep(5)
    return "deny" if irreversible else "allow"       # ④ silence: default-deny if irreversible

① The notification fires and the function returns control immediately; no thread blocks while a human thinks. ② The harness polls inside the deadline window, sleeping between checks so it is not a busy loop. ③ A decision is honored only if it landed inside the window, so a late approve cannot reverse an already-logged denial. ④ The timeout branch is the policy in code: irreversible silence becomes a denial, reversible silence becomes an audited proceed. Wire this behind either framework's gate and Sarah's booking cannot fire on a non-response.

Note: The full extracted listing at code/harness_engineering/part-7-human-in-the-loop/listings/04-await-decision.py shows the runnable version with the markers removed.

Do this today

Stand the gate up against Sarah's exact request and prove each branch by hand. Each check takes a few minutes and exposes a different failure mode.

• Send the agent Sarah's request and watch the split. Confirm that book_flight pauses for approval while search_flights runs untouched. If a read-only tool interrupts, your registry is too broad.
• Let the approval window expire on an irreversible action. Confirm that the gate denies and logs, rather than holding the run open or quietly proceeding. Silence must resolve to a denial.
• Approve a request one second after the deadline. Confirm that the late decision is discarded and the recorded denial stands. A stale approve must never reverse a logged denial.
• Pick one real notification channel. Wire the gate to Slack, email, or a dashboard so a paused run reaches a person. A gate that notifies no one never opens.
• Define what approval means in writing. For high-stakes actions, require a typed confirmation code tied to the session, not a single button click. Decide this before you need it.

The discipline to stop

Sarah's agent was not wrong about how to book a flight. It was wrong to book that flight without asking. The harness function here is judgment about its own limits: gate the small fraction of actions that are irreversible or costly, route them to a real person, define what approval means, and treat silence as a denial rather than a yes.

The mechanics are small. A registry of irreversible tools that lives in code. A hook or an interrupt_on map that pauses the run. A timeout policy that defaults to deny. None of it is clever, and that is the point. The decision about which actions a machine may take alone is too important to leave inside a prompt the model can talk its way around.

Build it in either framework and the agent gains the one thing autonomy alone cannot give it: the discipline to stop.

---