The case for deploying AI agents in regulated human domains — care homes, clinical settings, financial operations, logistics — nearly always includes the phrase "with a human in the loop." It appears in regulatory frameworks, procurement criteria, ethics guidelines, and board approvals. It is treated as the safeguard that makes agent deployment acceptable.

The paradox is that in most real deployments, the human-in-the-loop requirement is satisfied in name and violated in practice. Not through negligence — through arithmetic.

A care home running a hundred residents generates hundreds of clinical and welfare decisions per shift. An AI agent providing swallowing-safety screening, hydration monitoring, and medication cross-reference may flag or initiate thirty to fifty micro-decisions per carer per hour. If every decision requires individual human review, the agent creates more work than it removes. The carer either rubber-stamps everything — in which case there is no human in the loop, only a human on the loop — or they work through every item carefully, at which point the agent's value proposition collapses.

The same arithmetic holds in financial operations, autonomous logistics, and enterprise process agents. Scale and speed are the point. A human who must approve each decision at the pace the agent operates is not a loop; they are a bottleneck that the agent exists to route around.

What the problem is really asking

The demand for a human in the loop is not wrong. It is asking the right question, badly. What institutions actually need is not a human reviewing every decision — it is a system where consequential decisions cannot be made without appropriate human authority, where errors are reliably surfaced and correctable, and where the record is clear enough to reconstruct exactly what happened and why.

Those are accountability requirements, not review-rate requirements. They are satisfiable by architectures that look nothing like per-decision human approval.

Three substitutes that work

The first is categorical gate design. Not every decision carries the same consequence profile. An agent screening for swallowing risk can autonomously record a low-risk result, but flagging a potential dysphagia episode should require carer confirmation before any care plan change is recorded. The gate is not on every decision — it is on the decisions that carry meaningful clinical weight. Designed well, this redirects human attention to where it is genuinely required, rather than distributing it thinly across everything.

The second is statistical audit with forensic depth. If every action the agent takes is logged with full context, timestamp, and the reasoning state that produced it — and if those logs are signed by the hardware the agent runs on so they cannot be altered — then a regulator, a supervisor, or an investigator can reconstruct any decision sequence completely. This is not the same as reviewing decisions in advance, but it is often a stronger form of accountability: post-hoc review can be done carefully, at pace, by the right person, rather than under time pressure by whoever happens to be on shift.

The third is structured escalation. An agent that has calibrated uncertainty estimates for its decisions can route high-uncertainty actions to human review automatically. The architecture specifies: uncertainty above threshold X triggers human confirmation; certainty below threshold Y triggers immediate escalation to supervisor level. This concentrates human attention on genuinely difficult cases rather than routine ones.

What this requires architecturally

None of these substitutes work without two technical foundations. First, the agent's action log must be genuinely trustworthy — hardware-attested, append-only, and auditable by parties outside the deploying operator. An audit trail maintained by the operator in editable form is not an audit trail; it is a claim. Second, the scope and gate definitions must be set at deployment time by the appropriate authority — not adjustable by the agent, not configurable by the operator mid-deployment without a change-control record. The line between "automate this" and "require human" is a policy commitment, not a runtime parameter.

When these foundations exist, the human-in-the-loop question changes shape. The answer is not "a human reviews X% of decisions" but "certain categories of decision cannot proceed without human sign-off, all decisions are retrospectively accountable, and the architecture prevents the agent from reclassifying a gated decision as ungated." That is a stronger guarantee than human review at speed.

The real loop

The human-in-the-loop is not a rate limit on agent autonomy. It is a set of structural commitments about which decisions require human authority and which can be safely delegated, in a system where that boundary is enforced and auditable.

Agents that meet those commitments can be trusted to operate at scale. Agents that cannot — regardless of whether a human nominally signs off on each output — are not safe to deploy.

The loop is architectural. The question is whether it is honest.

在受监管的人类领域——安老院、临床环境、金融运营、物流——部署 AI 智能体的论据中，几乎无一例外地包含"人在回路中"这个短语。它出现在监管框架、采购标准、伦理指南和董事会审批中，被视为使智能体部署变得可接受的保障措施。

悖论在于：在大多数真实部署中，这一要求在名义上得到满足，而在实践中被违反。不是因为疏忽——而是因为算术。

一个运营着百名住客的安老院，每个班次产生数百个临床和福利决策。一个提供吞咽安全筛查、水分监测和药物交叉参考的 AI 智能体，每位护理员每小时可能标记或启动三十到五十个微决策。如果每个决策都需要人工逐一审查，智能体创造的工作量会多于减少的工作量。护理员要么对所有事项盖章通过——此时回路中根本没有人，只有一个站在回路上的人——要么认真逐一处理，此时智能体的价值主张便彻底崩塌。同样的算术问题适用于金融运营、自主物流和企业流程智能体。

这个问题真正在问什么

要求人在回路中并没有错。它在以一种糟糕的方式提出正确的问题。机构真正需要的不是对每个决策进行人工审查，而是一个确保高后果决策必须经过适当人类权威授权、错误能够可靠浮现和纠正、且记录清晰到足以完整重现发生了什么和为什么的系统。这些是问责要求，而非审查率要求。它们可以由与逐一人工审批截然不同的架构来满足。

三种有效的替代方案

第一是分类门控设计。并非每个决策都有相同的后果量级。筛查吞咽风险的智能体可以自主记录低风险结果，但对潜在吞咽障碍的标记，在任何护理计划变更被记录之前，应需要护理员确认。门控不针对所有决策，而是针对具有重大临床意义的决策。

第二是具有取证深度的统计审计。如果智能体的每个动作都以完整上下文、时间戳和产生该动作的推理状态记录，并由智能体运行的硬件签名以防篡改，那么监管机构、督导员或调查人员可以完整重现任何决策序列。这与事前审查不同，但往往是更强的问责形式：事后审查可以由合适的人员在充裕时间内仔细进行，而非在当班人员的时间压力下完成。

第三是结构化升级机制。具有校准不确定性估计的智能体，可以自动将高不确定性行动路由到人工审查。架构规定：不确定性超过阈值 X 触发人工确认；确定性低于阈值 Y 触发立即升级至督导级别。这将人类注意力集中在真正困难的案例上，而非分散在例行事务中。

真正的回路

人在回路中不是对智能体自主性的速率限制。它是关于哪些决策需要人类权威、哪些可以安全委托的一套结构性承诺——在一个这一边界得到强制执行和可审计的系统中。满足这些承诺的智能体可以被信任以规模化运作。无法满足的智能体，无论名义上是否有人对每个输出进行签署，都不安全，不应部署。

回路是架构性的。问题在于它是否诚实。

在受監管的人類領域——安老院、臨床環境、金融運營、物流——部署 AI 智能體的論據中，幾乎無一例外地包含「人在回路中」這個短語。它出現在監管框架、採購標準、倫理指引和董事會審批中，被視為使智能體部署變得可接受的保障措施。

悖論在於：在大多數真實部署中，這一要求在名義上得到滿足，而在實踐中被違反。不是因為疏忽——而是因為算術。

一個運營著百名住客的安老院，每個班次產生數百個臨床和福利決策。一個提供吞嚥安全篩查、水分監測和藥物交叉參考的 AI 智能體，每位護理員每小時可能標記或啟動三十到五十個微決策。如果每個決策都需要人工逐一審查，智能體創造的工作量會多於減少的工作量。護理員要麼對所有事項蓋章通過——此時回路中根本沒有人，只有一個站在回路上的人——要麼認真逐一處理，此時智能體的價值主張便徹底崩塌。同樣的算術問題適用於金融運營、自主物流和企業流程智能體。

這個問題真正在問什麼

要求人在回路中並沒有錯。它在以一種糟糕的方式提出正確的問題。機構真正需要的不是對每個決策進行人工審查，而是一個確保高後果決策必須經過適當人類權威授權、錯誤能夠可靠浮現和糾正、且記錄清晰到足以完整重現發生了什麼和為什麼的系統。這些是問責要求，而非審查率要求。它們可以由與逐一人工審批截然不同的架構來滿足。

三種有效的替代方案

第一是分類門控設計。並非每個決策都有相同的後果量級。篩查吞嚥風險的智能體可以自主記錄低風險結果，但對潛在吞嚥障礙的標記，在任何護理計劃變更被記錄之前，應需要護理員確認。門控不針對所有決策，而是針對具有重大臨床意義的決策。

第二是具有取證深度的統計審計。如果智能體的每個動作都以完整上下文、時間戳和產生該動作的推理狀態記錄，並由智能體運行的硬件簽名以防篡改，那麼監管機構、督導員或調查人員可以完整重現任何決策序列。這與事前審查不同，但往往是更強的問責形式：事後審查可以由合適的人員在充裕時間內仔細進行，而非在當班人員的時間壓力下完成。

第三是結構化升級機制。具有校準不確定性估計的智能體，可以自動將高不確定性行動路由到人工審查。架構規定：不確定性超過閾值 X 觸發人工確認；確定性低於閾值 Y 觸發立即升級至督導級別。這將人類注意力集中在真正困難的案例上，而非分散在例行事務中。

真正的回路

人在回路中不是對智能體自主性的速率限制。它是關於哪些決策需要人類權威、哪些可以安全委託的一套結構性承諾——在一個這一邊界得到強制執行和可審計的系統中。滿足這些承諾的智能體可以被信任以規模化運作。無法滿足的智能體，無論名義上是否有人對每個輸出進行簽署，都不安全，不應部署。

回路是架構性的。問題在於它是否誠實。