The semantic gap problem: accountability when intent and interpretation diverge

Every instruction given to an AI agent is expressed in language that is, to some degree, ambiguous. Words like "update," "monitor," "escalate," and "manage" carry meanings that shift with context, domain convention, and the specific circumstances at hand. The agent resolves that ambiguity — it must, to act at all — but it resolves it silently. The principal who issued the instruction receives no indication of how the agent interpreted "update the configuration" or "handle flagged cases." The action that follows is authorized in the sense that the principal issued an instruction; whether it is authorized in the sense of matching what the principal actually intended is a different question entirely. This is the semantic gap problem, and it sits at the foundation of every accountability claim made about AI agents.

Why this is not a UX problem

The semantic gap is sometimes treated as a communication design problem — if instructions were better specified, the gap would close. But this framing mislocates the problem. The gap does not arise from poorly written instructions. It arises from the structural mismatch between how principals communicate and how agents act. Principals communicate in language that assumes shared context, domain knowledge, and good-faith interpretation of intent. Agents act on their training-encoded interpretation of that language, which may diverge from the principal's intent without either party detecting the divergence.

No amount of better instruction design eliminates this gap entirely, because the instructions that reach an agent in production are not written by adversaries trying to confuse it — they are written by domain professionals expecting a reasonable reading. When the agent's "reasonable reading" differs from the professional's intent, the gap is structural.

The accountability consequence is serious: when harm results from an agent acting on a plausible-but-wrong interpretation of an ambiguous instruction, the principal may believe they authorized the correct action while the agent records that it complied with the instruction as given. Both parties have clean records. The gap between intent and interpretation is invisible in the audit log, which faithfully records what was instructed and what was done — but not the divergence between what was meant and what was understood.

The post-quantum crossing

Cryptographic migrations are particularly vulnerable to semantic gap failures. Instructions like "migrate to a quantum-resistant algorithm" or "prioritize forward-secrecy configurations" carry enormous implicit specification: which algorithms qualify? Under which performance constraints? For which key lengths and protocol versions? An agent that interprets "quantum-resistant" to mean "any algorithm on a standard compliance list" may select an algorithm that satisfies the label but not the underlying security intent.

The instruction was followed. The intent was missed. The difference may not surface for years — until the specific threat model the principal had in mind is tested against the migration that was actually performed. By then, the algorithm selected, the protocol versions configured, and the key material generated are deeply embedded in infrastructure. The audit log shows clean compliance. The semantic gap is invisible in it.

The hardware crossing

Fleet management agents receive instructions about maintenance, configuration, and intervention that are semantically dense. "Address anomalous power draw" does not specify whether to throttle, reboot, isolate, or alert. "Maintain within operational parameters" embeds all the complexity of what "operational" means for a device that operates across varied conditions. An agent that resolves these instructions by defaulting to its most commonly trained interpretation may act correctly on typical cases and produce failures disproportionately concentrated in novel conditions — precisely the conditions where the principal's intent was most specific and the agent's interpolation was least reliable.

The maintenance action recorded in the log matches the instruction verbatim. The divergence from intent is recorded nowhere. Investigators reconstructing the event trace see a compliant agent executing instructions as given. The accountability question — did the agent interpret the instruction as the operator meant it? — has no answer in the record.

The physical-world care crossing

Care instructions carry the most consequential semantic gaps. Instructions about when to escalate, how to interpret behavioral signals, and what constitutes "stable" or "distressed" are expressed in language that professionals in the same discipline interpret differently based on training, experience, and the specific individual in front of them. A care agent resolving "monitor for distress signals" applies its own interpretation of what counts as distress, drawn from its training distribution — which may not match the intent of the care team for this individual with this history under these conditions.

When harm results, the instruction log shows that monitoring was performed. The semantic gap — between what the care team meant by "distress signals" and what the agent understood as distress signals — is invisible in the record. The accountability that should attach to an agent acting outside the intended meaning of its instructions is obscured by documentation that shows surface-form compliance.

What accountability architecture requires

Closing the semantic gap entirely is not possible. Constraining its consequences is. Accountability architecture for AI agents operating on natural-language instructions requires, at minimum, that agents surface their interpretation of ambiguous instructions before acting on consequential decisions — not as a formality, but as a genuine checkpoint at which the principal can confirm or correct the reading. Systems that act first and record their interpretation afterward, or that never surface their interpretation at all, make the semantic gap permanently invisible.

Deployment in high-stakes domains — cryptographic infrastructure, fleet management, physical care — requires scope-specific interpretation frameworks: structured vocabularies that bound agent interpretation of domain-critical terms, and escalation requirements that trigger when instructions are novel, ambiguous, or lack precedent in the agent's training. Where interpretation-confirmation cannot be made mandatory, logging requirements should include the agent's operative interpretation alongside the instruction it was applied to, so that post-hoc accountability review can assess not only what was done but what was understood.

The alternative — agents that silently resolve ambiguity and produce audit records showing only the surface form of compliance — is an accountability architecture in which the most consequential interpretive judgments are made by the agent and recorded nowhere. When those judgments diverge from principal intent and harm follows, both the principal and the audit log will report clean hands. The semantic gap is the space between them.