The memory problem: accountability when agents remember across sessions

Most accountability frameworks for AI agents are designed around the session. An agent receives a task, reasons about it, acts, and produces an output. The authorization that governs the session specifies what the agent may do, which data it may access, and which principals it must consult. When the session ends, the accountability record is complete. The agent has no further influence on anything until it is invoked again.

Persistent memory breaks this model. When an agent retains information across sessions — summaries of prior interactions, learned preferences, inferred context about users or environments, accumulated trust assessments of other agents — it carries that information into future decisions. Those future decisions are shaped by information that was produced under a prior authorization, by a prior invocation, in a prior context. The authorization record for the current session does not reflect that influence. The accountability chain is broken before the session begins.

This is not a hypothetical architecture. Every agent system that stores conversation history, user profiles, environmental state, or cross-session context is operating with persistent memory. The scope of the problem scales with the reach of the agent: a personal assistant that remembers preferences is manageable; a care agent that accumulates clinical context across months of patient interaction, or a security agent that builds a long-running model of network behavior, is operating in a regime where the accountability questions are substantially harder.

The authorization problem

Memory is created under one authorization and consumed under another. The principal who authorized the first session did not authorize the memory's future influence. If the principal later modifies the agent's authorization — restricting what it may act on, changing which data sources it may consult, revoking access to a particular system — the accumulated memory may reflect assumptions that the modified authorization no longer permits. The agent's effective permissions include everything encoded in its memory, not just the current authorization document. That gap is invisible to standard permission audits.

The problem sharpens when principals change. An agent serving a care team accumulates context from interactions with Team A. When Team B takes over, the agent carries forward context shaped by Team A's practices, judgments, and priorities. Team B has no record of what Team A's interactions contributed to the agent's current behavior. They inherit an agent that has been influenced by a principal they may have no knowledge of and whose authority over that agent has since ended.

The post-quantum security crossing

An agent managing infrastructure security accumulates a model of normal behavior: which endpoints communicate, at what frequency, under which certificates. That model is built from observations made under classical cryptographic assumptions. When the network migrates to post-quantum algorithms, the behavioral fingerprints change — new handshake patterns, different latency profiles, altered certificate chains. An agent whose memory encodes the pre-migration baseline will initially interpret quantum-transition traffic as anomalous. More dangerously, it may have encoded trust assessments — this endpoint is trusted, this certificate chain is canonical — that were valid under classical assumptions and are no longer valid after migration. The memory does not expire with the algorithm it was trained on. Clearing the memory is often treated as an engineering inconvenience rather than a security requirement.

The hardware crossing

A physical-world agent operating in an industrial environment builds a model of that environment over time: sensor baselines, equipment behavioral profiles, normal operating ranges. That accumulated model is valuable precisely because it took time to build. It is also a liability: if the physical environment changes — equipment is replaced, a sensor is recalibrated, an operating process is modified — the agent's memory reflects the prior environment. Decisions made against a stale environmental model can produce actions that are correct relative to the memory and dangerous relative to the current state. The problem is amplified in hardware because the consequences of acting on incorrect context are physical and potentially irreversible.

Memory in hardware agents also creates a decommissioning problem. When a physical agent is replaced or retired, its memory persists in whatever store it wrote to. That memory may contain sensitive operational data, inferred behavioral patterns of personnel, or security-relevant environmental observations. The decommissioning record for the agent rarely includes a memory audit. The information continues to exist under no active authorization.

The physical-world care crossing

A care agent accumulating clinical context across sessions faces accountability questions that are simultaneously medical, legal, and technical. The context it carries — inferred pain levels, behavioral patterns, medication responses, family interaction signals — may be more granular and more personally sensitive than any record explicitly created by a clinician. It was not created by a clinician. It was inferred by an agent, stored automatically, and accessed in subsequent sessions without the explicit consent of the patient to that specific inference.

Consent for the agent's initial deployment does not constitute consent for every memory entry the agent creates during its deployment. Patients who revoke consent or transfer their care to a different provider do not automatically revoke the agent's memory. The care record may be transferred; the agent's accumulated context rarely is, and rarely with the same traceability requirements. The memory persists, shaping decisions in sessions that the patient has formally ended their relationship with.

What the memory problem requires

The minimum response is to treat memory as a first-class artifact of the authorization lifecycle. Every memory write should carry a provenance record: which session created it, under which authorization, with which consent scope. Every memory read that influences a decision should appear in the decision's audit record. Memory contents should be subject to the same access controls and expiry rules as the data they summarize.

Beyond provenance, memory requires explicit governance of its lifecycle. Principals should be able to audit what an agent remembers about them, request deletion, and restrict what categories of inference may be persisted. When an authorization changes, the system should evaluate whether existing memory is consistent with the new authorization — not only whether future actions comply.

The memory problem is not solved by making agents stateless. State is often exactly what makes agents useful. But the accountability architecture for agentic systems has not kept pace with the stateful deployments that are already in production. The gap between what agents remember and what the authorization record reflects is where accountability is silently lost — session by session, inference by inference, across every deployment that has not explicitly designed its memory governance.