The certification lag problem: accountability when the approved version is not the one running

When an AI agent causes harm in a high-stakes domain, one of the first questions in any accountability proceeding is deceptively simple: was the version that caused harm the version that was approved? In established engineering disciplines — pharmaceuticals, aviation, medical devices — this question has a clean answer. The approved artifact is frozen, the deployment record shows which version ran, and accountability can proceed from there.

For AI agents, the answer is almost always complicated. Certification takes time. Models improve continuously. The gap between when a version enters a certification process and when that process concludes can be twelve, eighteen, or thirty-six months. During that period, the model's developers have typically released several successor versions with measurably better performance, lower error rates, and fixed failure modes that the certified version still exhibits. By the time approval arrives, the operator faces a dilemma that no certification framework was designed to resolve.

The dilemma the lag creates

Using the certified version is defensible in a narrow sense — the certification is on record, the version is documented, the approval authority has validated it. But the certified version may have known limitations that newer versions have corrected. Deploying an older, worse model because it is certified, while a better one exists, is a choice that the certification framework makes look like prudence but which may produce worse outcomes for the people the agent is meant to serve.

Using an uncertified update introduces a different accountability gap. The operator may deploy the update because it genuinely reduces risk — fixing a safety-relevant failure mode, for instance — but any harm that follows happens outside the certification boundary. The operator cannot point to external validation. The accountability record shows a deployment that diverged from the approved version, and any investigation will spend considerable effort on whether that divergence caused the harm rather than on the substantive question of what the agent did.

Neither path is clean. The certification lag converts a question about model quality into a question about process compliance, and the two questions have different answers to different parties at different times.

At the post-quantum security crossing

Cryptographic standards have always had long certification cycles. The transition to post-quantum algorithms is playing out across a timeline measured in years, with standards bodies completing reviews long after the practical deployment of the algorithms they are reviewing. An AI agent handling post-quantum key operations that was certified against an earlier threat model is operating today against an evolved threat landscape that the certification process did not examine. The certification provides legitimate cover; it does not provide actual security assurance against threats that emerged after the review completed. Accountability for a compromise that exploits a gap in the certified threat model falls into the space between the certifying body's scope, the deployer's scope, and the vendor's scope — with each party pointing at the others.

At the hardware crossing

Hardware certification — for AI accelerators, embedded controllers, industrial compute platforms — involves lengthy testing against specific firmware versions and operating conditions. Security patches and firmware updates are a routine part of maintaining any deployed system. When a critical vulnerability is discovered, operators patch it, and the patched firmware is typically not the firmware that was certified. The alternative — running unpatched, vulnerable firmware because the patched version hasn't completed recertification — is not a viable choice in most operational contexts. Yet the accountability record for a system running patched, uncertified firmware is structurally similar to the record for a system running unauthorized modifications. The patch that improved security and the modification that introduced a vulnerability look the same in a certification audit.

At the care crossing

Medical AI faces the sharpest version of this problem. A device or software agent that achieves market authorization has been reviewed against a defined intended use, a defined patient population, and a specific version of the model. Real-world deployment rarely stays within those boundaries. Patient populations shift. The intended use expands. New model versions are released. Operators integrate the agent with other systems that the original authorization did not contemplate. At each step, the gap between the authorized artifact and the running system widens. When harm occurs and accountability proceedings begin, investigators must reconstruct which version ran, what its authorization covered, and which of the many departures from the authorization boundary contributed to the outcome. This is tractable in theory; in practice, the records needed to answer these questions are rarely maintained with the necessary precision.

What accountability architecture requires

The certification lag problem does not have a simple fix. Faster certification reduces the lag but does not close it; models will continue to improve faster than any institutional review process can operate. The structural solution is to treat certification not as a point-in-time approval of a frozen artifact, but as an ongoing relationship between the certifying authority, the deployer, and the version history of the agent.

This requires three things that current frameworks largely lack. First, deployment records must include version attestation — cryptographic evidence of which exact version ran at which time, preserved in a form that cannot be retroactively modified. Second, change impact assessments must distinguish certification-invalidating changes from minor updates, with clear accountability for the classification decision. Third, the certification boundary must be explicit in accountability proceedings: investigators should be required to state which aspects of an agent's behavior fell within the certified scope and which did not, before attributing harm to either.

The certified version and the deployed version are rarely the same. Accountability frameworks that treat them as interchangeable leave a structural gap that neither the certification body nor the deployer fully owns — and that gap is where most of the hard accountability questions live.