The dependency creation problem: accountability when AI care agents shift the balance of agency

The deployment rationale for AI care agents almost universally invokes independence. The goal is to keep someone in their home longer, to give a care recipient more autonomy, to reduce reliance on human caregivers for tasks the person can still manage with support. The agent supplements capacity; it does not replace it.

This framing is accurate at deployment. It does not remain accurate indefinitely.

An AI care agent that consistently handles a function — tracking medication timing, monitoring mobility patterns, flagging nutritional gaps, managing social scheduling — will, over time, reduce the active practice of that function by the care recipient. Not through any design flaw. Because that is what consistent, reliable assistance does.

Why functional dependency is a structural outcome, not a side effect

The same mechanism that makes AI care agents effective makes them dependency-creating: they absorb reliable management of consequential tasks, so the person using them no longer needs to manage those tasks directly. For care recipients in situations where capacity is already constrained, this absorption is rarely monitored as a risk. It is treated as the point.

The dependency created is functional. The care recipient may retain the cognitive and physical capacity for a function the agent handles, but they stop practicing it, stop trusting their own judgment about it, and may lose the information infrastructure — habits of tracking, maintained contacts, established routines — that would allow them to resume it independently. If the agent is removed or significantly modified, the recipient is not returned to their original position. They are returned to a position that is, in some dimensions, more precarious than the one they occupied at enrollment.

This is distinct from the automation atrophy problem, which concerns oversight principals losing the skill to check an agent's work. Here the concern is the care recipient's own functional agency — the erosion of practiced capacity in the person the system is ostensibly designed to serve.

The hardware infrastructure problem

Embedded AI care devices create a specific version of this dynamic. As a device becomes the persistent ambient layer of a home care environment — sensing, alerting, managing — it begins to function as infrastructure in the same way that heating and running water are infrastructure. A device failure is no longer a service interruption; it is a welfare event for the person whose daily routine has been structured around the device's presence.

The accountability structures for this transition are almost never established at deployment. Care agents are governed as optional tools. As long as the care recipient could, in principle, manage without the device, it is treated as a supplement. The moment when the device has become structural — when the recipient cannot, in practice, function safely without it — is typically not flagged, not formally documented, and not reflected in any consent record or care plan. It happens gradually, and the governance framework does not register gradual.

The accountability gap at the moment of change

The dependency creation problem becomes most visible at moments of change: the device is discontinued, the software platform is shut down, the underlying model is deprecated, or a firmware update significantly alters the agent's behavioral profile.

At that moment, accountability questions proliferate. Who is responsible for managing the transition for care recipients who have become functionally dependent on this agent? Who in the care system was aware of that dependency? Who can reconstruct, from the deployment record, what the agent had been doing and what the recipient would need in order to replace those functions through other means?

If the answers are "no one was assigned," "the dependency was never documented," and "the record does not support that reconstruction," then the care system has created a significant vulnerability — and has no structural mechanism for governing it. The agent was introduced as a supplement; it became infrastructure; accountability was never updated to reflect the transition.

What correct architecture requires

Governing the dependency trajectory requires treating it as a tracked dimension of the deployment, not an incidental outcome that falls outside the system's defined scope.

This means periodically assessing which functions the agent has absorbed and which the care recipient is no longer actively practicing — and treating that assessment as a care record entry, not an informal observation. It means building care transition protocols that include dependency audits as a structural requirement of long-running deployments, not optional good practice. It means ensuring that discontinuation or modification events trigger, by design, an assessment of what support the recipient needs to navigate the change.

And it requires intellectual honesty about a claim that almost every care AI deployment makes: that the agent supports independence. Supporting independence is a meaningful claim only when the deployment is actively monitored against the risk that the support has, over time, created a form of dependence that was not planned for, is not governed, and will not be visible until something about the system changes.

At Asaptic Labs, we think the right frame is not "does the agent support independence at deployment" but "is the care relationship, including its dependency trajectory, governed at each point in the deployment lifecycle." That question requires different instrumentation, different accountability assignments, and a more honest account of what long-running AI care deployments actually do to the people they serve.