← Notes from the Crossings
× PHYSICAL-WORLD CARE · × POST-QUANTUM SECURITY · × HARDWARE

The machine unlearning gap: accountability when the right to erasure cannot reach AI model weights

2026-06-14 6 min read

The right to erasure is a foundational data protection right. For static records, the mechanism is straightforward: locate the record, delete it, confirm deletion. For AI models trained on that data, the mechanism does not exist in any production-verified form. A patient can exercise their statutory right to have their health records deleted. The care AI system that was trained on those records will continue to operate, its weights encoding compressed statistical influence from data that no longer legally exists. Nobody can demonstrate that the influence has been removed, because the tools to verify removal have not been built into any deployed system.

Training an AI model using gradient descent encodes statistical patterns from the training dataset into the model's weight matrix. These weights are not records of individual data points. They are a distributed, high-dimensional compression of all training examples simultaneously — a superposition of everyone's data, encoded in a form that cannot be surgically decomposed by patient. There is no bit in the weight matrix that belongs to a specific person. The influence of their data is diffused across millions or billions of parameters, entangled with the influence of every other training example. Deleting the source record leaves the influence intact.

The machine learning research community has developed techniques aimed at removing the influence of specific training examples without complete model retraining. These "machine unlearning" methods range from full retraining from scratch — prohibitively expensive for large deployed models — to approximate unlearning approaches that are computationally cheaper but provide only probabilistic rather than verifiable guarantees. No deployed care AI system currently offers patients a mechanism to verify that unlearning has completed, or to confirm that the resulting model behaves identically to a model that was never trained on their data. The right exists. The enforcement mechanism does not.

The care AI context makes this gap especially acute. Longitudinal patient records are useful for AI training precisely because they capture rare event sequences, subtle deterioration patterns, and drug interactions across time. The data most valuable for training — and therefore most difficult to disentangle from model weights — is often the data patients are most motivated to erase: records of a past psychiatric crisis, a stigmatized diagnosis, a period of dependency. These are not the average training examples whose influence is small and evenly diffused. They are outlier events that are overrepresented in what models learn to weight, because they are the signals most predictive of future risk. The more clinically significant the data, the harder it is to remove its influence.

The post-quantum dimension adds a second layer. One mitigation that has been proposed for the unlearning problem is training on cryptographically protected data: if the source data was encrypted with keys that are then destroyed, the training inputs become inaccessible even if the raw storage medium persists. This approach has structural limits that the quantum transition makes worse. Practical homomorphic encryption over care AI training data at scale remains computationally intractable for most deployed systems. And data encrypted today under classical schemes to enable "erasure by key destruction" faces a harvest-now-decrypt-later exposure window: adversaries collecting encrypted training datasets now may be able to recover the underlying data within the regulatory retention period as quantum-capable cryptanalysis matures. Erasure by key deletion requires confidence in the long-term security of the encryption scheme — a confidence the quantum transition systematically erodes.

The hardware crossing surfaces the same problem in federated deployments. Federated learning routes gradient updates through edge hardware — patient-premises devices, hospital-local compute nodes — rather than transmitting raw patient data to a central server. When a patient exercises erasure rights, the raw data residing on the edge device can be deleted. The gradient update that device computed from that data, transmitted to and aggregated into the central model, cannot be unbaked. Hardware attestation schemes can verify that the edge device ran the correct computation. They cannot verify that the computation is reversible. The privacy architecture that makes federated learning more protective of source data makes machine unlearning harder to implement, because the gradients that need to be undone were computed under a protocol designed to protect them from inspection.

Three responses are available within current technical limits, none of them complete. The first is erasure honesty: any care AI deployment should disclose explicitly to patients, at the point of consent, that exercising the right to erasure removes source records but does not currently provide verifiable removal of training influence. This is not a comfortable disclosure. It is an accurate one. The second is data minimization at source: reducing the granularity and retention window of training data reduces any individual's statistical contribution to the model, making approximate unlearning more tractable and reducing the magnitude of the gap. The third is differential privacy training: models trained with formal differential privacy guarantees bound the maximum contribution of any single training example to model outputs. This does not solve the erasure problem, but it quantifies the privacy loss — replacing an unverifiable gap with a stated upper bound that can be disclosed, audited, and reduced over successive model versions.

The machine unlearning gap is not a failure of data protection law. The right to erasure was designed well, for the data architecture that existed when it was written. The gap is between that architecture — where data was a static file that could be located and destroyed — and the architecture that AI training has produced, where personal data is encoded as distributed statistical influence across billions of model parameters. Closing that gap requires either advances in verifiable unlearning algorithms that can be practically deployed at care AI scale, or honest disclosure to every patient that the system retains influence it cannot remove. The first is an engineering problem. The second is an accountability problem. Neither has been treated as urgent. Both are.

摘要 — 简体

删除权(被遗忘权)是数据保护法的核心权利。对于静态记录,删除操作直接有效;但对于已在该数据上训练的AI模型,删除机制在任何已部署的生产系统中均未得到验证。梯度下降训练将训练数据的统计影响分散编码至模型权重矩阵中,无法按患者进行手术式拆解。医疗AI场景尤为严峻——纵向患者记录正是因为包含罕见事件序列和微妙的恶化规律才对训练最具价值,而这类数据往往也是患者最希望删除的敏感信息,其影响也最难从权重中剥离。后量子维度进一步恶化了这一困境:以密钥销毁实现"删除"的方案依赖加密方案的长期安全性,而这一安全性正被量子转型所侵蚀。联邦学习部署在边缘硬件上计算梯度更新,使源数据删除后依然无法撤销已汇总至中央模型的梯度贡献。现实范围内的应对措施包括:诚实披露(删除源记录但无法保证移除训练影响)、源头数据最小化,以及差分隐私训练(以量化上界取代不可验证的缺口)。

摘要 — 繁體

刪除權(被遺忘權)是資料保護法的核心權利。對於靜態記錄,刪除操作直接有效;但對於已在該資料上訓練的AI模型,刪除機制在任何已部署的生產系統中均未得到驗證。梯度下降訓練將訓練資料的統計影響分散編碼至模型權重矩陣中,無法按患者進行手術式拆解。醫療AI場景尤為嚴峻——縱向患者記錄正是因為包含罕見事件序列和微妙的惡化規律才對訓練最具價值,而這類資料往往也是患者最希望刪除的敏感資訊,其影響也最難從權重中剝離。後量子維度進一步惡化了這一困境:以金鑰銷毀實現「刪除」的方案依賴加密方案的長期安全性,而這一安全性正被量子轉型所侵蝕。聯邦學習部署在邊緣硬體上計算梯度更新,使源資料刪除後依然無法撤銷已彙總至中央模型的梯度貢獻。現實範圍內的應對措施包括:誠實揭露(刪除源記錄但無法保證移除訓練影響)、源頭資料最小化,以及差分隱私訓練(以量化上界取代不可驗證的缺口)。

× 物理世界照护 · × 后量子安全 · × 硬件

机器遗忘缺口:当删除权无法触达AI模型权重时的问责

2026-06-14 6 分钟阅读

删除权(被遗忘权)是数据保护法的核心权利。对于静态记录,机制简单明了:定位记录、删除、确认。对于在该数据上训练的AI模型,这一机制在任何已验证的生产系统中均不存在。患者可以行使法定权利要求删除其健康记录,而使用这些记录训练出的医疗AI系统将继续运行,其权重中仍编码着来自已不再合法存在的数据的压缩统计影响。没有人能够证明这种影响已被移除,因为用于验证移除的工具从未被内置到任何已部署的系统中。

通过梯度下降训练AI模型,训练数据集的统计规律被编码到模型的权重矩阵中。这些权重并非单个数据点的记录,而是所有训练样本的分布式、高维压缩——每个人的数据同时叠加编码于其中,无法按患者进行手术式分解。权重矩阵中没有任何比特"属于"某一特定患者。其数据的影响被扩散到数百万乃至数十亿个参数中,与其他所有训练样本的影响相互缠绕。删除源记录,影响依然存在。

机器学习研究界已开发出旨在移除特定训练样本影响的技术,而无需完全重训练模型。这些"机器遗忘"方法从从头完整重训练(对大型已部署模型成本高昂)到近似遗忘方法(计算开销较低,但仅能提供概率性而非可验证的保证)不等。目前没有任何已部署的医疗AI系统向患者提供验证遗忘已完成的机制,也无法确认模型的行为与从未在其数据上训练过的模型相同。权利存在,执行机制不存在。

医疗AI场景使这一缺口尤为严峻。纵向患者记录之所以对AI训练有价值,正是因为它们记录了罕见事件序列、微妙的恶化规律和跨时间的药物相互作用。训练价值最高——因此最难从模型权重中分离——的数据,往往也是患者最有动力删除的数据:过去的精神危机记录、污名化的诊断、依赖症期间的记录。这些不是统计贡献微小且均匀扩散的普通训练样本,而是被模型赋予高权重的异常事件,因为它们是预测未来风险最具价值的信号。数据的临床意义越大,移除其影响就越困难。

后量子维度叠加了第二层问题。针对机器遗忘问题,有人提出在加密保护的数据上训练:若源数据使用随后销毁的密钥加密,即使原始存储介质仍然存在,训练输入也无法访问。这一方法存在量子转型使之恶化的结构性局限。在大多数已部署系统中,在医疗AI训练数据规模上实用的同态加密在计算上仍不可行。而今天以经典加密方案对训练数据集加密、通过密钥销毁实现"删除"的做法,面临"现在收割,以后解密"的暴露窗口:对手现在收集加密训练数据集,可能在监管保留期内随量子密码分析能力成熟而恢复底层数据。通过密钥销毁实现删除,需要对加密方案的长期安全性有信心——这种信心正被量子转型系统性地侵蚀。

联邦学习部署在硬件层面呈现出同样的问题。联邦学习将梯度更新路由至边缘硬件——患者自有设备、医院本地计算节点——而非将原始患者数据传输至中央服务器。当患者行使删除权时,边缘设备上存储的原始数据可以删除,但该设备在该数据上计算的梯度更新,已传输并汇总至中央模型,无法撤销。硬件证明方案可以验证边缘设备运行了正确的计算,但无法验证该计算是可逆的。使联邦学习对源数据保护更充分的隐私架构,反而使机器遗忘更难实施,因为需要撤销的梯度是在专为保护其不被检查的协议下计算的。

在当前技术限制范围内,有三种应对措施可用,但均不完整。第一是诚实披露:任何医疗AI部署都应在患者同意时明确告知,行使删除权将移除源记录,但目前无法提供可验证的训练影响移除保证。这并非令人舒适的披露,但这是准确的。第二是源头数据最小化:降低训练数据的粒度和保留窗口,可减少任何个人对模型的统计贡献,使近似遗忘更具可行性,并缩小缺口幅度。第三是差分隐私训练:以形式化差分隐私保证训练的模型,限制任何单个训练样本对模型输出的最大贡献。这并不能解决删除问题,但可以量化隐私损失——以可披露、可审计、并在模型迭代中可不断降低的具体上界,取代不可验证的缺口。

机器遗忘缺口不是数据保护法的失败。删除权的设计是好的,是为其创立时的数据架构设计的。缺口在于:那个架构——数据是可定位和销毁的静态文件——与AI训练创造的架构之间的落差:个人数据作为分散的统计影响,编码在数十亿模型参数中,在源记录消失后长期存在。弥合这一缺口,需要可在医疗AI规模上实际部署的可验证遗忘算法的进展,或向每位患者诚实披露系统保留着其无法移除的影响。前者是工程问题,后者是问责问题。两者都尚未被视为紧迫,但两者都是。

× 物理世界照護 · × 後量子安全 · × 硬件

機器遺忘缺口:當刪除權無法觸達AI模型權重時的問責

2026-06-14 6 分鐘閱讀

刪除權(被遺忘權)是資料保護法的核心權利。對於靜態記錄,機制簡單明確:定位記錄、刪除、確認。對於已在該資料上訓練的AI模型,這一機制在任何已驗證的生產系統中均不存在。患者可以行使法定權利要求刪除其健康記錄,而使用這些記錄訓練出的醫療AI系統將繼續運行,其權重中仍編碼著來自已不再合法存在的資料的壓縮統計影響。沒有人能夠證明這種影響已被移除,因為用於驗證移除的工具從未被內置到任何已部署的系統中。

透過梯度下降訓練AI模型,訓練資料集的統計規律被編碼至模型的權重矩陣中。這些權重並非單個資料點的記錄,而是所有訓練樣本的分散式、高維壓縮——每個人的資料同時疊加編碼於其中,無法按患者進行手術式分解。權重矩陣中沒有任何位元「屬於」某一特定患者。其資料的影響被擴散至數百萬乃至數十億個參數中,與其他所有訓練樣本的影響相互糾纏。刪除源記錄,影響依然存在。

機器學習研究界已開發出旨在移除特定訓練樣本影響的技術,而無需完全重新訓練模型。這些「機器遺忘」方法從從頭完整重新訓練(對大型已部署模型成本高昂)到近似遺忘方法(計算開銷較低,但僅能提供概率性而非可驗證的保證)不等。目前沒有任何已部署的醫療AI系統向患者提供驗證遺忘已完成的機制,也無法確認模型的行為與從未在其資料上訓練過的模型相同。權利存在,執行機制不存在。

醫療AI場景使這一缺口尤為嚴峻。縱向患者記錄之所以對AI訓練有價值,正是因為它們記錄了罕見事件序列、微妙的惡化規律和跨時間的藥物交互作用。訓練價值最高——因此最難從模型權重中分離——的資料,往往也是患者最有動力刪除的資料:過去的精神危機記錄、污名化的診斷、依賴症期間的記錄。這些不是統計貢獻微小且均勻擴散的普通訓練樣本,而是被模型賦予高權重的異常事件,因為它們是預測未來風險最具價值的信號。資料的臨床意義越大,移除其影響就越困難。

後量子維度疊加了第二層問題。針對機器遺忘問題,有人提出在加密保護的資料上訓練:若源資料使用隨後銷毀的金鑰加密,即使原始儲存介質仍然存在,訓練輸入也無法訪問。這一方法存在量子轉型使之惡化的結構性局限。在大多數已部署系統中,在醫療AI訓練資料規模上實用的同態加密在計算上仍不可行。而今天以經典加密方案對訓練資料集加密、透過金鑰銷毀實現「刪除」的做法,面臨「現在收割,以後解密」的暴露窗口:對手現在收集加密訓練資料集,可能在監管保留期內隨量子密碼分析能力成熟而恢復底層資料。透過金鑰銷毀實現刪除,需要對加密方案的長期安全性有信心——這種信心正被量子轉型系統性地侵蝕。

聯邦學習部署在硬體層面呈現出同樣的問題。聯邦學習將梯度更新路由至邊緣硬體——患者自有設備、醫院本地計算節點——而非將原始患者資料傳輸至中央伺服器。當患者行使刪除權時,邊緣設備上儲存的原始資料可以刪除,但該設備在該資料上計算的梯度更新,已傳輸並彙總至中央模型,無法撤銷。硬體證明方案可以驗證邊緣設備運行了正確的計算,但無法驗證該計算是可逆的。使聯邦學習對源資料保護更充分的隱私架構,反而使機器遺忘更難實施,因為需要撤銷的梯度是在專為保護其不被檢查的協議下計算的。

在當前技術限制範圍內,有三種應對措施可用,但均不完整。第一是誠實揭露:任何醫療AI部署都應在患者同意時明確告知,行使刪除權將移除源記錄,但目前無法提供可驗證的訓練影響移除保證。這並非令人舒適的揭露,但這是準確的。第二是源頭資料最小化:降低訓練資料的粒度和保留窗口,可減少任何個人對模型的統計貢獻,使近似遺忘更具可行性,並縮小缺口幅度。第三是差分隱私訓練:以形式化差分隱私保證訓練的模型,限制任何單個訓練樣本對模型輸出的最大貢獻。這並不能解決刪除問題,但可以量化隱私損失——以可揭露、可稽核、並在模型迭代中可不斷降低的具體上界,取代不可驗證的缺口。

機器遺忘缺口不是資料保護法的失敗。刪除權的設計是好的,是為其創立時的資料架構設計的。缺口在於:那個架構——資料是可定位和銷毀的靜態檔案——與AI訓練創造的架構之間的落差:個人資料作為分散的統計影響,編碼在數十億模型參數中,在源記錄消失後長期存在。彌合這一缺口,需要可在醫療AI規模上實際部署的可驗證遺忘演算法的進展,或向每位患者誠實揭露系統保留著其無法移除的影響。前者是工程問題,後者是問責問題。兩者都尚未被視為緊迫,但兩者都是。