Authored By: Ludovica Corsello
Trinity College Dublin
I. Introduction
In December 2024, Italy’s data protection authority fined OpenAI 15 million euros, finding that ChatGPT had been trained on the personal data of millions of Europeans without an adequate legal justification.1 The decision addressed how the data was collected, but it did not resolve a harder question that the same facts raise: once personal data has shaped a trained model’s parameters, can that data ever actually be erased? The right to erasure under Article 17 of the General Data Protection Regulation was written for a world of discrete, locatable records.2 Generative AI models store no such records; personal data is diffused across billions of statistical weights, and removing its influence typically requires retraining the model from scratch. This article argues that Article 17, as currently applied, cannot be reliably enforced against trained generative AI models. It further argues that regulators have so far sidestepped this problem rather than solved it, by focusing enforcement on unlawful collection rather than on erasure itself.3
II. Legal Framework and Enforcement Practice
Article 17 GDPR grants data subjects the right to obtain erasure of their personal data “without undue delay,” subject to limited exceptions.4 The provision assumes that personal data can be identified and isolated within a controller’s systems. This assumption holds for conventional databases, where a specific record can be located and deleted. It does not hold for a trained neural network, which does not store personal data as separate entries but instead embeds it into a collective result. Technically, extracting the influence of a single individual’s data without retraining the entire model remains, as researchers in the field describe it, one of the hardest open problems in machine learning.5
Regulators have not yet confronted this problem directly. Instead, enforcement to date has targeted the lawfulness of the initial collection. The Spanish data protection authority fined Google 10 million euros in 2022 for violations of Articles 6 and 17 GDPR, but the erasure finding concerned Google’s failure to honor individual deletion requests within its conventional search index, not any AI training pipeline.6 The Italian Garante’s action against OpenAI similarly rested on Article 6: the absence of a lawful basis for processing personal data during training, together with transparency and age-verification failures.7 Neither decision required OpenAI or Google to demonstrate that personal data could actually be removed from a trained model’s parameters. The European Data Protection Board has acknowledged that AI developers can qualify as data controllers under the GDPR, but has not set out how the erasure obligation itself is to be discharged once training is complete.8
A body of technical research, known as machine unlearning, has emerged in response, seeking methods to remove the influence of a data point from a trained model without full retraining.9 These methods remain experimental. There is no accepted standard for verifying that unlearning has actually occurred, and no consensus on what counts as successful erasure in a system that stores information probabilistically rather than as fixed records.
III. Critical Analysis and Argument
The pattern across enforcement to date is avoidance rather than resolution. Regulators have repeatedly found violations at the point of collection, where the GDPR’s traditional tools apply comfortably, while leaving open the question of whether erasure is achievable once training has occurred. This is a defensible short-term enforcement strategy, but it is not a solution to the underlying problem, and it leaves the central question unanswered indefinitely: what happens when a data subject’s objection arises only after a model has already been trained and deployed?
It is submitted that two consequences follow. First, regulators should treat the difficulty of post-training erasure as grounds for intensifying scrutiny at the collection stage, since this is the only point in the pipeline where compliance can currently be verified with confidence. Consent and legal-basis requirements under Article 6 should be enforced not merely as a formality but as the practical substitute for a right to erasure that cannot, at present, be reliably exercised once training has occurred.10 Second, machine unlearning techniques should not be treated as a presumptive compliance solution unless a developer can independently demonstrate that a specific individual’s influence has been removed to a verifiable standard, rather than merely asserting that unlearning has been “applied.” Absent such verification, a claim of unlearning is not meaningfully different from no remedy at all.
This is not an argument that generative AI cannot be reconciled with data protection law. It is an argument that the reconciliation currently being enforced is narrower than it appears: regulators are policing how models are built, not whether the rights the GDPR promises after the fact can actually be delivered.
IV. Conclusion
The right to erasure was designed for a data environment far simpler than the one generative AI now occupies. Enforcement to date, in Italy and Spain alike, has secured real accountability for unlawful data collection, but has not tested or resolved whether personal data can actually be removed from a model once training is complete. This article has argued that this gap is not incidental; it reflects a genuine technical limit that current unlearning methods have yet to overcome. Until verifiable unlearning becomes achievable, regulators should treat lawful collection as the primary safeguard, and should decline to accept unverified claims of unlearning as satisfying Article 17.11 The alternative, an erasure right that exists on paper but cannot be exercised in practice, does little to protect the autonomy interest the provision was designed to serve.
Footnote(S):
1 Garante per la Protezione dei Dati Personali, Provvedimento del 20 dicembre 2024 [2024] doc web n 10085432
2 Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 (General Data Protection Regulation) [2016] OJ L119/1, art 17
3 Garante (n 1)
4 ibid
5 Anni Huang, Zhen Cai and Zhitao Xiong, ‘A Survey of Machine Unlearning in Generative AI Models: Methods, Applications, Security, and Challenges’ (2025) 12(16) IEEE Internet of Things Journal 32563
6 Agencia Española de Protección de Datos, Decision PS/00337/2021 (Spanish Data Protection Agency, 2022); GDPR (n 2) art 6 and art 17
7 Garante (n 3)
8 European Data Protection Board, Opinion 28/2024 on Certain Data Protection Aspects Related to the Processing of Personal Data in the Context of AI Models (EDPB, 2024)
9 Lucas Bourtoule and others, ‘Machine Unlearning’ (2021) IEEE Symposium on Security and Privacy 141
10 GDPR (n 6) art 6
11 GDPR (n 6) art 17
Bibliography
Table of Legislation
Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 (General Data Protection Regulation) [2016] OJ L119/1
Regulatory Decisions
Agencia Española de Protección de Datos, Decision PS/00337/2021 (Spanish Data Protection Agency, 2022)
European Data Protection Board, Opinion 28/2024 on Certain Data Protection Aspects Related to the Processing of Personal Data in the Context of AI Models (EDPB, 2024)
Garante per la Protezione dei Dati Personali, Provvedimento del 20 dicembre 2024 [2024] doc web n 10085432
Secondary Sources
Bourtoule L and others, ‘Machine Unlearning’ (2021) IEEE Symposium on Security and Privacy 141
Huang A, Cai Z and Xiong Z, ‘A Survey of Machine Unlearning in Generative AI Models: Methods, Applications, Security, and Challenges’ (2025) 12(16) IEEE Internet of Things Journal 32563
