Algorithmic Bias and the Right to Equality: Reconciling Automated Decision -Making with POPIA and the South African Constitution

Algorithmic Bias and the Right to Equality: Reconciling Automated Decision -Making with POPIA and the South African Constitution

The promise of equality in the constitution is directly challenged by the growing use of automated decision-making systems in South Africa. Decisions with significant legal ramifications are now made without human intervention, from banks using algorithms to evaluate creditworthiness to employers using AI to screen job applicants. Although the Protection of Personal Information Act 4 of 2013 aims to control how personal data is processed, including automated decisions, it does so in a way that is inadequate to mitigate the possibility of algorithmic bias. This is especially troublesome in a society such as South Africa, where historical patterns of discrimination remain embedded in data.

This article contends that because POPIA does not sufficiently address algorithmic bias, it does not fully implement the right to equality in section 9 of the Constitution. Automated systems run the risk of solidifying systemic inequality in the absence of required audits and transparency mechanisms. Part II delineates the pertinent legal framework. The third section looks at algorithmic bias in South Africa. In order to bring POPIA into compliance with international and constitutional norms, Part IV suggests certain legislative changes.

The Legal Framework: POPIA, the Constitution and Automated Decisions.

A data subject has the right, under Section 71 of POPIA, to be exempt from decisions that are based only on automated processing and have legal ramifications for them. However, there are three major exceptions to this right: when the decision is made in accordance with a contract, a law, or the consent of the data subject. The majority of businesses actually rely on the contractual exception, which makes the protection essentially non-existent.

POPIA’s Section 11 mandates that personal data be handled legally, equitably, and without violating anyone’s right to privacy. However, discriminatory results are not included in the definition of “fairness.”. Section 9 of the Constitution must be read alongside this. Everyone is guaranteed equality before the law by Section 9(1). Measures to advance those who are disadvantaged by unfair discrimination are permitted under Section 9(2). Unfair discrimination by the state and private individuals is forbidden by Section 9(3). According to the Constitutional Court, eliminating systemic disadvantage is the goal of equality as a substantive concept. This goal is directly compromised by automated systems that replicate historical bias.

Additionally, the Promotion of Administrative Justice Act 3 of 2000 mandates administrative actions and justifications that are procedurally fair. Automated decisions with comparable outcomes should follow the same logic.

Algorithmic Bias in the South African Context

When an algorithm generates consistently biased results because of faulty training data or design, this is known as algorithmic bias. This risk is particularly severe in South Africa since datasets frequently show disparities in income, location, and service access from the apartheid era. Examples have already surfaced. Concerns regarding automated credit scoring were highlighted in a section 71 guidance note released by the Information Regulator in 2023. Research has revealed that black applicants from townships are disproportionately rejected by credit algorithms trained on South African data due to proxy variables like postal code rather than individual creditworthiness.

Examples are starting to appear. The Information Regulator raised issues with automated credit scoring in a section 71 guidance note published in 2023. Studies have shown that credit algorithms trained on South African data disproportionately reject black applicants from townships, not because of individual creditworthiness, but because of proxy variables like postal code.

Predictive policing, insurance pricing, and HR recruitment tools all have similar problems. The issue is made worse by a lack of openness. A person may object to an automated decision under s71, but there is no legal right to know “how” the decision was made. Determining whether discrimination has taken place is impossible without explainability, let alone contesting it.

POPIA’s framework is insufficient for three reasons.

First, s71(2) has too many exceptions. A data subject waives their rights by merely clicking “I agree,” according to the contractual exception. This goes against the Constitution’s guarantee of substantive equality.

Second, rather than outcomes, POPIA emphasizes procedural compliance. A business may be in compliance with POPIA even if its algorithm yields unfair outcomes. Bias is not subject to the “fairness” requirement of Section 11.

Thirdly, no accountability system is in place. In contrast to the EU AI Act and GDPR Article 22, POPIA does not mandate algorithmic impact assessments or offer a right to explanation. POPIA should be modified, it is argued. Three reforms are put forth:

1. The right to a meaningful explanation of the reasoning behind automated decisions should be guaranteed to data subjects.
2. Mandatory Algorithmic Impact Assessments: Before implementing high-risk AI, businesses should be obliged to audit systems for bias and submit reports to the Information Regulator.
3. * Expanded Regulator Powers*: The Information Regulator ought to have the authority to carry out audits and mandate that biased systems.

These reforms would align POPIA with section 9 and international best practice.

In South Africa, automated decision-making has major advantages for productivity and service provision. But in the absence of appropriate regulation, it also runs the risk of solidifying the very injustices that the Constitution aims to eliminate. Section 71 in its current form is insufficient, even though POPIA is a crucial first step. POPIA does not fully implement section 9 due to the wide exceptions, absence of outcome-based standards, and lack of a right to explanation.

Parliament should amend POPIA to include mandatory impact assessments, a right to explanation, and more authority for the Information Regulator in order to guarantee that technology upholds constitutional values. South Africa can only benefit from AI without compromising its commitment to equality and human dignity if these safeguards are in place.

Reference(S): 

1. Constitution of the Republic of South Africa, 1996
2. Protection of Personal Information Act 4 of 2013
3. Information Regulator, _Guidance Note on Automated Decision-Making in terms of Section 71 of POPIA_ (2023)
4. T Reddy, ‘Algorithmic Bias and Substantive Equality in South Africa’ (2023) 140 _SALJ_ 45
5. Regulation (EU) 2024/1689 on Artificial Intelligence [EU AI Act]