Authored By: LAYLA KAITLYN BUSSACK
University of South Africa
We trust algorithms to make decisions precisely because they are not human. A machine does not have emotions, prejudices or personal interests; it simply follows the code it has been given. At least, that is what we assume. Yet behind every seemingly objective algorithm lies human-written code, human-selected data and human-made decisions about what that code should do. When an algorithm makes a decision that affects a person’s rights, livelihood or freedom, a difficult legal question emerges: how do you prove what a machine actually did?
For instance, let us say that, in an application for a job interview, the algorithm used to identify the best candidate does not acknowledge or consider women. Who, then, is at fault for the obviously discriminatory practice: the company or the algorithm? In the world we now inhabit, interviewers may be artificial intelligence, the evidence needed to prove a crime is electronic, and decisions are rooted in, or assisted by, artificial intelligence. Where, then, is our protection, and are we adequately protected under the current framework?
This article argues that South African evidentiary law provides a foundation, or starting place, for admitting algorithmically generated evidence but does not yet adequately address the evidentiary challenges created by complex algorithms, particularly the need to establish the reliability, provenance and reproducibility of the computational process producing the evidence.
South African Legal Frameworks
Legal Recognition of Electronic Information
The Electronic Communications and Transactions Act1 (hereafter referred to as the “ECT Act”) is the legislation that gives legal recognition to electronic information and electronic communications in South Africa. The ECT Act2 defines “data” as an electronic representation of information in any form and further defines a “data message” as data generated, sent, received or stored by electronic means. Section 3 of the ECT Act3 provides for the interpretation of the Act in a manner that promotes its objects, including facilitating electronic communications and transactions. Most importantly, section 11 states that information may not be denied legal recognition merely because it is contained in a data message.4
The definition of a “data message” expressly includes data that is generated electronically. An algorithmic decision-making system inherently generates data when it processes information and produces an output or decision. As a result, an algorithmically generated output can fall within the concept of a data message and is therefore capable of receiving legal recognition under section 11 of the ECT Act.5 Although the ECT Act predates the widespread use of modern algorithmic decision-making, its technology-neutral approach to electronic information allows its provisions to extend to electronically generated information of this nature. The Act therefore provides an important starting point for recognising algorithmically generated information within the South African legal framework.6
Admissibility of Electronic Evidence
Section 15(1)(a) of the ECT Act7 provides that a data message cannot be denied admissibility in legal proceedings merely because it is in the form of a data message. Furthermore, section 15(2) states that information in the form of a data message must be given due evidential weight.8 Section 15(3) then provides the factors that must be considered when determining the evidential weight of a data message, including the reliability of the manner in which it was generated, stored or communicated, the reliability of the manner in which its integrity was maintained, the manner in which its originator was identified, and any other relevant factor.9
These provisions are particularly significant when considering algorithmic evidence. The issue may therefore not be whether an algorithmically generated output is capable of being admitted as electronic evidence, but whether the process through which that output was generated can be shown to be reliable. Sections 16 and 17 of the ECT Act10 further emphasise the importance of accessibility, integrity and the ability to accurately retain and reproduce electronic information. Section 16 requires that, where information is required to be retained, it must remain accessible for subsequent reference, be retained in its original or accurately reproducible format, and allow the origin and destination of the data message, as well as the date and time of its transmission, to be determined.11 Section 17 similarly requires that, where a document or information must be produced, the electronic form must be generated in a manner that provides a reliable means of assuring the integrity of the information and allows it to remain accessible for subsequent reference.12
Section 25 of the ECT Act13 is particularly relevant to algorithmic decision-making because it addresses the connection of a data message to its originator. The section provides that a data message may be attributed to the originator where it was sent by the originator, by a person authorised to act on the originator’s behalf, or by an information system programmed by or on behalf of the originator to operate automatically, unless it is proved that the information system did not properly execute that programming.14 This is significant in the context of algorithmic systems because the output may not have been manually created or reviewed by a human being.
The difficulty with algorithmic evidence therefore lies beyond the simple question of whether electronic information is recognised by law. The more difficult question is whether the party relying on the algorithm can demonstrate that the algorithm operated reliably, that the information used to generate its output was maintained with integrity, that the output can be attributed to the relevant system or party, and that the process through which the result was produced can be sufficiently established before a court. In other words, admissibility may open the door to algorithmic evidence, but reliability determines how much weight that evidence should ultimately carry.
POPIA and Automated Decision-Making
The Protection of Personal Information Act15 (hereafter referred to as “POPIA”) provides an additional layer to the regulation of algorithmic decision-making in South Africa. Section 71 of POPIA16 recognises that automated processing can produce decisions with significant consequences for individuals. Specific exceptions aside, a data subject generally may not be subjected to a decision producing legal consequences or substantially affecting them where the decision is based solely on automated processing of personal information intended to create a profile of that person.17
Of particular relevance to the evidentiary problem explored in this article, section 71(3) requires appropriate measures to protect the legitimate interests of the data subject. These measures must include an opportunity for the data subject to make representations and must require the responsible party to provide “sufficient information about the underlying logic” of the automated processing to enable those representations.18 Thus, POPIA recognises that an individual affected by an automated decision may require some understanding of the process by which that decision was reached. The question is therefore not simply whether the output exists in electronic form, but whether the process that produced it can be sufficiently explained and tested.
Case Law Analysis
South African courts have already dealt with the question of how evidence produced or processed by computers should be treated. The developing jurisprudence demonstrates that the crucial concern is not just whether information was produced electronically, but whether the reliability and integrity of the process producing that information can be established and verified.
In S v Ndiki and Others19, the court considered computer-generated printouts and distinguished information generated through human action from information independently processed by a computer. Where a computer processed existing information, performed calculations and generated additional information without human action, the resulting evidence could be considered real evidence. Thus, admissibility depended on the accuracy and reliability of the computer, its operating system and its processes, rather than on the credibility of a human witness.20 The case therefore places the reliability of the technological process itself at the centre of the evidentiary enquiry.
This principle was tested and developed further in Firstrand Bank Ltd t/a Futurefin Finance, a Division of Wesbank v Class A Trading (Pty) Ltd and Another21. The bank sought to rely on a computer-generated statement of account on the basis that this was how its operations normally occurred. The court rejected this approach, as it found that the witness lacked sufficient personal knowledge to establish how the system operated, how information entered the system and how the statement was actually generated.22 The court thus stated and emphasised that the ECT Act is “not a cure-all, nor is it a short-cut” and that a party relying upon it must provide sufficient evidence to enable the court to determine the nature of the data and how the relevant data system operates.23 In the absence of that evidence, the court cannot rely on the information produced by the system.
The recent decision in S v Mohlapi demonstrates the continued relevance of these principles. In this case, the Gauteng High Court considered automatically generated cellular records and accepted them as real evidence where their reliability, integrity and method of production had been sufficiently established. The court relied on Ndiki24 and emphasised that the process by which the information was captured had been explained through competent testimony.25 Thus, this case confirms that computer-generated evidence does not become inherently unreliable merely because it is produced without direct human action, but there needs to be sufficient evidence for the court to trust the process by which the evidence was generated.
Upon examination of these authorities, the principle that emerges is that electronic evidence is neither inherently reliable nor inherently unreliable simply because it was generated electronically. Rather, the party wishing to use such evidence must provide sufficient proof of the reliability and integrity of the process of generation.
Critical Evaluation
Upon examination of the emerging South African case law, the courts seemingly take a relatively straightforward approach when assessing computer-generated evidence. The focus of the court is to investigate how the information was captured, processed and produced, and the party relying on the evidence can call a witness capable of explaining the relevant system. The difficulty becomes more pronounced where the evidence is generated by a complex algorithmic system.
An algorithmic decision or decision-making process cannot necessarily be understood merely by examining the final result. The reliability of the final result may depend upon a variety of issues, such as the data supplied to the system, the algorithm or model used, the parameters applied, the version of the software, the environment in which it operated and the manner in which the output was recorded. Hence, proving that the final result is reliable may require more than stating or demonstrating that the computer produced it; the court may need to be satisfied that the entire computational process leading to the final result was sufficiently reliable.
This is where the matter becomes dangerous and complicated: even where there is access to the algorithm, the process the algorithm follows and its default settings may not be scrutinised in a meaningful way. This is particularly problematic where complex machine-learning systems are used, as such a system may identify patterns in large datasets and produce a final result without providing an explanation capable of being translated into legal reasoning.
If a party cannot simply tell a court that “the system produced it”, what information is needed from that party when the system itself is exceptionally complex? Gaining access to the source code seems a logical answer, but what if the underlying system configuration is faulty or biased? Who is tasked with investigating where the fault lies, or why the system has produced those results: the system creator, the processor or the user?
A further difficulty is reproducibility. Evidence produced electronically may not be capable of being recreated even when the same inputs and system are used, as an algorithmic system may be updated or altered. Thus, if the version of the system that produced the original output is not preserved, a later attempt to reproduce the decision may produce a different result. The evidentiary question therefore becomes temporal: it is not merely necessary to establish what the algorithm is, but what the algorithm did at the relevant time.
This is problematic because evidentiary reliability must now be balanced against competing interests, such as the intellectual property of those who produce the algorithms in question. Expert evidence therefore becomes increasingly important: someone must be able to explain the logic behind the algorithm so that the court is in a position to assess the integrity and reliability of the evidence.
Conclusion
The central difficulty is therefore not that algorithms are inherently unreliable, but that the legal system must determine what evidence is sufficient to establish their reliability. South African law already requires courts to examine the integrity and operation of computer systems. The emerging challenge is determining how those principles should operate when the system producing the evidence is complex, dynamic or insufficiently explainable. Until that question is answered, parties relying on algorithmic evidence will need to show not only what the output was, but how, when and on what data it was produced.
Footnote(S):
1 Act 25 of 2002
2 Ibid pg 13
3 Ibid.
4 Act 25 of 2002
5 ibid
6 Ibid.
7 Act 25 of 2002 pg 25
8 Ibid
9 Act 25 of 2002 pg 25
10 Ibid.
11 Act 25 of 2002 pg 25
12 Ibid.
13 Act 25 of 2002 pg 32
14 Ibid.
15 Act 4 of 2013
16 Ibid.
17 Protection of Personal Information Act 4 of 2013 s 71(1).
18 Protection of Personal Information Act 4 of 2013 s 71(3).
19 S v Ndiki and Others [2007] 2 All SA 185 (Ck) at 196–97; 2008 (2) SACR 252
20 Ibid.
21 Firstrand Bank Ltd t/a Futurefin Finance, a Division of Wesbank v Class A Trading (Pty) Ltd and Another (12837/2010) [2018] ZAGPJHC 135 paras 40–43.
22 Ibid.
23 Firstrand Bank Ltd t/a Futurefin Finance, a Division of Wesbank v Class A Trading (Pty) Ltd and Another (12837/2010) [2018] ZAGPJHC 135 paras 40–43.
24 Ibid 21.
25 S v Mohlapi (86/2019) [2019] ZAGPPHC 263 (19 June 2019)
