Authored By: Iswaree Kharug
Middlesex University Mauritius
Introduction
The administration of justice has long been rooted in the principles of fairness, transparency, reasoned adjudication and human accountability. Courts are not merely institutions of dispute resolutions, they are constitutional guardians of the rule of law. They are entrusted with ensuring that legal outcomes are reached through impartiality and intelligible reasoning. In recent years, however, the emergence and integration of artificial intelligence into legal and judicial systems have begun to challenge the traditional contours of adjudication.[1]
AI technologies are increasingly being employed across the justice sector in a variety of capacities, including legal research, document review, evidence management, predictive analytics, and case administration. More recently, the rise of generative AI systems capable of producing text, images, audio, and video has expanded the role of technology beyond administrative support into areas that directly intersect with judicial assessment and evidentiary evaluation. Simultaneously, algorithmic and predictive tools are being used to assist in risk assessments, sentencing recommendations, and outcome forecasting, thereby raising profound questions concerning the legitimacy of machine-assisted legal reasoning.[2]
This article argues that although AI has the potential to significantly enhance the administration of justice, its integration into judicial processes must remain subject to strict rule-of-law safeguards. In particular, judicial institutions must adopt robust accountability mechanisms, transparency requirements, and human-in-the-loop oversight to ensure that technological innovation does not erode the fundamental rights of access to justice and fair trial guarantees[3]. The article first examines the evidentiary challenges posed by generative AI before turning to issues of algorithmic bias, predictive modelling, and regulatory responses.
Evidentiary Challenges
One of the most immediate legal challenges posed by the rise of AI within judicial systems concerns the evidentiary implications of generative technologies. Unlike earlier forms of legal technology that primarily assisted with administrative functions, generative AI possesses the capacity to produce substantive content in the form of text, images, audio recordings, and video material.[4] This development has profound implications for judicial proceedings, particularly in relation to the assessment of evidence.
Where evidence is generated, altered, or assisted by AI systems, judges must engage in a more rigorous examination of its authenticity[5] and authorship. The question is no longer limited to whether evidence is relevant or admissible but extends to whether it genuinely reflects the facts it purports to establish.[6]
Similarly, AI-generated legal text presents its own challenges. If generative systems are used in the preparation of witness statements, legal submissions, or documentary evidence, courts must remain alert to issues of authorship and reliability. A document produced by AI may appear coherent and authoritative while containing inaccuracies, fabricated references, or misleading legal analysis. This risk is particularly serious where judicial actors or legal practitioners place undue reliance on machine-generated outputs without adequate verification.[7]
The judicial response to such developments must therefore be grounded in caution. Judges should not merely accept AI-generated material at face value but must actively interrogate its origin, method of generation, and evidentiary integrity.
The Rule of Law
The incorporation of algorithmic systems into judicial and quasi-judicial processes raises concerns that extend beyond efficiency and into the constitutional foundations of the legal order. At its core, algorithmic decision-making operates through rules embedded within technological systems: coded instructions, decision trees, statistical weightings, and machine-learning models that generate outputs on the basis of predefined logic and historical data. While such systems may offer consistency and speed, the legal question is whether the rules embedded within them are themselves lawful, intelligible, and free from discriminatory assumptions.
The rule of law requires that legal reasoning be transparent and that parties understand the basis upon which a determination affecting their rights has been reached. Where an algorithm materially influences a judicial outcome, the opacity of the system may undermine this requirement.[8]
The problem of algorithmic opacity, often referred to as the “black box” phenomenon, arises where neither the decision-maker nor the affected parties can fully understand the internal reasoning process of the system. This lack of intelligibility creates a tension with one of the most fundamental principles of adjudication: the duty to give reasons. If a litigant cannot ascertain how an algorithm arrived at a risk score or predictive recommendation, their ability to challenge the fairness of the process is materially diminished. In such circumstances, procedural fairness[9] is not merely weakened but potentially compromised.[10]
Accordingly, the use of algorithmic tools within the justice system must remain subordinate to human adjudicative authority. Judicial reliance on AI should be accompanied by rigorous scrutiny of the underlying rules, datasets, and assumptions upon which the system operates. Where transparency cannot be adequately ensured, the legitimacy of its use within adjudicative contexts becomes deeply questionable.[11]
Ultimately, the rule of law requires more than technological competence; it requires accountability, explainability, and fairness.[12] Any algorithmic system used within the justice sector must therefore be evaluated not merely for operational efficiency, but for its compatibility with constitutional values and human rights guarantees.
“Forecasts, Not Facts”
At a foundational level, data bias arises where datasets are incomplete, unrepresentative, or reflective of historically unequal social structures. In judicial contexts, such bias may manifest where datasets disproportionately capture certain populations, omit relevant contextual variables, or replicate patterns of systemic inequality. For example, if historical criminal justice data reflects over-policing of particular communities or disparities in prosecutorial decision-making, any predictive system trained on such data risks perpetuating those same patterns. The result is not merely a technical flaw but a potential distortion of justice itself. Such outputs are inherently probabilistic in nature. They do not establish what will occur, but rather what may occur based on past trends.
It is therefore essential to recognise a critical distinction: predictive AI produces forecasts, not facts. This distinction is not merely semantic but jurisprudential. Legal adjudication requires the determination of facts, the application of legal principles, and the exercise of reasoned judgment. By contrast, predictive systems generate probabilistic assessments that are contingent upon data inputs, modelling assumptions, and algorithmic design. To treat such outputs as determinative or authoritative risks conflating statistical inference with legal truth.
Where an autonomous system contributes to or influences a judicial outcome, it becomes increasingly difficult to identify who bears responsibility for any resulting error or injustice. The diffusion of responsibility between developers, institutions, and judicial actors risks creating accountability gaps that are incompatible with the rule of law.
Ultimately, the integration of predictive AI into the justice system demands a reaffirmation of core legal principles. Judicial decision-making must remain anchored in reasoned analysis, evidentiary evaluation, and individualised justice. While predictive tools may offer valuable insights, they must not displace the judicial function nor be treated as substitutes for legal judgment.
Accountability, Human Oversight, and the Human-In-The-Loop Principle
The integration of artificial intelligence into judicial processes necessitates not only critical scrutiny but also the development of robust accountability frameworks capable of preserving the integrity of the legal system. There is a requirement of a principled legal response grounded in the fundamental values of the rule of law, including transparency, accountability and fairness.[13]
At the centre of this response lies the concept of human oversight, often articulated through the principle of the human-in-the-loop (HITL). This principle requires that human decision-makers retain ultimate authority over any process in which AI is utilised, particularly where legal rights and obligations are at stake. In judicial contexts, this entails that AI systems must function strictly as assisting tools, supporting but never replacing judicial reasoning.[14] The final determination must remain the product of a human judge, capable of exercising discretion, interpreting legal principles, and engaging in moral and contextual evaluation.
The importance of maintaining meaningful human control is underscored by the limitations inherent in AI systems. Human oversight therefore serves as a critical safeguard, ensuring that such outputs are subject to interrogation, contextualisation, and, where necessary, rejection. Without this layer of scrutiny, there is a risk that judicial actors may defer to technological outputs in a manner that diminishes independent reasoning and undermines judicial responsibility.
Closely linked to the principle of human oversight is the requirement for clear accountability mechanisms. One of the most pressing challenges posed by AI integration is the potential diffusion of responsibility across multiple actors, including software developers, data providers, judicial institutions, and individual decision-makers. In the absence of clearly defined lines of accountability, it becomes difficult to attribute responsibility where errors or injustices arise. This is particularly problematic within the justice system, where accountability is not merely a procedural requirement but a constitutional necessity.
The development of AI governance instruments, including ethical guidelines, judicial protocols, and institutional policies, further reinforces the responsible use of technology within the justice system. Such instruments can provide practical guidance on issues such as permissible uses of AI, standards for evidentiary reliability, and safeguards against automation bias. Importantly, they also signal a broader commitment to aligning technological innovation with legal and ethical principles.
Conclusion
The integration of artificial intelligence into judicial systems represents a profound transformation in the administration of justice. While AI offers significant benefits, it also raises fundamental concerns relating to transparency, accountability, and fairness.
This article has argued that AI must not be treated as a neutral arbiter. Predictive systems generate forecasts rather than facts, and algorithmic outputs are shaped by data and design choices. Without appropriate safeguards, their use risks undermining equality before the law and fair trial rights.
The preservation of the rule of law requires that human oversight, accountability, and transparency remain central to judicial processes. The human-in-the-loop principle is therefore not merely a technical safeguard but a constitutional necessity.
Ultimately, justice must remain a human endeavour. Artificial intelligence may assist the judge, but it must never become the silent judge.
Bibliography
Primary Sources
Goldberg v Kelly (1969) 397 U.S. 254
Porter v Magill [2001] UKHL 67
R (Osborn) v Parole Board [2013] UKSC 61
R v Turnbull [1977] QB 224
State v Loomis (2016) 371 Wis.2d 235
Secondary Sources
Citron D.K, Deepfakes and the New Disinformation War (2019)
European Commission for the Efficiency of Justice, European Ethical Charter on the use of Artificial Intelligence in Judicial Systems and their Environment (2018)
European Commission for the Efficiency of Justice, European Ethical Charter on the use of Artificial
European Commission, Ethics Guidelines for Trustworthy AI (2019)
National Institute of Standards and Technology, AI Risk Management Framework (2023)
OECD AI Principles (2019)
UK Law Commission, Digital Evidence in Criminal Proceedings (2023)
Wachter S, Brent Mittelstadt and Luciano Floridi, ‘Why a Right to Explanation of Automated Decision-Making Does Not Exist’ (2017)
World Economic Forum, AI in Courts: Opportunities and Risks (2021)
[1] European Commission for the Efficiency of Justice, European Ethical Charter on the use of Artificial Intelligence in Judicial Systems and their Environment (2018).
[2] World Economic Forum, AI in Courts: Opportunities and Risks (2021).
[3] R (Osborn) v Parole Board [2013] UKSC 61.
[4] National Institute of Standards and Technology, AI Risk Management Framework (2023).
[5] R v Turnbull [1977] QB 224.
[6] UK Law Commission, Digital Evidence in Criminal Proceedings (2023).
[7] Danielle Keats Citron, Deepfakes and the New Disinformation War (2019), 150.
[8] Sandra Wachter, Brent Mittelstadt and Luciano Floridi, ‘Why a Right to Explanation of Automated Decision-Making Does Not Exist’ (2017).
[9] Goldberg v Kelly (1969) 397 U.S. 254.
[10] State v Loomis (2016) 371 Wis.2d 235.
[11] European Commission for the Efficiency of Justice, European Ethical Charter on the use of Artificial Intelligence in Judicial Systems and their Environment (2018).
[12] Porter v Magill [2001] UKHL 67.
[13] European Commission, Ethics Guidelines for Trustworthy AI (2019).
[14] OECD AI Principles (2019).
