The Invisible Trial: Should Predictive Policing Algorithms Be Allowed to Influence Criminal Investigations?

Criminal Procedure & Civil Rights

A patrol car idles at the corner of a block the algorithm has flagged as “hot.” Nobody on that corner has committed a crime this evening, not yet, anyway, and maybe not ever. But a model trained on a decade of arrest records has decided that this particular intersection, at this particular hour, deserves extra attention. The officer does not know why, not really. He only knows the score. That is the quiet, largely invisible transformation now underway in American law enforcement, and it deserves far more scrutiny than it has received.

Predictive policing refers to the use of statistical and machine-learning models to forecast where crime is likely to occur (place-based systems, such as PredPol) or who is likely to be involved in it (person-based systems, such as Chicago’s Strategic Subject List).[1] Departments have embraced these tools for an understandable reason: they promise objectivity in a field long criticized for relying on hunches, stereotypes, and the discretion of individual officers.[2] Yet the promise and the reality have diverged sharply, and the divergence raises a question that neither Congress nor the Supreme Court has squarely answered: can the government act on a machine’s prediction of criminality without running afoul of the Fourth and Fourteenth Amendments?[3]

This article’s thesis is narrow but, I think, correct: predictive policing should be restricted not because prediction itself is illegitimate (courts have always tolerated forward-looking judgments about risk), but because these systems allow historical, often discriminatory, policing patterns to be laundered through mathematics and re-emerge looking neutral, objective, and therefore constitutionally unimpeachable. The number on the screen feels clean. It is not.

The Fourth Amendment’s Thinning Individualized-Suspicion Requirement

Fourth Amendment doctrine has long insisted that a stop be justified by particularized, individualized suspicion, a requirement that traces back to Terry v Ohio, where the Court permitted brief investigatory stops only upon “specific and articulable facts” pointing to a particular person.[4] Illinois v Wardlow later confirmed that presence in a “high crime area,” standing alone, cannot justify a stop; it is merely one relevant factor among several, and never a substitute for individualized indicia of wrongdoing.[5] The Court has repeatedly stressed that suspicion must attach to a person, not merely to a place or a demographic profile.[6]

Predictive place-based systems sit uneasily beside this doctrine. When an algorithm designates a two-block radius as a hot spot, officers patrolling that zone are, in effect, primed to treat everyone within it as somewhat more suspect than they would be elsewhere, precisely the reasoning Wardlow warned against, dressed up in the language of data science rather than officer intuition.[7] The algorithmic hot spot does not create individualized suspicion. It reallocates law enforcement attention toward a population and then generates, through sheer increased contact, the very arrests that will justify flagging the same area again next month.

This feedback dynamic is not speculative. A widely cited simulation of PredPol’s methodology using Oakland drug-crime data found that the model would have directed patrols overwhelmingly toward Black neighborhoods, far out of proportion to survey-based estimates of where drug use actually occurred, because the training data reflected where police had previously chosen to look, not where crime actually was.[8] Los Angeles eventually shelved both of its flagship programs, LASER and PredPol, after an internal audit and years of community pressure exposed exactly this problem.[9] Chicago’s experience was, if anything, more damning still: a RAND Corporation evaluation of the city’s Strategic Subject List found no statistically significant reduction in either offending or victimization among those flagged by the model.[10] As the RAND researcher who ran that evaluation later put it, the list simply didn’t work: it identified people, sent letters, made lists longer, and moved the needle on almost nothing.[11]

None of this necessarily means an algorithmic tip is legally worthless. Courts assessing reasonable suspicion look to the totality of circumstances, and an officer’s subjective reliance on a tool does not, by itself, void an otherwise objectively reasonable stop.[12] But that is exactly the danger. Because Fourth Amendment review is objective, a score generated by a black-box model can supply the missing ounce of “articulable” suspicion an officer’s gut instinct could not, on its own, satisfy, and no judge reviewing the stop after the fact can meaningfully interrogate how that score was produced.

Equal Protection and the Problem of Facial Neutrality

The Equal Protection Clause guarantees that no state shall deny any person equal protection of the laws.[13] But its central limitation, for present purposes, is doctrinal rather than textual: Washington v Davis held that a facially neutral government policy violates equal protection only where a plaintiff proves discriminatory purpose, not merely disparate impact.[14] McCleskey v Kemp then closed off the most obvious workaround, rejecting even a sophisticated statistical study documenting racial disparities in Georgia’s capital sentencing because it could not tie discriminatory intent to the decisionmakers in McCleskey’s own case.[15]

Predictive policing algorithms are the ultimate expression of facial neutrality. They typically exclude race as an input variable entirely. And that is precisely the problem, not despite Washington v Davis and McCleskey, but because of them. Variables like prior arrests, 911 calls received, or presence within a given patrol beat function as close statistical proxies for race in cities with long histories of disproportionate enforcement, allowing the model to reproduce racially skewed outcomes while never once mentioning race.[16] A plaintiff challenging such a system faces the near-impossible task McCleskey imposed: proving that a specific model, or the specific officer relying on it, acted with discriminatory purpose, when the entire appeal of these tools is that no human purpose enters the calculation at all.

Scholars have described this as bias entering “in” and coming out the other side simply as “risk.”[17] The math is real. The disparity is real. But the doctrinal hook that would let a court say so has been effectively engineered out of existence, not through malice on the part of any single officer or programmer, but through the ordinary operation of feeding historical enforcement data into a model built to find patterns in it.[18]

III. Loomis and the Sentencing Analog

The clearest judicial engagement with algorithmic risk scoring to date comes not from an investigative stop but from a sentencing hearing. In State v Loomis, the defendant challenged a circuit court’s reliance on a COMPAS risk assessment attached to his presentence report, arguing that its consideration violated his right to due process.[19] The Wisconsin Supreme Court upheld the practice, holding that a COMPAS score may inform, but not determine, a sentence, provided the presentence report carries specific warnings about the tool’s limitations, including its basis in group data and its proprietary methodology.[20]

Loomis’s strongest argument, and the one the court handled least persuasively, was that COMPAS’s trade-secret status made it impossible for him to test the tool’s accuracy or interrogate how it had weighed the factors in his own case.[21] The United States Supreme Court denied certiorari, leaving the question unresolved at the federal constitutional level.[22] Meanwhile, ProPublica’s independent analysis of COMPAS scores in Broward County, Florida found that the tool was roughly twice as likely to mislabel Black defendants as high risk who did not subsequently reoffend, and roughly twice as likely to mislabel white defendants as low risk who did.[23]

Sentencing and investigation are not identical contexts; due process and Fourth Amendment doctrine ask different questions, protect different interests, and trigger at different procedural moments. But Loomis is instructive precisely because it shows how a court, faced with a genuinely useful predictive tool and no clean doctrinal box to put it in, chose transparency-lite disclaimers over any real mechanism for testing the algorithm’s accuracy.[24] If that is the judiciary’s answer at sentencing, where the defendant is at least present and represented, it is hard to feel reassured about how the same instinct will play out on a street corner, where no lawyer is watching and no transcript is being made.

Toward a Workable Restriction

None of this requires condemning statistical prediction as such. Actuarial reasoning already pervades bail determinations, parole decisions, and insurance underwriting, and outlawing forecasting wholesale would be both impractical and, frankly, a bit precious; humans predict constantly; we just don’t usually dress our guesses up as neutral math. The more defensible position, and the one this article urges, is structural: predictive policing outputs should never by themselves supply reasonable suspicion or probable cause, should be subject to mandatory pre-deployment bias audits examining proxy variables for race and geography, and should be discoverable by defendants notwithstanding vendor trade-secret claims.[25]

Professors Crawford and Schultz have proposed a “procedural data due process” framework requiring notice of an algorithm’s use and a meaningful opportunity to contest both its inputs and its outputs, a model well suited to policing contexts, where the stakes of an erroneous flag include not just an unwanted encounter but, in the worst cases, an escalation with lethal potential.[26] Professor Ferguson has separately urged legislative solutions: algorithmic impact statements before deployment, sunset clauses requiring reauthorization, and independent audits comparable to those RAND ran in Chicago, audits that, notably,, only happened because RAND was invited in, not because any law required it.[27]

Citron and Pasquale call this broader phenomenon the “scored society”, a world in which opaque, proprietary algorithms increasingly allocate opportunity, liberty, and suspicion alike, all while remaining functionally unreviewable by the people they affect.[28] Criminal procedure has not yet caught up to that world, and honestly, it’s not clear our current doctrinal tools were built for it at all.

Conclusion

Return to that patrol car on the flagged corner. The officer inside it is not doing anything unconstitutional in the traditional sense: no one has stopped anyone, no one has been searched. But the allocation of police attention is itself a form of power, and when that allocation is driven by a model trained on decades of uneven enforcement, the model doesn’t correct the bias. It memorializes it, launders it, and hands it back with the illusion of scientific authority.

Predictive policing is not the villain of this story, exactly; the villain, if there is one, is our willingness to treat a statistical output as though it were evidence rather than an echo. Courts applying Terry, Wardlow, Washington v Davis, and McCleskey were never asked to imagine a tool that could reproduce discriminatory patterns without a single discriminatory thought ever passing through a human mind. That is precisely what these systems do, and it is exactly why the doctrine, as written, cannot catch them. Until legislatures or courts close that gap, the fairest conclusion is also the simplest: let the machines keep guessing, if we must, but stop letting their guesses stand in for judgment the Constitution has always demanded be human, individualized, and, when challenged, actually explainable.[29] [30]

Reference(S):

[1]Andrew Guthrie Ferguson, ‘Policing Predictive Policing’ (2017) 94 Washington University Law Review 1109, 1115-17.

[2]Elizabeth E Joh, ‘The New Surveillance Discretion: Automated Suspicion, Big Data, and Policing’ (2016) 10 Harvard Law and Policy Review 15, 20-22.

[3]Aaron Shapiro, ‘Predictive Policing for Reform? Indeterminacy and Intervention in Big Data Policing’ (2019) 37 Surveillance and Society 452, 454.

[4]Terry v Ohio 392 US 1, 21-22 (1968).

[5]Illinois v Wardlow 528 US 119, 123-25 (2000).

[6]United States v Cortez 449 US 411, 417-18 (1981) (requiring a ‘particularized’ and objective basis for suspecting wrongdoing).

[7]Andrew D Selbst, ‘Disparate Impact in Big Data Policing’ (2017) 52 Georgia Law Review 109, 137-41.

[8]Kristian Lum and William Isaac, ‘To Predict and Serve?’ (2016) 13 Significance 14, 16-18 (finding that PredPol’s Oakland simulation would have directed patrols disproportionately toward Black residents relative to estimated drug use rates).

[9]The Los Angeles Police Department discontinued both its LASER and PredPol programmes in 2019 and 2020 respectively, following an inspector general audit and sustained community opposition: Caroline Haskins, ‘Academics Confirm Major Predictive Policing Algorithm Is Fundamentally Flawed’ (Motherboard, 14 February 2019); ‘LAPD Predictive Policing Tool Raises Racial Bias Concerns’ (Government Technology, April 2020).

[10]Jessica Saunders, Priscillia Hunt and John S Hollywood, ‘Predictions Put into Practice: A Quasi-Experimental Evaluation of Chicago’s Predictive Policing Pilot’ (2016) 12 Journal of Experimental Criminology 347, 362-65 (finding no statistically significant reduction in victimization or offending among individuals on the Strategic Subject List).

[11]John S Hollywood, ‘CPD’s “Heat List” and the Dilemma of Predictive Policing’ (RAND Corporation, 19 September 2016).

[12]Whren v United States 517 US 806, 813 (1996) (holding that an officer’s subjective motivations are irrelevant to Fourth Amendment reasonableness so long as objective justification exists).

[13]US Constitution, Fourteenth Amendment, section 1.

[14]Washington v Davis 426 US 229, 239-42 (1976) (holding that a facially neutral policy violates equal protection only upon proof of discriminatory purpose, not merely disparate impact).

[15]McCleskey v Kemp 481 US 279, 292-99 (1987) (rejecting a statistical showing of racial disparity in capital sentencing absent proof that decision-makers in the petitioner’s own case acted with discriminatory intent).

[16]Solon Barocas and Andrew D Selbst, ‘Big Data’s Disparate Impact’ (2016) 104 California Law Review 671, 682-87.

[17]Bernard E Harcourt, Against Prediction: Profiling, Policing, and Punishing in an Actuarial Age (University of Chicago Press 2007) 145-49.

[18]Cathy O’Neil, Weapons of Math Destruction (Crown 2016) 84-91 (describing feedback loops in which biased enforcement data trains models that direct further biased enforcement).

[19]State v Loomis 881 NW 2d 749, 753-54 (Wis 2016).

[20]ibid 763-64 (holding that consideration of a COMPAS risk score at sentencing does not violate due process provided the score is not the determinative factor and the presentence report contains specified warnings about the tool’s limitations).

[21]ibid 765 (Loomis argued that the proprietary, trade-secret nature of the COMPAS algorithm prevented him from challenging the scientific validity of the tool used against him).

[22]Loomis v Wisconsin 137 S Ct 2290 (2017) (denying certiorari).

[23]Julia Angwin and others, ‘Machine Bias’ (ProPublica, 23 May 2016) (finding that COMPAS was roughly twice as likely to falsely flag Black defendants as future criminals compared with white defendants, and roughly twice as likely to falsely flag white defendants as low risk).

[24]Sandra G Mayson, ‘Bias In, Bias Out’ (2019) 128 Yale Law Journal 2218, 2224-27.

[25]Rashida Richardson, Jason M Schultz and Kate Crawford, ‘Dirty Data, Bad Predictions’ (2019) 94 New York University Law Review Online 15, 33-38 (documenting the use of crime data from police departments under consent decrees for unlawful practices as training inputs for predictive systems).

[26]Kate Crawford and Jason Schultz, ‘Big Data and Due Process’ (2014) 55 Boston College Law Review 93, 109-14 (proposing a ‘procedural data due process’ model requiring notice and a meaningful opportunity to contest inputs and outputs of predictive systems).

[27]Ferguson (n 1) 1146-50 (proposing judicial and legislative oversight mechanisms, including algorithmic impact statements and mandatory bias audits, prior to law enforcement deployment).

[28]Danielle Keats Citron and Frank Pasquale, ‘The Scored Society’ (2014) 89 Washington Law Review 1, 5-10.

[29]Andrew Guthrie Ferguson, The Rise of Big Data Policing (New York University Press 2017) 128-34.

[30]Brennan Center for Justice, ‘Predictive Policing Explained’ (2020).