Supreme Court justices have a tough job. They are required to hand down decisions that can affect millions of people and cost billions of dollars. And while some of the issues before them are purely legal, many turn on complex policy questions: Do black voters in the South still face substantial discrimination? How reliable are eyewitnesses? Are people convicted of sex offenses very likely to reoffend?
These questions can be answered only by understanding what the data says. Unfortunately, a report released by ProPublica on Tuesday suggests that the justices struggle with that task. Looking at a random sample of cases from 2011 to 2015, ProPublica found that the court cited faulty research or introduced their own errors in nearly a third of the 24 cases that relied on such facts.
In 2013, for example, Shelby County v. Holder invalidated a critical portion of the Voting Rights Act of 1965, making it arguably one of the most consequential cases in recent years. Justice John G. Roberts Jr., arguing that the South had taken great strides that made the protections of the act unnecessary, based his decision in part on a Senate Judiciary Committee analysis that misinterpreted how the Census Bureau reports race and ethnicity data and wrongly suggested that registration gaps between minorities and whites had shrunk significantly, an error that neither he nor his clerks caught.
As ProPublica itself acknowledges, its sample is too small to draw any solid statistical inferences, but the results are still troubling. They are also not particularly surprising. None of the justices has any serious training in statistics, and the clerks who assist them are almost all recent law school graduates, who rarely have any formal statistical background. Empirical facts are central to what the court does, but its members lack expertise.
