Measuring justice
| Author | William Cullerne Bown |
| Published date | 01 October 2019 |
| Date | 01 October 2019 |
| DOI | http://doi.org/10.1177/1365712719851136 |
| Subject Matter | Articles |
Article
Measuring justice
William Cullerne Bown
London, UK
Abstract
The possibility of measuring the success of the criminal justice system in distinguishing the guilty
from the innocent is often dismissed as impossible or at least impractical. Here I claim to
demonstrate that such epistemic measurement would only be difficult. All measurement
consists of two steps, the acquisition of observations and their processing through a compu-
tational framework. The law has lacked both, but I have recently put forward a computational
framework and here I set out how the necessary observations can be obtained. This completes
the conceptual foundations necessary for the development of jurisprudence as a social science,
for policymaking in the law that is rooted in rational concern for epistemic outcomes, and for
us to fulfil the modern, trustworthy and democratic promise that our forebears found in
Blackstone’s ratio.
Keywords
Blackstone’s ratio, democracy, empirical legal studies, epistemology, f-measure, standard of
proof
The possibility of measuring the success of the criminal justice system in distinguishing the guilty from
the innocent is often dismissed as impossible or at least impractical. Here I claim to demonstrate that
such epistemic measurement would only be difficult. All measurement consists of two steps, the acqui-
sition of observations and their processing through a computational framework. The law has lacked both,
but I have recently put forward a computational framework and here I set out how the relevant observa-
tions can be obtained. This completes the conceptual foundations necessary for the development of
jurisprudence as a social science, for policymaking in the law that is rooted in rational concern for
epistemic outcomes, and for us to fulfil the modern, democratic promise that our forebears found in
Blackstone’s ratio.
We don’t attempt to quantify the law’s success or failure in convicting the guilty and acquitting the
innocent. Perhaps the lack of interest in empirical measurement of the epistemic outcomes of the
criminal justice system stems from the common belief that it is impossible. For example, Epps wrote
recently, ‘ ...because of the very nature of the problem at issue, we have no effective way to measure the
phenomenon empirically’ (Epps, 2014: 1145).
Corresponding author:
William Cullerne Bown, 18 Church Terrace, London, SE13 5BT UK.
E-mail: wockbah@gmail.com
The International Journalof
Evidence & Proof
2019, Vol. 23(4) 399–421
ªThe Author(s) 2019
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/1365712719851136
journals.sagepub.com/home/epj
This article will argue that we can measure outcomes and, to convince, I will explicitly set out the
methods to be used. That work is the primary purpose of this article and, for reasons of space, only very
briefly at the end will I make the argument that the quantification of concerns that measurement allows
would be a good thing. More useful perhaps at the outset is to remind ourselves of three immediate
consequences of the lack of measurement.
First, within the academy, lack of measurement of what is a central concern is one of the things that
separates jurisprudence from the social sciences.
Second, Porter has documented how it was the army not teachers that insisted on the quantification of
IQ tests; tax collectors not companies that insisted on codification of accounting practices; and politi-
cians not engineers who insisted on cost-benefit analyses. The role of quantification in such cases is to
act as a ‘technology of trust’ in far-flung democracies. Without measurement, no similar oversight can
be established over the law, denying it an important source of democratic accountability and legitimacy
(Porter, 1996).
Third, as Tribe noted, when we thin k of justice, the lack of mathemati cs shifts our focus from
outcomes to ritual (Tribe, 1971: 1393).
In short, lack of measurement is a fundamental fault line. If we could measure epistemic outcomes,
we might expect the law to become a very different thing.
Given its reliance on numbers, the kind of measurement described here bears comparison to
attempts to apply principles of probability or economics to the law. However, it is distinct from those
two siblings and has its own underpinning theory (see for example Hand, 2004). To clarify what it
entails, when we talk of measuring something, we usually think of making observations. Yet these are
meaningless if they are not embedded in a computational framework that allows them to be evaluated.
Often this framework is so simple that we don’t even notice it; as Hand has noted, measurement is
often invisible (Hand, 2004: 1). For example, suppose you are investing $10 and with god-like insight
know that policyX would result in the first outcomein Table 1 and policy Y the second.Which outcome is
better?
Answer: Y. Furthermore, we have a general rule that we can apply to any two options: the bigger
number is better.
Now letus turn to the law. Supposeyou are trying 10,158 casesand with god-like insightknow that policy
X would result in thefirst bundle of outcomes in Table 2 andpolicy Y the second bundle. Which is better?
More importantly,what is the rule that would allow you to decide between any two bundle of outcomes?
Now the answer is far from obvious. Thus it becomes clear that when considering measurement the
law has not one but two problems: a) the well-discussed one of observations, of knowing which verdicts
are true and false; and b) the hidden one of establishing a computational framework for evaluating such
observations according to a linear scale of what is often called effectiveness.
Table 1. A simple policy choice.
$
X100
Y1000
Table 2. A more difficult policy choice.
True conviction False acquittal True acquittal False conviction
X 560 1300 7913 385
Y 288 1572 8197 101
400 The International Journal of Evidence & Proof 23(4)
To continue reading
Request your trial