The Epistemology of Statistical Evidence

Date01 April 2011
Published date01 April 2011
DOI10.1350/ijep.2011.15.2.373
Subject MatterArticle
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
The epistemology of
statistical evidence
By Amit Pundik*
Tel Aviv University
Abstract Numerous accounts have been suggested to explain differences
between statistical and individualised evidence and to justify restrictions on the
use of statistical evidence in court. A dominant direction in such accounts is the
idea that statistical evidence is lacking in some certain quality, making it
epistemologically unwarranted for establishing the fact it is introduced to prove.
Among the qualities suggested are luck, appropriate causal connection, weight,
and case-specificity. Epistemic accounts thus constitute attempts to identify an
intrinsic quality which individualised evidence (such as eyewitness testimony,
confession, the individual’s medical records) has but statistical evidence lacks, a
quality which makes inference from the statistical evidence to the particular
case epistemologically deficient. To the existing debate around epistemic
accounts, this article adds several generic and inherent problems from which
any epistemic account suffers. Based on these generic problems, it is concluded
that the epistemic direction lacks potential. If there is any justification for
restricting the use of statistical evidence in court, it does not lie in episte-
mology; it has to lie elsewhere.
Keywords Probability; Accuracy; Cohen; Bayes; Policy
e need both theoretically and practically to distinguish between those
types of statistical evidence which should be used in court and those
types which should not be used in court. In particular, case law seems to
lack a systematic approach to statistical evidence.1On the one hand, statistical
evidence is regularly adduced in litigation. For example, statistics are used in
doi:10.1350/ijep.2011.15.2.373
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF (2011) 15 E&P 117–143 117
1 In the United States, statistical evidence was rejected in cases such as Cimino vRaymark Industries
151 F 3d 297 (5th Cir 1998); In re Fibreboard Corporation 893 F 2d 706 (5th Cir 1990), but was deployed
in Blue Cross vPhilip Morris 113 F Supp 2d 345 (EDNY 2000); Hilao vEstate of Marcos 103 F 3d 767 (9th Cir
1996). For descriptive research on the use of statistical evidence in US courts, together with an
attempt to identify patterns when statistical evidence is used or rejected, see J. J. Koehler, ‘When
Do Courts Think Base Rate Statistics Are Relevant?’ (2002) 42 Jurimetrics Journal 373.
W
* Email: amitp@post.tau.ac.il. I am grateful to Adrian Zuckerman, John Gardner, David Enoch and
the anonymous referee for their helpful comments. I am also indebted to Elizabeth Pender and Uri
Preisman for their valuable technical assistance and to the PHG foundation for its generous
financial support.
relation to DNA evidence (to calculate the probability of a random DNA match in
criminal identification and civil parentage cases);2compensation for loss of
earnings (to calculate the life expectancy which the claimant would have enjoyed
had the defendant not unlawfully harmed her);3toxic torts (to prove causation);4
medical law (to prove loss of chance);5human rights (to prove the extent of
damage caused by the violation of the claimant’s human rights);6product liability
(to determine the market share for proportionate market-share liability);7
employment law (to prove group-based discrimination);8and competition law (to
calculate the economic damage resulting from price fixing).9On the other hand,
some types of statistical evidence have been subject to strong intuitive objections
in some well-known examples, both hypothetical and real. Statistical evidence has
been notoriously misused in some criminal cases. For instance, at the end of the
19th century in France, Alfred Dreyfus was convicted of treason after statistical
evidence was used creatively to prove his identity as the author of documents
118 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
2 For the criminal context, see RvDoheny and Adams [1997] 1 Cr App R 369. For the civil context, see
Family Law Reform Act 1969, s. 20 and N. Lowe and G. Douglas, Bromley’s Family Law, 10th edn
(Oxford University Press: Oxford, 2007) 325–31.
3 In comparison with the United States, the use of actuarial statistical methods is relatively limited
in the United Kingdom and preference is given to a simpler multiplicand and multiplier method.
See Hunt vSevers [1994] 2 AC 350, HL and S. Deakin, A. Johnston and B. Markesinis, Markesinis and
Deakin’s Tort Law, 6th edn (Oxford University Press: Oxford, 2007) 986. However, whether it is
acknowledged or not, even this method of calculation relies on some statistical evidence (e.g.
average longevity). For comparative work between the English methods and actuarial methods
which are used in the United States, see R. Lewis, R. McNabb, H. Robinson and V. Wass, ‘Court
Awards of Damages for Loss of Future Earnings: An Empirical Study and an Alternative Method of
Calculation’ (2002) 29 Journal of Law and Society 406. Based on their empirical findings, the authors
criticise the English system for under-compensating claimants.
4 Such evidence played a key role in the tobacco litigation (see, e.g., Blue Cross vPhilip Morris 113 F
Supp 2d 345 (EDNY 2000)). However, in other cases such evidence was rejected (see, e.g., the
asbestos cases Cimino vRaymark Industries 151 F 3d 297 (5th Cir 1998) and In re Fibreboard Corporation
893 F 2d 706 (5th Cir 1990)).
5Gregg vScott [2005] UKHL 2. The claim failed, but this case is still an example where statistical
evidence was admitted and used, even though the facts established by the evidence were
insufficient to support the claim.
6Hilao vEstate of Marcos 103 F 3d 767 (9th Cir 1996).
7Sindell vAbbott Laboratories 26 Cal 3d 588 (1980); In re ‘Agent Orange’ Product Liability Litigation 597 F
Supp 740 (EDNY 1984).
8 Perhaps the most notable case in this area is a class action of gender discrimination on behalf of
two million women against Walmart, the world’s largest private employer (see Dukes vWal-Mart Inc
(Dukes I) 222 FRD 137 (ND Cal 2003)). The decision to grant the claimants a class action certificate
was affirmed by the 9th Circuit Court first in 2007 (see Dukes vWal-Mart Inc (Dukes III) 474 F 3d 1214
(9th Cir 2007)) and then again in 2009(Dukes vWal-Mart Stores Inc.(Duke IV) No. 04-16688 (9th Cir Apr
26, 2010); the judgment can be found at
04/26/04-16688.pdf>, accessed 14 January 2011). The admission of such evidence in support of
discrimination is not new (see Alabama vUnited States 304 F 2d 583 (5th Cir) aff’d, 371 US 37 (1962)
and, more recently, Caridad vMetro-North Commuter Railroad 191 F 3d 283 (2d Cir 1999)).
9United States vColumbia Pictures 25 FRD 497 (SD New York 1960).
which were leaked to the Germans.10 Almost 70 years later, a couple were
convicted of robbery on the sole basis that they matched some characteristics
described by eyewitnesses (the woman was white with blond hair, the man was
black with a beard and moustache, they were found in a yellow convertible car,
and so on). The prosecution was allowed to use a rough statistical calculation to
prove how unlikely it would be to find a couple who shared all these character-
istics.11
As well as by real-life cases, the academic literature on statistical evidence has also
been troubled by contradictory paradoxes and strong yet conflicting intuitions. It
discusses several hypothetical examples in which the use of some types of statis-
tical evidence seems intuitively wrong. Consider the following variation on
Jonathan Cohen’s famous gatecrasher paradox.12 Humpty, a football enthusiast, is
accused of gatecrashing a local football match. He was seen sneaking through the
fence by Alice. Alice’s visual identification ability is tested and shown to be
accurate nine times out of ten. This case seems straightforward and a finding that
Humpty gatecrashed intuitively right.13 In another football match, Hatter,
another football enthusiast, is also accused of gatecrashing. This time, the only
evidence available is that just 100 tickets had been sold while 1,000 spectators
were counted. The evidence against Hatter, who is sued for gatecrashing, is based
on the fact that nine out of ten spectators gatecrashed. Surprisingly, this case is
relatively straightforward too: finding that Hatter gatecrashed upon this statis-
tical evidence seems intuitively wrong.14 It is surprising because in both cases, at
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 119
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
10 A full description of this case can be found in L. H. Tribe, ‘Trial by Mathematics: Precision and
Ritual in the Legal Process’ (1971) 84 Harvard L Rev 1329 at 1332–4.
11 People vCollins 438 P 2d 33, 68 Cal 2d 319 (1968). This case was at the centre of a lengthy scholarly
debate (see, e.g., W. B. Fairley and F. Mosteller, ‘A Conversation about Collins’(1974) 41 UChicago L
Rev 252). Decades later, the case has continued to occupy the literature (see, e.g., M. L. Huffman,
‘When the Blue Bus Crashes into the Gate: The Problem of People v Collins in the Probabilistic
Evidence Debate (comment)’ (1992) 46 Miami L Rev 975). It was even used in discussion of the
controversial O. J. Simpson case (see J. J. Koehler, ‘One in Millions, Billions, and Trillions: Lessons
from People v. Collins (1968) for People v. Simpson (1995)’ (1997) 47 Journal of Legal Education 214).
12 L. J. Cohen, The Probable and the Provable (Clarendon Press: Oxford, 1977) 74. The legal context of the
type of proceedings (criminal or civil) and the precise heading of the claim are left vague
intentionally.
13 It could be argued that if the case were criminal, then the fact that Alice is mistaken in one tenth of
her identifications is sufficient to establish reasonable doubt. However, the example may be
adjusted to accommodate one’s desired threshold of reliability (e.g. Alice ismistaken only once in
100 times).
14 However, this intuition is not accepted by everybody (see, e.g., J. Brook, ‘The Use of Statistical
Evidence of Identification in Civil Litigation: Well-worn Hypotheticals, Real Cases, and Contro-
versy’ (1984) 29 St Louis ULJ 293 at 330). Brook compares the opposite intuitions in the responses of
Eggleston and Williams to Cohen’s gatecrasher paradox (see R. Eggleston, ‘The Probability Debate’
[1980]Crim LR679 at 681–2; G. Williams, ‘TheMathematics of Proof—I’ [1979]Crim LR 297 at 304).
least prima facie, the risk of error is similar. In both cases, there is a one in ten
probability that a finding against the defendant would be mistaken.
Another paradigmatic example is the red and blue buses scenario,15 where a lady
who has been hit by an unidentified bus claims damages from the red bus
company on the basis that 60 per cent of the buses in that area are operated by that
company. To many, it would seem intuitively wrong to impose liability solely
upon this evidence.16 This statistical evidence may be compared to the testimony
of an eyewitness who testifies that the bus was red, where his colour identification
ability in similar lighting conditions has been found to be accurate 60 per cent of
the time.17 Many people would have a strong intuition that there is a fundamental
difference between the evidence given in the first scenario and other types of
evidence, such as eyewitness testimony, confession, and medical records.
Empirical research has shown that, when other evidence is available (such as
Alice’s testimony that she saw Humpty sneaking through the fence), people tend
to disregard background statistical evidence (such as the fact that 900 out of 1,000
spectators avoided paying for admission).18 These findings suggest that many
people believe that statistical evidence should be treated differently.
It is important to note that when discussing ‘statistical evidence’, this article
refers to base rates, background probabilities, or ‘the relative frequency with
which an event occurs or an attribute is present in a population’.19 The consti-
120 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
15 This example is a quantified variation on Smith vRapid Transit Inc 58 NE 2d 754 (Mass 1945). It is an
example which is very popular in the literature about statistical evidence and is used also in
Kahneman and Tversky’s empirical studies (D. Kahneman and A. Tversky, ‘Causal Schemas in
Judgment under Uncertainty’ in M. Fishbein (ed.), Progress in Social Psychology (Erlbaum: Hillsdale NJ,
1980) 49–72). It even found its way to an influential psychological experiment (see G. L. Wells,
‘Naked Statistical Evidence of Liability: Is Subjective Probability Enough?’ (1992) 76 Journal of
Personality and Social Psychology 533). For an illuminating and entertaining discussion of this
example, see Brook, above n. 14 at 298–9 and 301–3.
16 But recall that not everybody shares this intuition (see above n. 14).
17 For troubling findings about the reliability of eyewitnesses, see P. Roberts and A. A. S. Zuckerman,
Criminal Evidence (Oxford University Press: Oxford, 2004) 218–21. For a recent survey of these
studies, see K. J. Heller, ‘The Cognitive Psychology of Circumstantial Evidence’(2006) 105 Michigan
L Rev 241 at 248–50 (human tendency to give undue weight to eyewitnesses) and 252–3 (unreli-
ability of identification parades).
18 See Kahneman and Tversky, above n. 15; and also D. Kahneman, P. Slovic and A. Tversky, ‘Evidential
Impact of Base Rates’ in D. Kahneman, P. Slovic and A. Tversky (eds), Judgment under Uncertainty:
Heuristic and Biases (Cambridge University Press: Cambridge, 1982) 153–60. See also the Wells effect,
according to which if people are presented only with statistical evidence, they are likely to acquit
even when their subjective probabilities of guilt are sufficient to convict (Wells, above n. 15).
19 J. J. Koehler and D. N. Shaviro, ‘Veridical Verdicts: Increasing Verdict Accuracy through the Use of
Overtly Probabilistic Evidence and Methods’ (1990) 75 Cornell L Rev 247 at 252.
tutive characteristic of statistical evidence focused upon here is population, which
in statistical terms is a group of people or events.20 Two types of population are
particularly relevant for the legal use of statistical evidence. The first is the
population constructed from other people similar to the litigant in relation to
whom the statistical evidence is adduced. For example, in the American case of
Shonubi,21 Judge Weinstein used statistics about 117 other similar drug smugglers
in order to determine the drug quantities smuggled by Shonubi, for the purpose of
sentence enhancement. The second type of population is constructed from events
in the litigant’s past. The most immediate example is the accused’s previous
convictions.22
Some common objections to the use of statistical evidence in court rely on the risk
of misusing statistics. Several difficulties relating to how to use and present
statistics correctly have been discussed both in the academic literature23 and in
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 121
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
20 Regarding statistical evidence as a separate type of evidence might be viewed as an assertion that
statistical evidence is qualitatively different from other types of evidence. However, the use in this
article of the term ‘statistical evidence’ does not presuppose any qualitative difference between
statistical and individualised evidence. All that is assumed in this article is that some pieces of
evidence are intuitively perceived as ‘statistical’ and some are not. Whether this perceived
difference relies on a real difference or merely on a different manner of representation is a central
question of the debate around the issue of statistical evidence.
21 United States vShonubi 103 F 3d 1085 (2d Cir 1997) [Shonubi-IV]. This legal ordeal has received
intense scholarly attention (see, e.g., M. Colyvan, H. M. Regan and S. Ferson, ‘Is it a Crime to
Belong to a Reference Class?’ (2001) 9 Journal of Political Philosophy 168, and Tillers’ fierce criticism,
P. Tillers, ‘If Wishes Were Horses: Discursive Comments on Attempts to PreventIndividuals from
Being Unfairly Burdened by their Reference Classes’ (2005) 4 Law Probability and Risk 33). More
recently, the case served as a central example in Allen and Pardo’sclaim, according to which the
problem of the reference class creates difficulties forany attempt to use mathematical models in
evidence law (R. J. Allen and M. S. Pardo, ‘The Problematic Value of Mathematical Models of
Evidence’ (2007) 36 Journal of Legal Studies 107). For others, this case has been viewed as evidence of
the judiciary’s incompetence in apprehending and using statistical evidence. Tillers, e.g., has
suggested that ‘The opinion of the court of appeals is ... depressing because ... it suggests that quite
a few judges ... still may not have a grasp of some basic characteristics of probabilistic and
statistical methods and arguments’ (P. Tillers, ‘Introduction: Three Contributions to Three
Important Problems in Evidence Scholarship’ (1997) 18 Cardozo L Rev 1875 at 1879, emphasis
original).
22 For this issue in general, see Roberts and Zuckerman, above n. 17 at ch. 11.
23 See, e.g., the debate between Tribe and Fairley and Finkelstein, about the correct statistical analysis
of an example brought by Fairley and Finkelstein (W. B. Fairley and M. O. Finkelstein, ‘A Bayesian
Approach to Identification Evidence’ (1970) 83 Harvard L Rev 489; L. H. Tribe, ‘Trial by Mathematics:
Precision and Ritual in the Legal Process’ (1971) 84 Harvard L Rev 1329; W. B. Fairley and M. O.
Finkelstein, ‘The Continuing Debate over Mathematics in the Law of Evidence: A Comment on
“Trial by Mathematics”’ (1971) 84 Harvard L Rev 1801; L. H. Tribe, ‘The Continuing Debate over
Mathematics in the Law of Evidence: A Further Critique of Mathematical Proof’ (1971) 84 Harvard L
Rev 1810).
practice.24 While the objections to the use of statistical evidence which are based on
such difficulties are important, they are nevertheless practical in nature. Rather
than restricting the use of statistical evidence in court, these difficulties could be
addressed through better education and training of the legal profession25 and/or
through more assistance from expert statisticians.26 Acknowledgement of the
practical difficulties should therefore be distinguished from principled accounts
against the use of statistical evidence per se. Such accounts, if correct, will apply even
when the statistics are gathered, analysed and presented with the utmost profes-
sionalism by experienced expert statisticians, and correctly interpreted by the judge
and jury. This article focuses on this type of principled argumentation, that is,
arguments which seek to establish the need for differential treatment of statistical
evidence in court, even when the calculations used are statistically sound.
The existing literature on statistical evidence attempts to identify, explain, and
justify the differences between problematic statistical evidence and other types of
evidence which are clearly acceptable.27 Substantial discussion is dedicated to the
122 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
24 A recent example is the controversy concerning the use of statistical evidence in the case of Sally
Clark (RvClark (No. 1) [2000] EWCA Crim 54; RvClark (No. 2) [2003] EWCA Crim 1020). Following this
case, the Royal Statistical Society expressed concerns, in a public letter, about the misuse of
statistics in that case, urging the courts ‘to ensure that statistical evidence is presented only by
appropriately qualified statistical experts, as would be the case for any other form of expert
evidence’ (see Royal Statistical Society, Press Release on the Sally Clark Case, 23 October 2001,
available at , accessed 15 January 2011).
25 Several scholars have rightly called for such training (see, e.g., J. J. Koehler, ‘The Probity/Policy
Distinction in the Statistical Evidence Debate’ (1992) 66 Tulane L Rev 141 at 148–9).
26 For example, Dawid compares the current legal approach to statistics to the state of science before
Galileo. He states that ‘statisticians … have much to contribute towards identifying and clarifying
many delicate issues in the interpretation of legal evidence’ (Dawid refers to both statistical and
non-statistical evidence) (A. P. Dawid, ‘Bayes’s Theorem and the Weighing of Evidence by Juries’
(2002) 113 Proceedings of the British Academy 71 at 71–2). These remarks resonate with similar
suggestions made decades ago in the United States (see, e.g., I. J. Good, Probability and the Weighingof
Evidence (C. Griffin: London, 1950) 66–7 and W. B. Fairley and M. O. Finkelstein, ‘A Bayesian
Approach to Identification Evidence’ (1970) 83 Harvard L Rev 489 at 502, 516–17).
27 Acceptable types of evidence might include both non-statistical evidence (e.g. eyewitness testimony,
confession, medical records) and some specific types of statistical evidence (e.g. DNA evidence and the
various examples in the text after n. 1). Proponents of the distinction need to provide a more refined
distinction between acceptable and problematic statistical evidence, together with some explanation as
to why any differential treatment of objectionable statistical evidence should not apply to types of
statistical evidence they consider acceptable. I have only notedthe challenge here, rather than following
it up, because I do not intend to pursue this line of argument. However, it should be noted that the
difficulty of distinguishing between DNA evidence and other statistical evidence has been used to
challenge some of the proposed distinctions (see, e.g., A. Pundik, ‘Epistemology andthe Law of Evidence:
Four Doubts about Alex Stein’s Foundations of Evidence Law’ (2006) 25 Civil Justice Q 504 at 511–12 and the
exchange between Wasserman and Cohen, D. T. Wasserman, ‘The Morality of Statistical Proof and the
Risk of Mistaken Liability’ (1992) 13 Cardozo L Rev 935; L. J. Cohen, ‘A Comment on Wasserman’s “The
Morality of Statistical Proof and The Risk of Mistaken Liability”’ (1991) 13 Cardozo L Rev 977; D. T.
Wasserman, ‘Reply to Cohen: Common Sense about Naked Statistics’ (1991) 13 Cardozo L Rev 981).
question of what makes certain types of statistical evidence problematic.28
Amongst the various attempts it is possible to distinguish between two kinds of
argumentation or accounts. The first kind of account, which has received most of
the scholarly attention, seeks to identify an intrinsic quality of individualised
types of evidence, which statistical evidence, or the inference drawn from it, lacks.
Among the qualities suggested are weight,29 appropriate causal connection,30
case-specificity,31 inability to provide the best explanation,32 and vulnerability to
the problem of the reference class.33 Proponents of these accounts share the view
that statistical evidence lacks a quality which makes inference from it to the
particular case epistemologically deficient (hereafter, these accounts are referred
to as ‘epistemic accounts’). The second kind of account, which has received far less
attention, focuses on the context in which the evidence is used. Rather than
focusing on the statistical evidence itself, this type of account tries to identify
something in the nature of the legal process which makes the use of statistical
evidence inappropriate. Among such attempts are emphases on the legitimacy of
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 123
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
28 It could be argued that the question in the text is framed too broadly because the issue is relevant
only to a small and close list of examples such as the gatecrasher paradox and the red and blue
buses scenario. Instead of discussing ‘statistical evidence’ in general, one could hold that a more
fruitful analysis would focus on analysing these specific examples (such an approach is
exemplified by the thorough analysis of M. Redmayne, ‘Exploring the Proof Paradoxes’ (2008) 14
Legal Theory 281). However, as illustrated by the real-life cases mentioned above, the issue is not
limited to hypothetical examples, even though the latter attract much of the scholarly attention.
More importantly, legal practice constantly provides new examples of questionable uses of
statistical evidence. See, e.g., United States vVeysey 334 F 3d 600 (7th Cir 2003); United States vShonubi
103 F 3d 1085 (2d Cir 1997) [Shonubi-IV]; and most recently, the Dutch case of Lucy De Berk, a
nurse who was convicted of seven murders and three attempted murders, based in part on a statis-
tician’s testimony, according to which the probability of so many natural deaths all occurring
during the shifts of a single nurse would be 1 in 342 million (for a detailed analysis of this case, see
T. Derksen and M. Meijsing, ‘The Fabrication of Facts: The Lure of the Incredible Coincidence’ in H.
Kaptein, H. Prakken and B. Verheij (eds), Legal Evidence and Proof: Statistics, Stories, Logic (Ashgate:
Farnham, 2009) 39–70). De Berk was recently acquitted and released after six and a half years in jail
(see BBC, ‘Dutch nurse Lucy De Berk acquitted of patient murders’, 14 April 2010, available at
, accessed 15 January 2011). Since the matters
discussed here cannot be restricted to a set of specific cases (such as proof paradoxes), the article
concentrates on the broader category of ‘statistical evidence’.
29 Cohen, above n. 12.
30 J. J. Thomson, ‘Liability and Individualized Evidence’ (1986) 49 Law and Contemporary Problems 199;
R. W. Wright, ‘Causation, Responsibility, Risk, Probability, Naked Statistics, and Proof: Pruning the
Bramble Bush by Clarifying the Concepts’ (1988) 73 Iowa L Rev 1001.
31 A. Stein, Foundations of Evidence Law (Oxford University Press: Oxford, 2005).
32 M. Dant, ‘Gambling on the Truth: The Use of Purely Statistical Evidence as a Basis for Civil Liability’
(1989) 22 Columbia Journal of Law and Social Problems 31; R. J. Allen and M. S. Pardo, ‘Juridical Proof and
the Best Explanation’ (2008) 27 Law and Philosophy 223.
33 R. J. Allen and M. S. Pardo, ‘The Problematic Value of Mathematical Models of Evidence’ (2007) 36
Journal of Legal Studies 107.
the legal system,34 the over-transparency of standards of proof,35 equality between
litigants,36 and the individuality and autonomy of the litigant or defendant
against whom the statistical evidence is used.37 Proponents of this type of account
share the view that, even if statistical evidence may be useful in other contexts
(science, policy-making, and so on), its use in court conflicts with fundamental
values of the legal system. A distinctive feature of this type of account is that the
justification it provides applies only or mainly to the legal context, while the
epistemic accounts provide a justification which could, in principle, be applied to
many other contexts (even though the proponents of these accounts might not
develop these applications explicitly, see below).
Over the years, both types of accounts have been scrutinised and criticised again
and again, yet no conclusive justification for differential treatment of statistical
evidence in court has been identified.38 Instead of scrutinising some of the specific
accounts and exposing their flaws yet again, this article takes a new approach by
stepping back and trying to identify the type of justification which is needed. It is
argued here that the epistemic accounts suffer from generic problems which
make all epistemic accounts, regardless of their specific strengths and weak-
nesses, insufficient for providing such a justification. Based on these generic
problems, it is concluded that if there exists any justification for restricting
the use of statistical evidence in court, that justification is necessarily specific to
the legal context and must integrate the special characteristics of this context. The
point is not that epistemology is irrelevant to the issue of statistical evidence. If
epistemology is responsible merely for providing a definition or clarification of
the difference between statistical and individualised evidence, then it should be
welcomed. The point is rather that epistemology cannot play a justificatory role in
124 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
34 Tribe, above n. 10.
35 C. R. Nesson, ‘Reasonable Doubt and Permissive Inferences: The Value of Complexity’ (1979) 92
Harvard L Rev 1187; C. Nesson, ‘The Evidence or the Event? On Judicial Proof and the Acceptability
of Verdicts’ (1985) 98 Harvard L Rev 1357.
36 Stein, above n. 31.
37 Wasserman, above n. 27; A. A. S. Zuckerman, ‘Law, Fact or Justice?’ (1986) 66 Boston U L Rev 487.
38 For a recent critical survey of the various attempts, together with a similar conclusion about the
openness of the problem, see Redmayne, above n. 28. An excellent philosophical critique of the
various attempts to distinguish between individualised and statistical evidence can be found in F.
Schoeman, ‘Statistical vs. Direct Evidence’ (1987) 21 Nous 179. For a critical survey of the epistemic
accounts, see A. Pundik, ‘What is Wrong with Statistical Evidence? The Attempts to Establish an
Epistemic Deficiency’ (2008) 27 Civil Justice Q 461. Contextualist accounts have also received some
scrutiny. For a thorough critique of Nesson and Tribe’s accounts, see D. Shaviro,
‘Statistical-Probability Evidence and the Appearance of Justice’ (1989) 103 HarvardL Rev 530. For an
analysis of Wasserman’s account, see A. Pundik, ‘Statistical Evidence and Individual Litigants: A
Reconsideration of Wasserman’s Argument from Autonomy’ (2008) 12 E&P 303.
explaining why statistical evidence ought to be treated differently from individu-
alised evidence when it is used in court. The justification for such a differential
treatment, if it exists, has to lie elsewhere. More effort should therefore be devoted
to developing and exploring contextualist accounts.
The generality of epistemic accounts
Those accounts which attempt to provide a justification for restricting the use of
statistical evidence in court by seeking to identify a quality which individualised
evidence has but statistical evidence lacks are here grouped together under the
description ‘epistemic accounts’; in these accounts, the absence of this quality
makes inference from the statistical evidence to the particular case epis-
temologically deficient, so that neither the statistical evidence nor the inference
drawn from it can establish the material fact that this evidence was adduced to
prove. Epistemology is the branch of philosophy which addresses questions about
the nature, possibility and structure of knowledge, justification, belief and other
related conditions.39 Unlike psychology, which attempts to describe how human
beings actually think, epistemology usually asks how a rational agent ought to
think and form his rational beliefs. Epistemology is therefore normative in
nature.40
It thus seems plausible to expect that epistemic principles will have some degree
of general applicability outside of the legal context. If it is epistemologically unwar-
ranted to form a belief using certain irrational methods, then it should be
expected that this epistemic principle will apply regardless of whether the agent is
a juror, a scientist or a layman. Notably, some proponents of epistemic accounts
explicitly endorse this general applicability. For example, when applying his
theory to medical diagnoses, Jonathan Cohen concludes that ‘it is in the patient’s
interest … to have his physician rely as little as possible on the calculation of
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 125
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
39 The above description is based on J. L. Pollock and J. Cruz, Contemporary Theories of Knowledge, 2nd
edn (Rowman & Littlefield: Oxford, 1999) 11–12.
40 This description of ‘epistemology’ doesnot include some types of naturalised epistemology, such
as that outlined in W. V. O. Quine, ‘Epistemology Naturalized’ in W. V. O. Quine (ed.), Ontological
Relativity and Other Essays (Columbia University Press: New York, 1969). Such epistemology, as
Quine himself emphasises (e.g. ibid. at 75), seeks to move away from the normative project to a
descriptive one, the purpose of which is to describe how we in fact reason. The accounts referred
to as ‘epistemic’ in this article do not aim merely to describe how we reason from statistical
evidence, but to justify or explain why the use of such evidence in court ought to be restricted.
Therefore, while naturalised epistemology is an importantpart of the epistemic scholarship, it is
not directly relevant to the accounts discussed here and thus is not included in the description
above.
mathematical probabilities’.41 Another example is Alex Stein, who goes much
further and claims not only that statistical evidence (or, in Stein’s terminology,
‘general evidence’)42 is irrelevant to case-specific hypotheses,43 but also that
case-specific evidence is irrelevant to general hypotheses.44 Other proponents of
epistemic accounts try to limit the general applicability of their account in some
respects. Judith Thomson’s attack on statistical evidence (or, in her terminology,
‘non-individualized evidence’) is limited to cases where the epistemic guarantee
necessary for the judgment is similar to the guarantee required for establishing
claims of knowledge.45 However, for Thomson’s theory to have any implications
for the legal context, it is necessary to assume that legal fact-finding has to comply
with similar epistemic standards to the search for knowledge, at least with respect
to epistemic guarantees.46 Under this assumption, Thomson’s account still has
some degree of general applicability because it applies to any context, legal or
non-legal, where the evidence is used to provide the epistemic guarantee needed
126 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
41 L. J. Cohen, ‘Bayesianism versus Baconianism in the Evaluation of Medical Diagnoses’ (1980) 31
British Journal of Philosophy of Science 45 at 60–1. Cohen then concedes that pragmatic non-epistemic
considerations (such as finance, professional effort, and so on) might justify the opposite policy
(ibid.).
42 Stein does not define ‘general’ or ‘case-specific’ evidence, but these terms are constantly used in his
work, together with other undefined terms. The lack of proper definitions has attracted criticism
from various reviewers. Redmayne, e.g., criticises Stein for failing to define what he means by
‘unevidenced’: ‘[t]his word [unevidenced] occurs frequently in the exposition of PMI, though we are
nowhere told what Stein means by it’ (M. Redmayne, ‘The Structure of Evidence Law: Review of
Alex Stein, Foundations of Evidence Law’ (2006) 26 OJLS 805 at 811). Nance comments that ‘[a] number
of Stein’s important distinctions, arguments, and conceptual taxonomies appear and reappear in
his various chapters, sometimes repeated (in slightly altered form) as if they had not been
previously introduced, sometimes explicitly repeated but with important differences that may not
have been intended’ (D. A. Nance, ‘Allocating the Risk of Error (Review Essay)’ (2007) 13 Legal Theory
129 at 129). However, given the examples and paradoxes to which Stein refers (Stein, above n. 31 at
77–99), it seems plausible that his category of general evidence is similar if not identical to the
category of statistical evidence used in this article.
43 Stein’s theory may give rise to two distinct accounts of statistical evidence. First, Stein refers to the
value of equality, which may provide a non-epistemic justification for restricting the use of
statistical evidence in court. Secondly, Stein considers ‘general evidence’ to be epistemologically
deficient in comparison to ‘specific evidence’ in its ability to support case-specific propositions.
This alleged epistemic inferiority could easily give rise to an epistemic account which would
justify the restriction of statistical evidence in court. It remains unclear which of the solutions is
intended by Stein, whether they are alternative or cumulative, and if the latter, how the two
elements work together. This article focuses on the epistemic elements in Stein’s theory.
44 ‘For any such proposition [that is general in nature], evidence highlighting some specific factual
setting would be irrelevant’ (Stein, above n. 31 at 92). If this were true, it would be hard to explain
the gain from scientific experiments because, according to Stein, the specific evidence obtained in
these experiments is irrelevant to the general laws of nature which they are meant to confirm (see
Pundik, above n. 27 at 515–16).
45 Thomson, above n. 30.
46 The context on which Thomson focuses is clearly the legal context, as is evident from the
references to ‘jurors’, ‘courts’, and so on.
to establish claims of knowledge (such as in the sciences). Ronald Allen and
Michael Pardo’s account of inference to the best explanation can be understood as
merely attempting to describe how statistical evidence is treated in court.47
However, Allen and Pardo draw on a theory of reasoning which is used both in
daily life and in scientific reasoning, as demonstrated persuasively by Peter
Lipton.48 Even if their account does not apply to every possible context, it is surely
not limited to the legal context, and thus it also has some degree of general appli-
cability. If one of the epistemic accounts is correct, it will be applicable to several
uses of statistical evidence, in legal and non-legal contexts alike. According to
epistemic accounts, using statistical evidence in any of these contexts to decide
particular cases will always be epistemologically unwarranted, and hence
undesirable.
Yet such a conclusion goes against too many contemporary intuitions and
practices. Ferdinand Schoeman cites the following helpful example.49 Assume that
you hear on the radio that 70 per cent of the eggs in a certain supermarket have
been contaminated. Proponents of epistemic accounts have argued that,
compared with individualised evidence (for example, examining the particular
eggs you have purchased in laboratory tests), statistical evidence lacks weight,
case-specificity, causal connection, or another property. According to these
accounts, inferences from the statistical evidence on the rate of egg contami-
nation to the particular case are epistemologically deficient. But if this is true,
should we avoid throwing these eggs away because of the evidence? Are we
irrational in wasting money by refusing to eat those eggs because we are relying
on evidence or an inference which the epistemic accounts claim is
epistemologically deficient and even irrelevant? To avoid answering these
questions in the affirmative, proponents of epistemic accounts need to explain
why the same missing quality, which they hold to be so significant in legal
fact-finding, should not be equally applicable to the eggs example, making our
decision to throw the eggs away epistemologically unwarranted or even irrational.
A preliminary concern with the eggs example might be that it conflates the
pragmatic question of ‘should we throw the eggs away?’ with the epistemic
question of ‘do we know that the eggs are contaminated?’. Proponents of
epistemic accounts might retort that even if the decision to throw away the eggs is
rational, there is still a difference in quality between the epistemic support that
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 127
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
47 Allen and Pardo, above n. 33.
48 P. Lipton, Inference to the Best Explanation, 2nd edn (Routledge: London, 2004).
49 Schoeman, above n. 38 at 184–5. Schoeman provides this example to support a different argument
from that which is made in this section (he uses it to refute the claim that statistical evidence is
irrelevant to particular cases).
the statistical evidence provides for this decision and the support provided by an
individualised examination of the specific eggs in the lab. Be that as it may, this
response only highlights the gap between abstract epistemic concerns on the one
hand, and both practical reasoning and legal fact-finding on the other. Granting,
for the sake of argument, that there is an epistemic difference between the two
types of evidence, the eggs example illustrates that whatever the alleged epistemic
difference amounts to, it does not significantly influence our decision about
whether or not to throw the eggs away. Proponents of epistemic accounts need to
provide an additional theory to explain why the alleged epistemic difference
matters nonetheless, and in what circumstances it should affect our decision-
making procedures and how. In particular, they need to provide yet another
theory to justify differential treatment of statistical evidence in court. Perhaps the
epistemic difference between the two types of evidence could play a role in such a
further theory, for example, by identifying the types of evidence to which that
theory would apply. However, the epistemic difference between the types of
evidence, in and of itself, does not lead to the conclusion that legal decisions
should not be made based on such evidence. It is also unclear what justificatory role
this difference has: how would the existence of an epistemic difference help
justify differential treatment of statistical evidence in court (or in any other
practical context)? This gap between epistemic and legal concerns is further
explored in the last section.
To avoid condemning the decision to throw the eggs away as epistemologically
unwarranted or irrational, proponents of epistemic accounts need to explain why
the same missing quality that they hold to be so significant in legal fact-finding
should not be equally applicable to the eggs example. One way to do so is to draw a
distinction between the two pieces of evidence, or between the inferences drawn
from them, and to hold that unlike the legal examples described above, the statis-
tical evidence in the eggs example is not epistemologically deficient. However, it is
hard to explain how this evidence or inference differs from any other clear
example of statistical evidence. Consider the following example. An individual is
accused of a violent crime and the prosecution seeks to adduce statistical evidence
showing the high rate of violent crime in the individual’s neighbourhood. Using
this evidence to support the individual’s conviction is clearly objectionable,
because the individual should not be judged on the way others living in her neigh-
bourhood tend to behave. However, there seems to be little difference in the type of
evidence or inference between the two examples. In both, the material fact
(whether the individual committed the violent crime or whether the purchased
eggs are contaminated) is proven with evidence about the frequency of a specific
feature (the rate of violent crime or the rate of contamination) in other similar
cases (residents of the same neighbourhood or similar eggs from the same super-
128 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
market). While there are several obvious differences between the cases (the
content of the evidence, the purpose for which it is used, etc.), the type of evidence
and inference is similar between the two cases. Epistemic accounts, with their
emphasis on the missing epistemic quality in the evidence or inference itself,
would struggle to explain why the crime rate evidence is problematic but the egg
contamination evidence is not.
One possible way to respond to this challenge is to resort to contextualist episte-
mology. Contextualist epistemology (or ‘contextualism’) is based on the idea that
the truth of propositions which include epistemic concepts such as ‘knowledge’
and ‘justification’ depends on the context of the conversation and on the speaker’s
identity. For example, if Alice says ‘I know that tomorrow will be rainy’, this propo-
sition may be true if it is made as part of an ordinary conversation or for the
purpose of choosing what clothes to pack for her trip, but the very same propo-
sition may not be true if said in a conversation with a radical sceptic or for the
purpose of establishing a solid scientific finding.50 Proponents of epistemic
accounts might try to resort to contextualist epistemology in order to explain and
justify why the appropriateness of using the same piece of statistical evidence
varies across different contexts. However, it is important to emphasise that
adopting a contextualist epistemology does not undermine the claim of this
article, but is likely to embrace it. In order to set aside any of the problems
identified here by adopting a contextualist epistemology, one would have to refer
to a particular characteristic of the (legal) context in which the statistical evidence
is used in order to justify the intuitive objection to its use in that context. This
article argues that restricting the use of statistical evidence cannot be justified by
any reason that is insensitive to the fact that the evidence is used in court rather
than in any other context. Adopting a contextualist epistemology and resorting to
a particular characteristic of the legal context inter alia accepts this argument.
A related response to the need to explain the difference between the use of statis-
tical evidence in court and its use in the eggs example could be based on the claim
that using statistical evidence in the eggs example is less objectionable because
not much is at stake (losing a box of eggs). By contrast, in the legal context,
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 129
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
50 Some of the main works in this area are K. DeRose, ‘Solving the Skeptical Problem’ (1995) 104
Philosophical Review 1; S. Cohen, ‘Contextualist Solutions to Epistemological Problems: Scepticism,
Gettier, and the Lottery’ (1998) 76 Australasian Journal of Philosophy 289; D. Lewis, ‘Elusive Know-
ledge’ (1996) 74 Australasian Journal of Philosophy 549. It should be emphasised that contextualist
epistemology does not require a relativist position on truth. It is possible to hold that the truth of
the proposition ‘the agent knows that p’ is context-dependent but that the truth of the proposition
pis context-independent, that is, pis true (or false) regardless of whether the agent knows it or not.
For more on the relation between contextualism and relativism, see M. Richard, ‘Contextualism
and Relativism’ (2004) 119 Philosophical Studies 215.
especially in the criminal context, the cost of error is much higher and so the use
of statistical evidence is more objectionable. However, even if it is possible to
integrate the cost of error into an epistemology, this response does not help much.
It requires only that the evidence in the criminal context be more reliable than in
other contexts (for example, in order to satisfy a higher standard of proof).
Moreover, the higher cost of error means that there is an extra motivation for
using statistical evidence in the criminal context, because using it may improve
the overall accuracy of the legal system and thus reduce the risk of error.51 If it
turns out that the cost of error influences which types of evidence are used, it
would seem that statistical evidence should be used especially in the criminal context
whenever its use contributes to the accuracy of legal fact-finding.
There are plenty of other practices in which we determine our conduct based only
on statistical evidence, even when the cost of error is high and even when using
similar evidence to convict an individual in court would seem seriously
problematic.52 Some people are persuaded to stop smoking on statistical evidence
about the connection between smoking and lung cancer. Insurance companies
determine their premiums on statistical evidence about other people with similar
characteristics. In what respect, then, does the type of evidence or inference in
these cases differ from the evidence or inference used in the gatecrasher paradox
or in the red and blue buses scenario? If the type of evidence or inference is similar,
and if statistical evidence always lacks weight, case-specificity, or another
property, as the epistemic accounts allege, then these decisions are unwarranted
according to the epistemic accounts. If this is the case, should a rational agent
challenge them? Is it irrational to stop smoking 40 cigarettes a day in light of statis-
tical evidence on the connection between heavy smoking and lung cancer?
Epistemic accounts might be pushed to condemn too many contemporary
practices and intuitions, and support radical and implausible reforms of those
practices.
The conclusion which emerges is that the use of the same piece of statistical
evidence might be inappropriate in court but appropriate outside court. For
example, consider again the evidence regarding the rate of violent crime in a
certain neighbourhood. Using this statistical evidence to support the conviction of
an individual for a violent crime is highly objectionable; it is less objectionable
130 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
51 The connection between accuracy and the use of statistical evidence is expounded in the next
section.
52 Behavioural scientists have shown that, when both statistical and non-statistical evidence is
available, human beings tend to ignore the former (see Kahneman and Tversky, above n. 15 and
also Kahneman, Slovic and Tversky, above n. 18). The point in the text above, however, refers to
cases in which only statistical evidence is available.
when it is used to allocate more police forces to patrol that neighbourhood; and it
is not objectionable at all when used to determine the budget of the Accident &
Emergency (A&E) department of the local hospital. If the evidence, or the inference
drawn from it, lacks some epistemic quality (as proponents of epistemic accounts
claim), and if epistemic principles should apply to a rational agent regardless of
whether he is a juror, police commander, or public health policy-maker, then the
differences between the attitudes to the same piece of evidence in each context
cannot be explained or justified.
It is of course possible to quibble with the neighbourhood example by arguing that
either the evidence or the inference is not really the same in all three contexts
(conviction, patrolling, and budgeting). For example, it could be argued that the
A&E case does not belong here because no individual is involved. However, in the
A&E case, a piece of statistical evidence which is clearly problematic when used in
court is used to draw an inference about a specific A&E department. It is true that
it does not involve a specific individual, but it is unclear why this difference should
have any bearing on the epistemic quality of the evidence or inference involved.
This is not to say that this difference is not important. On the contrary, as pointed
out by David Wasserman, ‘what is objectionable is the reliance on others’ conduct,
or the defendant’s past conduct, to infer his commission of a wrongful act’.53 The
point here, however, is that it is difficult to explain this important difference by
reference to an epistemic deficiency in the evidence or inference itself. It is
necessary somehow to integrate into the explanation a particular characteristic of
the legal context which makes the use of statistical evidence problematic only
when it is used to decide about an individual. The appropriateness of the use of a
certain piece of statistical evidence therefore strongly depends upon the purpose for
which the evidence is used. Epistemic accounts, with their general and context-insen-
sitive applicability, can neither explain nor justify this.
Epistemology, accuracy, and statistical evidence
The second generic problem with the epistemic analysis of statistical evidence is
that restricting the use of this evidence might itself be epistemologically deficient.
If successful, epistemic accounts will identify a quality which individualised
evidence has but statistical evidence lacks. However, this is only the first step. They
need to do more than this: they need to show that relying on information that
misses this quality is epistemologically unwarranted. In other words, they need to
establish that the use of statistical evidence makes the fact-finder’s epistemic
position inferior or undermines his knowledge of the facts.
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 131
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
53 Wasserman, above n. 27 at 942–3.
In contrast, Jonathan Koehler and Daniel Shaviro argue that using statistical
evidence, regardless of how problematic it might appear to be, improves the
accuracy of the legal fact-finding.54 They claim that:
[A] refusal to employ overtly probabilistic evidence and methods has a
cost, namely, increasing the probability of inaccurate verdicts. At
times, this cost may be worth incurring if the expected benefits are
sufficient; but it should not be denied or ignored.55
It is important first to clarify the way in which the term ‘accuracy’ is used in this
discussion.56 This term refers to the fact-finder’s ability to determine correctly
factual disputes rather than questions of law.57 Improving accuracy means
increasing the probability that the fact-finder’s findings match the facts as they
are in reality.58 It refers both to the probability of correctly determining any single
finding of fact and to the overall rate of correct findings of fact among all findings
of fact made by the court.59
Koehler and Shaviro’s paper has attracted various scholarly critiques, leading to a
lengthy and passionate debate.60 Some of the issues are not directly relevant here
(for example, the disagreement between the authors about statistical methods
132 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
54 Koehler and Shaviro, above n. 19.
55 Ibid. at 248–9. They insist that improvement occurs regardless of the specificity of the reference
class or the availability of other non-statistical evidence. See also Koehler, above n. 25 at 145. It
should also be noted that Koehler and Shaviro refer to both statistical evidence and statistical
methods (such as Bayes’ Theorem). However, this article focuses only on statistical evidence.
56 For Koehler and Shaviro’s definition, see Koehler and Shaviro, above n. 19 at 249–50.
57 Although the distinction between fact and law is constantly used in practice, it has been
repeatedly challenged by scholars. See, e.g., R. J. Allen and M. S. Pardo, ‘Facts in Law and Facts of
Law’ (2003) 7 E&P 153 and J. Jackson, ‘Questions of Fact and Questions of Law’ in W. Twining (ed.),
Facts in Law (Franz Steiner: Wiesbaden, 1983) 85. See also Zuckerman’s critique, above n. 37.
However, for the purpose of this article, even if deciding factual disputes requires normative
judgment, as Zuckerman persuasively argues, it is assumed that it is still possible to divide
propositions about which the court needs to decide into descriptive (about actual states of affairs)
and normative (about how the law should apply to these states of affairs). The term ‘accuracy’ here
refers to the former.
58 Koehler and Shaviro, above n. 19 at 249–50. It should be noted that the theory of truth on which
this view relies is correspondence rather than coherence. For a detailed introduction, see R. L.
Kirkham, Theories of Truth: A Critical Introduction (MIT Press: Cambridge Mass, 1992).
59 Koehler and Shaviro, above n. 19 at 258–9.
60 C. R. Callen, ‘Adjudication and the Appearance of Statistical Evidence’ (1991) 65 Tulane L Rev 457; R.
J. Allen, ‘On the Significance of Batting Averages and Strikeout Totals: A Clarification of the “Naked
Statistical Evidence” Debate, the Meaning of “Evidence” and the Requirement of Proof Beyond a
Reasonable Doubt’ (1991) 65 Tulane L Rev 1093. For responses, see D. Shaviro, ‘A Response to
Professor Allen’ (1991) 65 Tulane L Rev 1111; Koehler, above n. 25.
such as Bayes’ Theorem).61 Other issues are immaterial to this article.62 These issues
aside, it is worth noting the intuitive appeal of Koehler and Shaviro’s claim. In
general, most fact-finding procedures rely on the assumption that the more
good-quality evidence you have,63 the more likely you are to find the truth.64 The
idea that more evidence, statistical or not, improves the accuracy of factual deter-
mination is not unique to the legal context. The starting point should therefore be
that good-quality evidence should be used freely, unless it can be shown that there
is a persuasive reason not to do so. Koehler and Shaviro merely hold that this
general wisdom also applies to statistical evidence.
If evidence is needed to support the claim that using statistical evidence is
conducive to accuracy, we only have to look into many of our daily practices,
which assume that, everything else being equal, using statistical evidence will
enhance the chance of making accurate findings, in comparison with ignoring
such evidence. To take an additional example to those mentioned in the previous
section, mortgage applications are decided on the basis of the default rate of other
people who are similar to the applicant. A similar assumption appears not only in
predictions but also in diagnosis: if there is statistical evidence about a correlation
between certain symptoms and a certain disease, it is difficult to explain why such
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 133
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
61 Koehler, above n. 55 at 145, responding to Shaviro’s concession in Shaviro, above n. 60 at 1112. The
reason that this issue is irrelevant here is that this article focuses only on statistical evidence, not
statistical methods.
62 See, e.g., Callen’s comment that by referring to ‘overtly statistical evidence’, the authors make ‘a
misleading assumption that other evidence or inference is covertly statistical or probabilistic’
(Callen, above n. 60 at 458, fn 7). Even if Callen is right, the lack of such a distinction merely
strengthens the authors’ claim that statistical evidence improves accuracy (and see Koehler’s
response in Koehler, above n. 55 at 143).
63 Notably, ‘evidence’ is qualifiedwith ‘good-quality’. The reason for this qualification is that this
article considers principled objections to the use of statistical evidence, that is, problems that
will still arise even if the evidence is gathered, analysed and presented with the utmost
professionalism by experienced expert statisticians (see the text after n. 24). As notedthere, in
cases where practical problems undermine the probative value of the evidence, it is better to
address and rectify the practical problems than to ignore the evidence altogether. And see also
Koehler and Shaviro’s response to practical problems: Koehler and Shaviro, above n. 19 at
275–7.
64 It could also be argued that evidence with very little relevance should be ignored, since the cost of
its processing is greater than its contribution to fact-finding. For such an argument, see Wigmore’s
concept of ‘legal relevance’, discussed in Roberts and Zuckerman, above n. 17 at 103–4, and more
recently, R. Friedman, ‘Irrelevance, Minimal Relevance, and Meta-relevance’ (1997) 34 Houston L
Rev 55. But see Roberts and Zuckerman’s forceful critique of these concepts (above n. 17). In any
case, to enable reliance on such concepts when arguing that statistical evidence should be
restricted, it should first be shown that statistical evidence by definition has such little relevance
that its restriction is justified. As this article argues, it is hard to accept thatall statistical evidence
suffers from an epistemic deficiency (of no relevance, of little relevance or whatever).
evidence should be ignored when encountering a patient who suffers from those
symptoms.65
That the use of statistical evidence is conducive to accuracy is also part of almost
every philosophical interpretation of ‘probability’.66 Since this article focuses on
base-rate statistical evidence,67 it sits most comfortably within a frequentist inter-
pretation of probability, which understands probability as merely reporting the
frequency of a certain phenomenon within a well-defined population.68 Never-
theless, the view that using statistical evidence is conducive to accuracy is by no
means limited to that interpretation, since base-rate statistical evidence plays an
important role in almost any interpretation of probability. Propensity theories
understand probabilities as objective chances which are part of our non-determin-
istic world.69 However, even these theories rely on base-rate statistical evidence to
acquire knowledge of objective chances.70 Even among the various strands of
Bayesianism, most versions leave some room for base rates. True, a radical subjec-
tivist may allow an agent to ignore good data as long as the agent’s beliefs are
134 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
65 Note that this is a significantly weaker assumption than the more controversial thesis thatdoctors’
clinical examinations and personal judgments do not achieve any better results (or even achieve
worse results) than statistical diagnosis, which does nothing more than apply the relevant
statistical evidence to the individual case. No view is taken here on this thesis, which seems to
stand at the heart of the debate around evidence-based medicine. It is assumed only that even if
clinical examination is necessary and beneficial for accurate diagnosis, using the relevant
statistical evidence at some point in the process of diagnosis is better than ignoring it altogether.
Notably, Cohen seems to deny even this weak assumption, above n. 41. The assumption is further
discussed below.
66 The precise meaning of ‘probability’ has been at the centre of a long-running and detailed debate
in philosophy. Unfortunately, elaborating on the various interpretations would require lengthy
exposition and would divert this article towards other debates that are not relevant to its focus.
The above paragraph assumes substantial background in those areas and is directed only to those
who are interested in that debate. An effective, if not fully comprehensive, introduction to these
interpretations may be found in Roberts and Zuckerman, above n. 17 at 120–4. Gillies provides a
good detailed overview of the philosophical debate. See D. Gillies, Philosophical Theories of Probability
(Routledge: London, 2000). Interestingly, his book does not even mention Cohen’s ‘Baconian’
interpretation of probability, which attracted immense attention in legal scholarship. Perhaps
Gillies ignores Cohen’s interpretation because of its well-intended rejection of the axioms of
probability.
67 See text circa n. 19.
68 See Gillies, above n. 66 at ch. 5. His description focuses mainly on the work of Von Mises (see R. von
Mises, Probability, Statistics and Truth, 2nd edn (Hodge: London, 1939) and R. von Mises, Mathematical
Theory of Probability and Statistics (Academic Press: New York, 1964)).
69 See Gillies, above n. 66 at ch. 6. Gillies proposes a further development of this theory in ch. 7 of his
book, accompanied by a pluralist view which attaches different interpretations of probability to
different disciplines (ibid. at chs 8–9).
70 D. Lewis, ‘A Subjectivist’s Guide to Objective Chance’ in Philosophical Papers (Oxford University
Press: Oxford, 1980) 83.
coherent.71 However, more sophisticated developments of Bayesianism allow that,
in order to be rational, the agent’s beliefs should be constrained by his back-
ground knowledge.72 This knowledge includes any base rates which are available
to that agent. Statistical evidence therefore has an important role to play under
almost any philosophical interpretation of probability.73
Interestingly, the assumption that using statistical evidence is conducive to
accuracy also implies that to the extent that epistemology is concerned with
arriving at the truth, it is the restriction of the use of statistical evidence which
seems epistemologically deficient and therefore requires justification. Without
such justification, epistemic considerations actually require that the use of statis-
tical evidence be intensified rather than restricted (hereafter ‘the argument from
accuracy’).
Proponents of epistemic accounts might respond to the argument from accuracy
in two ways. The first is to challenge the connection between statistical evidence
and accuracy. One attempt to do so is particularly interesting because it directly
confronts the claim that using statistical evidence improves the accuracy of legal
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 135
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
71 For an overview of subjective Bayesianism, see Gillies, above n. 66 at ch. 4. For an excellent
criticism of its philosophical foundations, see M. Kaplan, DecisionTheory as Philosophy (Cambridge
University Press: Cambridge, 1998) chs 1–3. Radical subjective Bayesianism argues that an
agent’s beliefs are rational if and only if they would not allow a Dutch Book tobe made against
him or her. A Dutch Book is a set of bets in which the agent would lose whatever the outcomeof
the gamble (for the DutchBook as the central feature of subjective Bayesianism, see Gillies, above
n. 66 at 58–65). This view is problematic because according to it, an agent’s belief may be rational
even if he chooses completely to ignore good data for no sound reason. For example, consider an
agent who believes that the probability that a dice will roll six is 100 per cent and the probability
that it will roll any other number is zero. According to radical subjective Bayesian, his beliefs
may be rational even if he is aware of the fact that the number six has appeared among approxi-
mately one-sixth of the recent numerous throws of this dice (no DutchBook canbe made against
him because the agent will not always lose irrespective of the outcome). Moreover, the agent’s
beliefs comply with the Kolmogorov axioms of probability (Ramsey proved that avoiding the
Dutch Book is a necessary and sufficient condition for beliefs to conform to the Kolmogorov
axioms of probability and vice versa, see Gillies, above n. 66 at 64). Hence, radical subjective
Bayesianism is not a very attractive position.
72 Most notably, see Williamson’s recent account of objective Bayesianism (J. Williamson, Bayesian
Nets and Causality: Philosophical and Computational Foundations (Oxford University Press: Oxford, 2005)
ch. 5). Several other versions of Bayesianism (which Williamson titles ‘empirically-based
Bayesianism’) also accept this type of constraint on the agent’s beliefs (see ibid.).
73 Notably, the brief discussion above has not included the logical interpretation (see Gillies, above n.
66 at ch. 3). This interpretation suffers from various paradoxes which render it unsustainable (see
Gillies, above n. 66 at 37–49). Moreover, it has been repeatedly criticised for lacking detail about how
probabilities are determined and become known (see F. P. Ramsey, ‘Truth and Probability’ in R. B.
Braithwaite (ed.), Foundations of Mathematics (Kegan Paul Trench Trèubner: London, 1931) 161; Gillies,
above n. 66 at 52–3; Williamson, above n. 72 at §5.4, 79–80). As a result of this deficiency, it is hard to
predict its approach to empirical knowledge in general and to statistical evidence in particular.
fact-finding. In his attempt to explain away Jonathan Cohen’s gatecrasher
paradox,74 David Kaye argues that the lack of other pieces of evidence is itself
an additional piece of evidence.75 Kaye then adopts Laurence Tribe’s idea that
statistical evidence should be restricted (they both support a restriction of insuffi-
ciency) in order to create an ‘incentive for plaintiffs to do more than establish the
background statistics’.76 According to this argument, imposing legal liability
based on statistical evidence alone (for example, that xper cent of the spectators
did not pay for admission) might in fact reduce accuracy because it ignores an
additional implicit piece of evidence, namely, the fact that the only evidence
which the claimant could come up with was statistical. If this objection is
successful, it may show that using statistical evidence does not actually improve
the accuracy of legal fact-finding.
Yet Kaye’s argument proves both too little and too much.77 It proves too little
because there are cases in which it simply does not work. Even if Kaye’s argument
holds, one might respond that this additional implicit piece of evidence should be
combined with the statistical evidence (for example, using Bayes’ Theorem). If the
statistical evidence contains figures which are high enough (for example, 90 or
even 99 per cent),78 then the combined probability may well be above 50 per cent
(assuming that the civil standard of proof can be reduced to a threshold of proba-
bility of more than 50 per cent).79 Therefore, Kaye’s argument is able to account for
only some of the cases. The higher the figures involved, the less likely the
argument is to succeed. Secondly, and most importantly, Kaye’s argument proves
too much because it can be applied to statistical and individualised evidence
alike.80 Recall the red and blue buses scenario.81 ‘If the accident took place at a busy
intersection, why did the plaintiff not produce two or three witnesses who were
there?’82 Kaye needs to explain what quality statistical evidence lacks which makes
136 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
74 For a description of this paradox, see the text after n. 12.
75 D. H. Kaye, ‘The Paradox of the Gatecrasher and Other Stories’ (1979) Arizona State LJ 101.
76 Tribe, above n. 10 at 1349. Repeated in Kaye, above n. 75 at 106.
77 The discussion here is based on Brook, above n. 14 at 324–9.
78 For a detailed calculation based on the example of 95 per cent, see Brook, above n.14 at 327, fn 151.
79 For a good overview of this issue, see M. Redmayne, ‘Standards of Proof in Civil Litigation’ (1999) 62
MLR 167.
80 This point was independently noted by Brook, above n. 14 at 324–6 and Schoeman, above n. 38 at
181–2.
81 See the text after n. 15.
82 Brook, above n. 14 at 325. Kaye’s explanation is focused on cases where the statistical evidence is
unjustifiably used as a sole basis for the litigation, that is, where we have reason to expect that more
evidence exists, but that it is being held back (thanks are due to the anonymous reviewer for
emphasising this point). Although this raises an obvious difficulty in distinguishing between such
cases and other cases, where there is no reason to suspect that further evidence is being held back,
the more important point is that the same line of reasoning could equally be applied to individu-
all statistical evidence, and only statistical evidence, always insufficient. This
objection to the argument from accuracy implicitly relies upon a certain
difference in quality. However, identifying such a qualitative difference is
difficult.83 Even Tribe, on whose approach Kaye’s response to the gatecrasher
paradox is based, explicitly rejects the existence of any difference in kind between
statistical and individualised evidence.84
More generally, none of the epistemic accounts has provided any reason why
relying on statistical evidence reduces the accuracy of legal fact-finding. On the
contrary, they seem to admit that this would not be the case because they
emphasise that the epistemic quality of evidence or of an inference is not a
function of its accuracy alone and there are other epistemic considerations that
matter. Thomson argues that high probability is not a sufficient condition for
justification, and by that she seems to imply that using statistical evidence raises
the probability of the decision being correct, even though she considers that it
would be less justified.85 Cohen argues that statistical evidence lacks weight (and
hence fails to raise the ‘Baconian probability’),86 but he does not deny that statis-
tical evidence raises the probability that the proposition is true (the ‘Pascalian
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 137
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
alised evidence. There could just as easily be cases where only one piece of individualised evidence,
say an eyewitness testimony, is available, and we have reason to believe that other pieces of either
statistical or individualised evidence are being held back. Yet even in these cases, we do not regard
this suspicion as a problem in the evidence itself, or in the inferences made from it. Hence, while
the possible existence of other evidence might be important for other purposes, it is not partic-
ularly helpful in distinguishing between statistical and individualised evidence and justifying
differential treatment of the former.
83 See the various attempts scrutinised in the sources mentioned in above n. 38.
84 ‘I am, of course, aware that all factual evidence is ultimately “statistical” and all legal proof
ultimately “probabilistic”’ (Tribe, above n. 10 at 1330, fn 2, emphasis original).
85 Thomson, above n. 30 at 207.
86 In his groundbreaking work,Cohen puts forward an inductive interpretation of probability, which
he links to Francis Bacon’s concept of probability (Cohen, above n. 12 at 35). One of the main
differences between Cohen’s ‘Baconian’ interpretation of inductive probability and other interpre-
tations of probability (which Cohen terms ‘Pascalian’, after Blaise Pascal) is Cohen’s well-intended
rejection of the Kolmogorov axioms of probability. In particular, he rejects an important
consequence of the Kolmogorov axioms, the conventional principle of negation, and formulates a
new inductive version of this principle (ibid. at 177–81). Cohen’s general theory of Baconian
inductive probability drew attention and criticism in both philosophy and law. For responses
within philosophy, see H. Kyburg, ‘Book Review: L. Jonathan Cohen, The Probable and the Provable
(1980) 14 Nous 623; S. Stoljar, ‘Book Review: L. Jonathan Cohen, The Probable and the Provable’ (1981) 90
Philosophical Review 457; F. Schoeman, ‘Cohen on Inductive Probability and the Law of Evidence’
(1987) 54 Philosophy of Science 76 and Cohen’s response in L. J. Cohen, ‘On Analyzing theStandards of
Forensic Evidence: A Reply to Schoeman’ (1987) 54 Philosophy of Science 92. For responses within law,
see C. G. Wagner, ‘Book Review: The Probable and the Provable by L. Jonathan Cohen’ (1979) Duke LJ
1071; D. A. Schum, ‘A Review of a Case against Blaise Pascal and his Heirs’ (1979) 77 Michigan L Rev
446; the exchange between Kaye and Cohen, Kaye, above n. 75; and Cohen’s response, L. J. Cohen,
‘Subjective Probability and the Paradox of the Gatecrasher’ (1981) Arizona State LJ 627.
probability’, in Cohen’s terminology).87 Stein supports a requirement of case-speci-
ficity, but accepts that statistical evidence (or, in his terminology, ‘general
evidence’)88 increases the probability of finding the truth, despite carrying too
little weight.89 Mary Dant explicitly accepts that statistical evidence is not worse
than any other type of evidence when it comes to the ability to lead the fact-finder
to the truth.90 That most of the major epistemic accounts do not deny that the use
of statistical evidence improves accuracy is indicative of the unpromising
prospects of such a strategy for responding to the argument from accuracy.
The second way in which proponents of epistemic accounts could respond to the
argument from accuracy, according to which epistemic considerations require
that the use of statistical evidence should be intensified rather than restricted, is
to challenge the connection between accuracy and the fact-finder’s knowledge of
the facts. Knowledge is commonly conceived of as requiring some sort of justifi-
cation.91 It could be argued that having knowledge of the facts cannot be reduced to
the probability of getting a correct answer, but knowledge is (also) dependent on
having a proper justification, which proponents of epistemic accounts suggest as
138 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
87 This is demonstrated in the gatecrasher paradox where Cohen insists that the statistical evidence
increases the mathematical probability to a level above the required standard of proof in civil
proceedings (see Cohen, above n. 12 at 75).
88 For the relation between the categories of ‘general evidence’ and ‘statistical evidence’, see above n.
42.
89 This is nicely exemplified in Stein’s solution to the red and blue buses scenario: ‘[a]lthough it is
logically possible to ascribe [the claimant’s] allegation a 0.8 probability—in the sense that systematic
rulings in favour of the claimants in similar cases would produce, in the long run, eighty correct decisions out of
one hundred—this probability ascription would not carry much weight’ (Stein, above n. 31 at 85,
emphasis added).
90 Dant, above n. 32 at 58.
91 The traditional analysis of knowledge is discussed by Plato in the Theaetetus, as the third definition
which Socrates discusses and rejects (Plato, Theaetetus, 201d-210a in H. Cairns and E. Hamilton, The
Collected Dialogues of Plato: Including the Letters (Princeton University Press: Princeton, 1961)), though
it is questionable whether or not Plato himself accepts this definition of knowledge (see M.
Burnyeat, The Theaetetus of Plato (Hacket Publishing: Indianapolis, 1990) 234–41: ‘[i]t is not to be
taken for granted that there is a definite answer to the question “What does Plato think knowledge
is?”’ (ibid. at 235)). The traditional analysis argues that the agent knows pif and only if three
conditions are met: (1) pis true; (2) the agent believes that pis true; and (3) the agent is justified in
believing that pis true. As for the relation between the truth condition and the justification
condition, Plato makes an analogy to Daedalus’ statueswhich were so good thatthey looked alive.
Socrates says that these statues, like true beliefs, have to be tied with chains (justifications) (see
Plato, Dialogues,Meno, 97d–98a). It should be noted, however, that this traditional account came to
the centre of intensive controversy as a result of Gettier’s counter-examples (E. L. Gettier, ‘Is
Justified True Belief Knowledge?’ (1963) 23 Analysis 121). Gettier cites examples in which the three
conditions are satisfied and yet no knowledge is achieved. These examples show that the three
conditions are insufficient to define knowledge. In the text above, the justification condition is
assumed to include whatever additional conditions are necessary to overcome the Gettier
problems.
having the appropriate causal connection, weight, case-specificity, or another
property.92 The fact that knowledge requires something more than a high proba-
bility might have intuitive appeal. Consider the following hypothetical example.
Alice, a City trader, has access to software which predicts the price of shares the
next day. These predictions are correct 95 per cent of the time. However, nobody
understands how this software works (perhaps the source code was lost, there is
no documentation and the programmer fled to Namibia). Some people might have
a strong intuition that Alice’s belief that the price of the share tomorrow will be as
predicted by the software lacks a proper justification, and that hence she does not
know what the price of the share will be tomorrow.93
However, challenging the connection between accuracy and knowledge is more
difficult than epistemic accounts seem to assume. Proponents of epistemic
accounts could attempt to challenge the connection between accuracy and
knowledge in two ways. They could argue that the accuracy of the evidence is not
the only important element in securing the fact-finder’s knowledge of the facts
(the weaker claim).94 Alternatively, they could argue that the accuracy of the
evidence is completely irrelevant to the fact-finder’s knowledge of the facts (the
stronger claim). If they adopt the stronger claim, then their conception of episte-
mology can be questioned. Can it be argued seriously that accuracy does not
matter from an epistemic perspective? And if this is the case, does it matter for
other non-epistemic reasons? If so, what are they and how do they relate to the
epistemic reasons? Furthermore, if the probability of getting the correct answer is
not relevant for the fact-finder’s knowledge, why should we be interested in
securing such knowledge? An epistemology in which finding the truth does not
matter seems awkward and unattractive.
If proponents of epistemic accounts commit themselves only to the weaker claim
thattheaccuracyoftheevidenceisnottheonlyimportantelementinsecuring
the fact-finder’s knowledge of the facts, they still have to identify another
criterion (such as weight) and explain why statistical evidence fails to satisfy it.
Most importantly, they need to explain why this other criterion which statistical
evidence allegedly fails to satisfy is more important than the criterion of
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 139
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
92 See, e.g., Thomson, above n. 30 at 208.
93 Clearly, others might have the opposite intuition, according to which the impressive record of the
software in the past does suffice to justify Alice’s belief in the truth of its future predictions. The
objection to epistemic accounts which appears in the next paragraph should be even more
appealing to those with this opposite intuition.
94 A similar approach is taken by Chisholm (see R. M. Chisholm, ‘The Indispensability of Internal
Justification’ (1988) 74 Synthese 285).
accuracy that statistical evidence does satisfy. They also need to say something
about the appropriate balance between accuracy and their suggested criterion.
How much accuracy should be sacrificed to maintain this other criterion of
causation, weight, case-specificity, or another property? Insisting on this
additional criterion even at the price of lesser accuracy would mean that more
decisions would be factually incorrect. One may rightly be concerned about how
many additional mistaken decisions might result from this insistence on the
additional criterion of causation, weight, case-specificity and so on. The
challenge for epistemic accounts is therefore much greater than merely identi-
fying an epistemic difference between statistical and individualised evidence. To
date, existing epistemic accounts have been occupied with trying to establish a
defensible epistemic difference between the two types of evidence, or between
the inferences which are drawn from these two types. They have neglected the
need to identify the extent to which accuracy can be sacrificed without
rendering legal fact-finding so inaccurate that the entire enterprise risks
becoming epistemologically deficient.
To conclude, proponents of epistemic accounts might find it difficult to refute the
argument that epistemic considerations require that the use of statistical
evidence in court should be intensified rather than restricted. Challenging the
connection between statistical evidence and accuracy is difficult, as is indicated
by the reluctance of the existing accounts to do so. It is also difficult to challenge
the connection between accuracy and the fact-finder’s knowledge of the facts.
Hence, it is hard to accept that the use of statistical evidence is epistemologically
unwarranted. If epistemic considerations have anything to do with the issue of
statistical evidence, they probably require that the use of statistical evidence be
intensified rather than restricted. This is the second generic problem in any
epistemic account which attempts to establish the epistemic inferiority of statis-
tical evidence.
Epistemology and policy considerations
The shortcomings of epistemic accounts raise a principled question about the
connection between epistemology and law. Even if epistemic accounts can
support the general applicability of their arguments and explain why epistemic
considerations require that the use of statistical evidence in court be restricted
rather than intensified, these accounts face another generic difficulty. Epistemic
accounts have to explain why the identified epistemic quality is important for legal
purposes.
Establishing the importance of an epistemic quality for legal purposes is relatively
easy when the epistemic quality improves the accuracy of legal fact-finding. This is
140 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
because the accuracy of legal fact-finding is important for most theories of law,
since it lowers the risk of mistaken acquittals and mistaken convictions. In
criminal law, for instance, accuracy is important from both consequentialist and
deontologist perspectives. In terms of consequences, mistaken acquittals are
clearly undesirable because they undermine deterrence, prevention and rehabili-
tation. Mistaken convictions undermine consequentialist goals, such as the
prevention of the actual criminal’s future crimes and the deterrence effect on the
actual criminal and on everyone else who knows the truth about the case.
Accuracy is also important for deontologist theories which emphasise giving the
offender what he deserves, not more and not less. Mistaken acquittals mean that
the actual criminal does not receive the punishment he deserves and mistaken
convictions mean that an innocent person receives a punishment he does not
deserve. If statistical evidence lacked an epistemic quality related to accuracy, it
would be relatively straightforward for epistemic accounts to explain why the law
should give preference to evidence which possesses this quality.
However, as shown in the previous section, epistemic accounts usually admit that
statistical evidence tends to improve accuracy.95 Instead, proponents of epistemic
accounts resort to other epistemic considerations, such as the lack of justification,
weight, causal connection, and so on, as reasons for preferring individualised
evidence.96 Even if statistical evidence lacks such an epistemic quality, it is still
necessary to show that this quality is important for legal purposes. In addition to
identifying the epistemic quality which statistical evidence lacks, epistemic
accounts need to explain why the law should prefer evidence which possesses this
epistemic quality.
In principle, it might be possible to justify differential treatment of statistical
evidence in court by alluding to other moral or social goals. The argument would
run that there is a further moral or social goal which is sufficiently important to
justify the reduction of accuracy. It is important to remember that in the criminal
context, a reduction of accuracy means convicting more innocents or acquitting
more guilty people, or both. Such a justification will thus be successful if it is able
to establish that the benefit to the other goal from restricting the use of statistical
evidence in court is worth the price of that restriction. In contrast, epistemic
accounts merely identify a certain epistemic quality that is necessary for
achieving an epistemic goal (for example, securing knowledge). In order to justify
differential treatment of statistical evidence in court, epistemic accounts should
supplement their epistemic analysis with an additional non-epistemic argument,
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 141
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
95 See the text after n. 84.
96 For a detailed examination of the specific epistemic accounts, see the various sources in n. 38.
which would connect the epistemic goal, allegedly achieved by restricting the use
of statistical evidence, to a moral or social goal, which is sufficiently important to
justify the reduction of accuracy. Without such an additional non-epistemic
argument, it remains unclear why more innocents should be convicted, or more
guilty people acquitted, just in order to achieve an abstract epistemic goal,
especially if that goal cannot be translated into any moral or social goal.
Therefore, the epistemic analysis alone cannot explain why the law should insist
on a certain epistemic quality, despite the fact that this epistemic quality comes at
the price of damaging the accuracy of legal fact-finding.
It might be argued that this point conflates epistemic arguments and their legal
implications. Proponents of epistemic accounts might concede that more work
is needed before such accounts can result in detailed guidance for their use in
law, but might reply that such technical work should be left for professionals (for
example, law commissioners). The point is, however, that the missing elements
in the epistemic discourse are not merely the technical details about how to
translate general epistemic principles into detailed legislation or law reforms.
There is a significant gap between epistemic arguments and policy-making in
general, because it is unclear why the epistemic quality which the statistical
evidence allegedly lacks should matter for legal fact-finding. Before epistemic
accounts can yield any guidance for legislators or judges, they need to be
grounded in other types of non-epistemic justification. Until epistemic accounts
are supplemented with such non-epistemic justification, they will be insuffi-
cient to yield any recommendation in regard to how statistical evidence should
be treated in court.
In response to the last point, or to any of the other generic problems of epistemic
accounts identified in this article, one might retort that an account which aims
to justify the restriction of statistical evidence in court can be both epistemic
and non-epistemic, or at least have twoelements,onepurelyepistemicand
another non-epistemic. However, the question still remains as to what role the
purely epistemic element has in such a hybrid account. As mentioned above, if it
is responsible merely for providing a definition or clarification of the difference
between statistical evidence and individualised evidence, then it should be
welcomed. However, if the purely epistemic element is supposed to play a justifi-
catory role, then the same objections made here against the epistemic accounts
can also be applied to this purely epistemic element.
The point that this article has sought to establish is that epistemic accounts (or
purely epistemic elements in hybrid accounts) cannot provide a justification for
restricting the use of statistical evidence in court. It has been argued here that the
142 THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE
epistemic analysis of statistical evidence as such suffers from several generic
problems which make this direction futile. Accordingly, if there is a justification
for restricting the use of statistical evidence in court, it does not lie in episte-
mology; it has to lie elsewhere.
THE INTERNATIONAL JOURNAL OF EVIDENCE & PROOF 143
THE EPISTEMOLOGY OF STATISTICAL EVIDENCE

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT