A Dark and Stormy Night: The Mystery of the Missing Science in Fingerprint Identification

IN MAY of 2004, the FBI issued an unprecedented apology to Brandon Mayfield, the Portland, Oregon lawyer who was tied to the Madrid train bombings via an erroneous fingerprint identification.1 The print in question had been positively identified as Mayfield's by three highly qualified FBI examiners

and confirmed by Mayfield's own expert. Yet it was ultimately shown to belong to a far more plausible suspect, an Algerian named Ouhnane Daoud, who was linked to the bombings through other evidence.2 Especially perplexing to the casual observer was the fact that Mr. Mayfield was arrested even though the fingerprint match had been disputed by police in Spain. An even greater mystery, however, was raised by Dr. Simon Cole, Professor of Criminology at U.C. Irvine, who was quoted in news coverage of the apology as stating that fingerprinting methodology has never been scientifically validated.3

This article argues that fingerprint methodology should become more science-based in the future because the adversary system is not structurally suited to prevent or correct the types of mistakes that led to the Mayfield fiasco.4 It is already slowly evolving in that direction, in response both to the pressures created by those same mistakes, and to the evolving relationship between courts and scientific evidence in general. Additionally, defense attorneys should adopt a more nuanced approach to challenging fingerprints than they have been doing if they want judges to take them seriously.

I. Setting the Stage: Colonial India to Illinois

Fingerprints have a deep history of use for personal identification. A Persian historian wrote in 1303, "Experience shows that no two individuals have fingers precisely alike."5 This statement illustrates two highly significant aspects of fingerprints: anyone can see their patterns without special training or instruments, and there is strong intuitive appeal in the conclusion that they are unique to the individual. This section considers how the history of fingerprint identification in Western forensics shaped its current status in the U.S.

A. Record Keeping in the British Empire

The British Empire first became interested in forensic applications of fingerprints in the 1800's, in connection with identifying repeat offenders. William Hershel began to collect fingerprints as a hobby in 1859, and while serving as a colonial law enforcement administrator in India in the 1870's he had the opportunity to employ his experience attempting to identify recidivist criminals.6 The obvious drawback to this system in the pre-computer age was the prohibitively labor-intensive nature of comparing each new print set with numerous previous sets to try and find a match.7 At about the same time, fellow British colonial Henry Faulds observed that potters left identifiable fingerprints on their clay wares, and he was inspired to consider their potential for personal identification. Faulds was a physician in a hospital in Tokyo when he reportedly solved a petty crime using "greasy finger-marks." Faulds's early role is significant because he raised concerns about the potential for false identification. He examined thousands of sets of ten prints, and became convinced no two were alike, but he also believed systematic study should be used to determine whether single prints were also unique. Notably, the rest of the fingerprint community was less enthused about undertaking such a study.

When you glance at your own fingerprints, you see the first level of detail of the pattern, i.e., whether it is a loop, an arch, a whorl, etc. The idea of matching sets of ten prints to already-convicted criminals for the purpose of identifying repeat offenders has great intuitive appeal, particularly when you look closer and begin to pick out the second level of detail, the way individual ridges begin and end and bifurcate to create a wealth of easily identifiable features. Two questions in the current debate are the most fundamental ones addressed by Faulds and Hershel: (1) Is every print unique? and (2) Do prints change over time? Both of the esteemed Englishmen convinced themselves through personal observation that fingerprints remain stable over time. Concerning the first question, however, the science community of their day closed ranks around the conclusion that every print is unique, although it is virtually impossible to demonstrate. If I look at 100 people's prints every year for a decade, and never see a change (except scarring), there's no reason to suppose that adding another 1,000 or 100,000 people will yield a different result. Fingerprints change or they don't, yet how does one conclusively "prove" that no two are alike, without physically comparing every print on the planet?

Before we turn to how fingerprints gained acceptance in U.S. courts, it will be useful to consider the first documented case in British law where fingerprints were used as evidence in a murder trial. In 1898, in Bengal, a man's former servant was released from prison, and began spending large sums of money whose source he could not explain. The former master was found murdered, and a single bloody fingerprint left at the scene was introduced into evidence.8 Since there were no "fingerprint experts" yet, the print was examined by the judge and assessors, who convinced themselves it must belong to the suspect. First, to prove absolutely that the print was unique would have required fingerprinting every other person on earth. second, the higher the quality of partial print(s) taken from crime scenes, in size and clarity, the higher the probability that some obvious discrepancy will be found, leading to exclusion-a positive non-match. The poorer the quality of the print in evidence, the greater the subjectivity of any human assessment, and the greater the chance of erroneous conclusions of a positive "match." More ominously, if you believe fingerprints are unique, the more different-from-all-others you believe each one is, the less information you think you need for a positive match. This creates a powerful disincentive to delve further into the subject of uniqueness. Faulds, a physician, campaigned long and unsuccessfully for systematic verification of just how unique human fingerprints may be. For his trouble, he was effectively shut out of the fingerprint community.9

B. Rolling Over in the States

It didn't take long for the new science of fingerprint identification to cross the Atlantic. In 1911, the Supreme Court of Illinois became the first state high court to declare evidence derived from the technique of matching fingerprints to be admissible, in People v. Jennings.10 This landmark case is analyzed in detail by Simon Cole in a paper focused on "the nitty-gritty of expert [fingerprint] testimony."11 In Jennings, no less than five fingerprint examiners, all of whom were trained by a Scotland Yard detective, agreed that fingerprints left at a murder scene "matched" the defendant's rolled prints. One of them even testified, in response to a question as to whether it was his "opinion" that the prints were a match, "I am positive. It is not my opinion."12 This response foreshadowed an approach to fingerprint testimony which has prevailed for ninety-five plus years. Absolute certainty is presumed, thereby implying infallibility of the expert testimony.

The defense argued that the expert's testimony was unnecessary, on the grounds that "the jury can take the magnifying glass and then come to a conclusion as to whether or not these photographs are identical."13 The court concluded otherwise, stating flatly that, "the classification of finger print impressions and their method of identification is a science requiring study."14 This case also set the stage for another feature of the present debate concerning fingerprint evidence, specifically, whether or not it is a "science." As Cole notes, evidence that convincingly implies that "only one person could be the source of a piece of trace evidence" is an enormously powerful tool for the prosecution.15

Shortly after Jennings, a New York case illustrated the power of fingerprint evidence under this approach. In People v. Crispi, a fingerprint was the only evidence against a defendant whose family provided an alibi, contrasted with Jennings, where the fingerprint supplemented strong circumstantial evidence.16 As in Jennings, the defense attacked the need for expertise, but the court disagreed, calling the technique a "new science."17 As Cole observes, "fingerprint evidence benefited from its position on the cusp between scientific and lay knowledge ... . A fingerprint ... was commonplace enough to be readily understood by the jury, yet sufficiently esoteric to require expert interpretation."18 Crispi's attorney challenged the fingerprint expert by pointing out a discrepancy between the latent and rolled prints, to no avail. The expert declared the two prints "identical," explaining, "[y]ou must take into consideration that the prints may be under different conditions, more pressure being used."19 This illustrates a feature of the fingerprint art which remains true today, that such an expert "walks a thin line between transparency and opacity, between science and common sense."20 In theory the Crispi jurors could have disregarded the fingerprint testimony, but in fact they probably "lacked the epistemological authority for doing so."21 Following the fingerprint testimony, the hapless Crispi pled guilty, ending the trial.

C. The Chimera: Is it Science, Art, or Something Else Entirely?

One further aspect of fingerprint identification is pivotal to the present status. Early practitioners "developed a novel occupational norm: unanimity."22 For example, a 1917 instruction manual asserts that "[t]he testimony of a finger print expert is not subject to contradiction by another finger print expert."23 In 1919, another author suggested that examiners explain their testimony as follows: "The finger print expert has only facts to consider; he reports simply what he finds. The lines of identification are either there or they are absent."24 Furthermore, "[n]o matter how may finger print experts may be [involved in an identification], their verdict must be the same."25 Cole makes several good points about this choice to insist upon agreement in the fledgling fingerprint community.26 First, (to the extent it is followed in practice) it limits a practitioner to the role of technician rather than scientist and eliminates (in theory) the possibility of claiming more than can actually be seen in a latent print. If an examiner claimed to see a feature, he must be sure others with similar training will see it, too. This distinguished the new field's practitioners from various other "scientific experts" such as psychiatrists and handwriting experts where a high degree of subjectivity meant it was the norm for a defendant to find an "expert" who would disagree with the prosecution's "expert." "The skill of the fingerprint expert lay in restraint, in not overreaching, [i.e.] in no longer performing miracles." In addition however, it virtually eliminated the possibility of dueling experts, which was the norm in other fields. The upside for examiners was an unchallengeable aura of respectability (if not infallibility) on the witness stand. The downside for the justice system has been to virtually eliminate the adversarial role of defense attorneys in challenging fingerprint evidence. As Cole notes, "the record is almost devoid of cases with defense experts challenging the conclusions of the state's experts."28

To understand why this strategy has been so successful, imagine an examiner, who is usually a law-enforcement professional, trained exclusively within the community. Training has always been by apprenticeship to an experienced examiner. Practitioners are a self-selected group willing to pledge themselves to the proposition that, once mastered, their technique will render them infallible. Standards of practice are up to individual organizations to set; there is no national uniformity. There cannot be dissent from within, nor competition among agencies, so there is no effective pressure for growth or change. Nor is there motivation for the growth of a competing community outside of law enforcement since the justice system is the sole consumer. Nearly all fingerprint evidence arises from criminal, not civil suits, so there is no market for a competing private training ground for examiners. While the defense bar hires examiners, they are retired law-enforcement officers, who are not in a position to change the system.

Defense lawyers also face an uphill battle challenging fingerprint identifications. First, if they are convinced such evidence is virtually unassailable, they won't waste limited resources challenging it. Even when the defense hires an "independent" expert to verify a match, that person will approach the print knowing that another member of the (infallible) team has already declared it a match. This would be effective to challenge outright fraud,29 but has slim chance of detecting honest error, since the honest examiner who errs presumably will have declared a match where the truth is a close call.

Further, it is logically inevitable that once infallibility is assumed, honest errors can result only from incompetence. In a close call, a second examiner may be loath to declare his fellow examiner incompetent on the witness stand. Although he is, in a sense, being paid to find errors, mistaken identifications usually arise from subtle ambiguities in print features, not clear errors. In theory, defense fingerprint experts could get on the stand and trash their own field by stressing its failings, but this has not happened. These early choices among fingerprint experts led to a system where their testimony has become virtually unchallengeable.

D. The Frye Standard for Admissibility of Expert Testimony

Before addressing the modern, post-Daubert era, it is important to understand the standard which prevailed for seventy years regarding admissibility of expert testimony. That standard was set in 1923 in Frye v. United States.30 The testimony in question in Frye concerned a crude form of lie detector based on blood pressure. The defendant, who had been convicted of murder, sought to exonerate himself using the results of a test he had undergone. Upholding its exclusion, the Court of Customs Appeals famously wrote,

Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while courts will go a long way in admitting expert testimony deduced from a wellrecognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.31

It is notable that, unlike most biologybased forensic techniques such as lie detection, the fingerprint identification "field" is the technique. Physicians and biologists are not recruited to be examiners; police officers are. "General acceptance" is turned on its head if one attempts to apply it to fingerprints, because it was never a new technique in an established field; it has always been a field unto itself. Under the Frye standard, fingerprint evidence flourished and grew to be the archetypal forensic science. By the end of the 1930's all but five states had accepted it.32 Indeed, when genetic identification evidence was a newcomer to the courtroom, proponents purposefully called it "DNA fingerprinting" in order to invoke the "gold standard" status of the granddaddy of forensic techniques.33

II. Expert Testimony Under Daubert/Kumho and FRE 104

The Supreme Court's landmark opinion in Daubert was long in coming. The specific question the Court addressed was whether the Federal Rules of Evidence had superseded the Frye standard for admissibility of expert scientific testimony, and if so, what that should mean.34 The Rules had taken effect in 1975, nearly two decades earlier. The Court noted in its opinion that "the debates over Frye are such a well-established part of the academic landscape that the distinct term -, "Fryeologist" - has been advanced to describe those who take part."35

A. Daubert v. Merrell Dow Pharmaceuticals, Inc.

Daubert involved medical testimony regarding whether the drug Bendectin caused birth defects in the offspring of mothers who took it during pregnancy. Defendant Merrel Dow submitted into evidence thirty epidemiologic studies, all of which failed to link the drug to birth defects.36 The plaintiffs' evidence was impressive. They proposed to put on the witness stand eight well-credentialed scientists who had conducted studies supporting the link. These consisted of statistical re-interpretations of the epidemiological studies as well as test-tube and animal studies showing teratogenic properties of the drug. The district court had, however, excluded the plaintiffs' evidence, and granted defendants' motion for summary judgment.

Because Daubert was specifically a medical case, the Court's holding initially applied only to scientific testimony until it was extended by Kumho. Twenty-two amicus briefs were submitted, most notably one from the American Association for the Advancement of Science and the National Academy of Sciences, upon which the Court relied substantially in its opinion.37 The amici were concerned about the lax standards of admission of testimony purporting to be "scientific." They urged the Court "to uphold the broad authority of courts to exclude putatively scientific evidence that does not, according to the standards applied by the scientific community, have the earmarks of scientific reliability."

The relevant Rules identified by the Court were 401, 402, 702, and 104(a).38 Rule 402 stated at the time: "All relevant evidence is admissible, except as otherwise provided by the Constitution of the United States, by Act of Congress, by these rules, or by other rules prescribed by the Supreme Court pursuant to statutory authority. Evidence which is not relevant is not admissible." Rule 401 defines "relevant evidence" liberally as including whatever has "any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence." Rule 702, governing expert testimony, provides: "If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise." The Daubert Court called the Frye standard "rigid" and "at odds with the liberal thrust of the Federal Rules and their general approach of relaxing the traditional barriers to opinion testimony," then declared it to be superseded by the Federal Rules.

The Court next proceeded to interpret the Federal Rules of Evidence, declaring first that a trial judge "must determine at the outset, pursuant to Rule 104(a), whether the expert is proposing to testify to (1) scientific knowledge that (2) will assist the trier of fact to understand or determine a fact in issue."39 The Court explicitly equated evidentiary reliability based on trustworthiness with scientific validity, i.e., "does the principle support what it purports to show?"40 The opinion modestly states, "we do not presume to set out a definitive checklist or test. But some general observations are appropriate."41 These "observations," were, of course, quickly enshrined as the "Daubert factors" in legal scholarship and case law:

Whether a given theory or technique "can be (and has been) tested." The Court explicitly refers to scientific hypothesis testing in explaining this factor.

Whether a theory or technique has been "subjected to peer review and publication." For this factor, the Court noted "submission to the scrutiny of the scientific community . . . increases the likelihood that substantive flaws in methodology will be detected."

"[I]n the case of a particular scientific technique, the court ordinarily should consider the known or potential rate of error, and the existence and maintenance of standards controlling the technique's operation."42

"Finally, general acceptance can yet have a bearing on the inquiry." In preserving a vestige of Frye, the Court noted that where a "known" technique has not attracted much support in the relevant scientific community, it "may properly be viewed with skepticism."43

Given the heavily bio-medical-science nature of the evidence at issue in Daubert, it is hardly startling that the factors derived are not an especially good fit for testimony based on technical expertise, including fingerprint identification. This led to considerable uncertainty regarding a question posed by the Daubert dissent: Whether "all of this dicta" applied as well to "an expert seeking to testify on the basis of 'technical or other specialized knowledge.'"44 The Court finally addressed this question in 1999, over a decade later.

B. Kumho Tire Company, Ltd. v. Carmichael

Kumho involved a fatal car accident following a tire blow-out, in which plaintiffs alleged that the tires were defective.45 Plaintiffs initial complaint relied heavily on an affidavit from an expert in tire failure analysis, who employed a highly technical methodology to conclude that the tire had not been abused and must therefore have been defective. The district court excluded his testimony and granted defendants' motion for summary judgment, finding that none of the Daubert factors supported the reliability of the tire expert's methods. The Eleventh Circuit reversed, concluding the expert's testimony was not within the scope of Daubert. The Supreme Court granted certiori to address the question left open by Daubert, and declared unequivocally that "it applies to all expert testimony."46

Although Kumho did not involve a pure science, the issues raised were not far removed from the core of Daubert, because the underlying field of tire analysis is an established engineering discipline. As the Court noted, engineering is applied science and science frequently relies on engineered equipment. The expert in question had a Masters degree in engineering and ten years of experience at Michelin. The validity of his field was not at issue, nor was his personal expertise. The decision to exclude his testimony turned on the fact that his method was apparently personal to him and not established in the wider community. Additionally, "[t]he relevant issue was whether the expert could reliably determine the cause of this tire's separation."47 The Court was troubled by his assertion that he could definitively rule out abuse (i.e., under-inflation or overloading) based on the absence of two out of four known indicators, and concluded his testimony "fell outside the range where experts might reasonably differ...."48

C. Force-Fitting Forensics to Daubert/Kumho

Significantly, the Kumho opinion emphasizes the discretionary nature of Daubert analysis, noting that the factors will vary in their applicability to different disciplines or types of testimony. It also specifically notes that "nothing in either Daubert or the Federal Rules of Evidence requires a district court to admit opinion evidence that is connected to existing data only by the ipse dixit of the expert."49 A brief concurring opinion by Scalia emphasized that

[T]he discretion it endorses-trialcourt discretion in choosing the manner of testing expert reliability-is not discretion to abandon the gatekeeping function. . . . [I]t is not discretion to perform the function inadequately. Rather, it is discretion to choose among reasonable means of excluding expertise that is fausse and science that is junky.50

Although Kumho gave blanket approval to district courts to apply Daubert to all manner of expert testimony, it was not entirely clear how well the factors would actually extrapolate beyond pure science and engineering disciplines. In particular, if notions such as testability or peer review are excised from their well-understood mooring within science- and technology-based fields, can they be adapted meaningfully to forensic disciplines? It is useful to know how other forensic techniques have fared post-Kumho, especially those involving human identification.

III. The Handwriting on the Wall

A. Forensic Evidence Under Daubert

A fascinating study of post-Kumho admission of forensic evidence was carried out in 2002-2003.51 Using welldocumented methods, the study identified ninety-three Daubert analyses by federal district courts of forensic evidence from disciplines which could be classified as scientific or "those closely related to [science] using generally accepted methods of applied technical forensic techniques ...."52 The authors noted Daubert's reference to Karl Popper's statement that "the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability."53 Of the evidence analyzed, twenty-eight instances involved engineering techniques, twenty-five concerned human identification, sixteen accounting or economics, eleven toxicology, five fire science, and four each physics and pathology.

In general, the authors concluded that "judges have been remarkably consistent in applying Daubert to forensic techniques," especially those "truly scientific areas," including pathology, physics, engineering and fire science.54 They also note, however, that in the more "technical" areas, outcomes have been less consistent, most particularly in forensic identification techniques. Of the twenty-five instances of these, eight involved fingerprints, eleven involved handwriting analysis, and the remainder involved a potpourri, including DNA, footprints and hair. Concerning nonfingerprint identifications, courts have been relatively consistent in a way that is unlikely to raise any scientific eyebrows, permitting individual "positive identifications" only where they are appropriate, i.e., in DNA analysis, while otherwise allowing the expert only to testify that the evidence is consistent with a match.

Handwriting analysis provides the most interesting comparison with fingerprints, because it predates fingerprinting techniques, and prior to Daubert/Kumho, practitioners routinely claimed, like fingerprint examiners, to know that a match was certain. After Kumho, a split appeared in courts' treatment of handwriting testimony, with most excluding ultimate conclusory statements about unique identification. Experts were allowed to testify about the handwriting mechanics, and about similarities between documents in dispute and known exemplars. However, a few courts have still permitted ultimate conclusions as to identity of the author.

B. What About Fingerprints?

By contrast with handwriting, Daubert challenges to fingerprint testimony have produced the remarkable result of perfectly consistent unrestricted admission. Admission of testimony challenging the scientific validity of fingerprint techniques has been less so. To date, the sole inconsistency regarding testifying to absolute identification occurred in 2002 when a district judge reversed himself on rehearing in the Llera Plaza decisions.55 One state appeals court went so far as to sarcastically call testimony by Simon Cole regarding the lack of scientific underpinnings in fingerprint methods "junk science."56

To fully appreciate this outcome, a closer look at the numbers is warranted. For example, one web site which catalogued total fingerprint challenges (including state court cases) listed forty from 1999-2002.57 The numbers for federal courts are presented in Table 1, which was compiled by drawing on various sources cited herein.58 Looking at the temporal distribution of cases suggests that a flurry of post-Kumho interest in challenging fingerprints came to an end shortly after the disheartening reversal of Llera Plaza II and a spike in appeals court rulings uniformly upholding fingerprint testimony. This growing body of consistent case law has been countered with a growing body of scholarship from the mainstream science and legal communities asserting, ever more emphatically, that fingerprint analysis can and should become a full-fledged science subject to the strictest application of Daubert.

C. The More Things Change, the More They Stay the Same

Fingerprint examiners in the 1920's successfully claimed for themselves the status of infallible and interchangeable technicians, whose testimony was, for all practical purposes, unassailable. Generally, it still is. To understand why, we must first consider the identification process from the practitioner's standpoint. Recall the difference between latent and rolled prints. The latter are full sets of ten fingers. Each print contains an estimated 75-175 ridge features.59 The former are created by touching things casually, and may be distorted by pressure, the shape of the surface on which they are left, or vagaries of the medium used to lift the print, or may otherwise be rendered difficult to interpret. Also, the actual fraction of the "whole" print present in each latent print varies. The average for prints which are used in identification has been estimated by the Department of Justice to be about twenty percent.60 Thus, when an examiner compares a latent print to a set of rolled prints, the quality and quantity of information available will vary enormously.

Because the community has operated organically through an apprenticeship arrangement without the need for coordination or uniformity, every organization has its own standards.61 Overall patterns and specific features of the ridge detail are used universally, but wide variation exists concerning, for example, the number of matching features necessary to declare a match. Examiners also adopt individual opinions as to the frequency with which specific characteristics occur in the absence of quantitative research.62 There is also diversity in use of "tertiary characteristics" i.e., sweat pore locations, which are more difficult to detect than ridge features.63

Some universal rules do apply, in theory. Under the "one discrepancy rule," sensibly enough, a single discrepancy in a feature should lead to exclusion. Another rule dictates three possible outcomes to a comparison: definite exclusion, inconclusive, or absolute identification. The last is called "individuation" and is arrived at through an internal process of persuasion; at some point in the examination-usually performed without the creation of records of any kind-the examiner has a "Eureka!" moment when he or she knows the prints match, with one hundred percent certainty. From then on, additional points of similarity become gravy.

The system lends itself to experimentation and statistical validation. Plentiful quantitative elements exist, and a probabilistic framework for prints themselves could be established, analogous to what has been accomplished with DNA techniques. Further, valid procedures to determine error rates are entirely feasible. There are technical disciplines for which these things are not true, but fingerprinting is simply not one of them. If you're perfect, why change?

D. Mistakes are Made

The epistemological root of science lies in its ability to generate hypotheses that can be tested and proven wrong. One can never be absolutely certain. One fingerprintexaminer-turned-skeptic, David Grieve, famously stated:

[T]his categorical requirement of absolute certainty has no particular scientific principle but has evolved from a practice shaped more from allegiance to dogma than foundation in science. Once begun, the assumption of absolute certainty as the only possible conclusion has been maintained by a system of societal indoctrination, not reason, and has achieved such a ritualistic sanctity that even mild suggestions that its premise should be re-examined are instantly regarded as acts of blasphemy. Whatever this may be, it is not science.64

As a prominent law enforcement fingerprint examiner, Grieve demonstrates that converts are often the most ardent believers. That this passage is frequently cited attests to its resonance with the scientifically literate. The fingerprint community has, in fact, argued circularly and successfully that the identification process is not a science, and therefore the science-based considerations of Daubert don't apply. Non-forensic scientists have uniformly disagreed.

Since Daubert, numerous papers have addressed themselves at length to the proposition that fingerprints can and should be dragged out of the 19th century and made just as Daubert-complaint as DNA techniques have become.

Daubert Factor 1: Has the theory or technique been tested?

This is the granddaddy factor, which asks whether testable hypotheses can be generated, i.e., is it science? It also asks if the technique has been tested. An obvious example is, do identical twins have identical fingerprints? That is, if every fingerprint is unique, twins are the obvious starting point in testing genetic determinism versus embryonic development. In the technical arena, "testing" is more prosaic. For example, to validate a scale, it is tested by placing a series of known weights on it and looking to see if it reads the expected weight.

How does this apply to fingerprint identification? Two testable hypotheses underlie the method: uniqueness and permanence. The latter is easy enough to test. Even Faulds was convinced from his personal observations that fingerprints are permanent, and this has never been seriously disputed. The remaining issue concerns the degree to which prints are sufficiently "unique" to allow reliable conclusions as to matching. As a threshold matter, basic research is needed on statistical frequencies of patterns and features, and the effects of genetics. Such research would involve difficult problems of statistical modeling.65 Dr. Stoney recently reviewed all of the models which have been developed to date, and concluded that all of them-including the one he developed in his dissertation research-are flawed, and none of them has been tested.66 Nevertheless, no one has suggested that the goal is impossible. Such a model would permit establishment of an acceptable threshold of common features for a definite match.

This discussion presupposes that a probabilistic model in the mode of DNA analysis would be desirable. From a fingerprint analyst's perspective, if the method is infallible, such a model is superfluous. However, it is implicit in the conclusion of a match that the examiner has made exactly such a probability-based determination, albeit unconsciously and on the basis of untestable personal assumptions and beliefs. The actual method has been described by a leading defender of the status quo, David Ashbaugh, as follows:

How much is enough? Finding adequate ... formations in sequence, that one knows are specific details ... , and in the opinion of the ... specialist there are sufficient uniqueness within those details to eliminate all other possible donors in the world, is considered enough. At that point individualization has occurred, and the print has been identified.67

Note that the combination of (1) a purely internal, subjective, Eureka !-moment determination and (2) the routine practice of not keeping any records of how it was arrived at, renders the current method itself inherently resistant to actual testing. One might like to know to what extent different examiners agree as to matching of specific details, especially in the case where a match is being reviewed by a second examiner. Creating no paper trail makes it harder for the opposition to challenge a determination than it would be if they knew the precise matching points proposed.

Why haven't validation studies been conducted? Dr. Cole addressed this question recently and drew the rather sinister conclusion that "[g]overnment institutions do not appear to be merely passively resisting validation research." For example, the National Institute of Justice (NIJ) issued a Solicitation in 2000 (the year after Daubert) because the Institute had "identified the need for validation of the basis for friction ridge individualization and standardization of comparison criteria."69 However, the NIJ quickly disavowed any interpretation of the solicitation as indicating a current lack of validation, then withdrew the funding for it altogether. The NIJ had, in addition, allegedly delayed release of the solicitation until after the outcome of a then-ongoing trial which involved the first formal Daubert hearing on fingerprint admissibility.70 More alarming, the National Academies was induced to drop a proposed and partially funded panel on the use of science in forensics "because the government insisted on rights of review that the Academies have, at least in the recent past, refused to grant a sponsor."71

The government has offered up one study using a computer database of 50,000 rolled prints, which purported to estimate the probability of finding two identical prints at ten.97 This study and its conclusions have been thoroughly excoriated by reviewers with science training, one of whom also pointed out that the Lockheed Martin employees who performed it presumably knew better. Among other fatal flaws, it compared photographs of rolled prints with themselves and each other, not with latent prints or repeat rolled prints. One scholar concluded that the study actually showed that "the chance of any two fingers on earth producing rolled prints that appear to be identical . .. could approach 1."73 Notably, this latter conclusion appeared in a peerreviewed journal, unlike the original study. This fact has not dissuaded the government from continuing to tout its conclusions in court, however.

Following Daubert, the DOJ flirted briefly with the notion that fingerprinting methods should be validated in a way that would satisfy mainstream science, but ultimately turned back to its roots. To what extent was this prompted by DOJ's early clean sweep of courtroom successes? Courts have embraced the notion that, even if fingerprints are subject to Daubert, a century of adversarial courtroom testing is adequate to meet this standard -if not overwhelmingly abundant.

Daubert Factor 2: For a particular technique, is the known or potential error rate high, and are there standards controlling the technique's operation?

The remaining three factors can be viewed as corollaries to the first, at least in relation to fingerprint methods. Regarding the second factor, standards and error rates in science or technology arise out of data generated through testing. First, every technique in practice has a non-zero error rate because nobody's perfect. In order to measure the error rate for fingerprint techniques, practitioners of the art would have to acquiesce. That is, the relevant errors are made by those who testify in court, so practitioners who testify would have to be the ones tested. Second, to satisfy Daubert's criterion of standards in the "relevant community," a minimum threshold of information to make an identification would have to be adopted nationally by law enforcement agencies.

Error rates: The fingerprint community claims for itself an error rate of zero.74 Dr. Cole has written recently and comprehensively concerning this claim, beginning with a catalogue of known instances of exposed error, which are admittedly rare. However, the nature of the beast is such that errors are only exposed in extraordinary circumstances. In the absence of any way to compute how frequently errors go undetected, rarity of exposed errors is meaningless. Examiners have motive to root out false negative errors, but only the defense bar is concerned with the rate of false positive errors, i.e., erroneous attributions of identity. Using DNA-based exonerations of persons convicted using only forensic identification evidence, Cole estimates a conservative lower bound of false-positives for fingerprint evidence of 0.2% - 2.5%.75 This estimate is a very rough one, and Cole hardly advocates it as a preferred approach to the question. He merely offers it to counter the asserted rate of zero.

Even more persuasive on this point, however, are three types of real-world evidence: (1) results of internallyadministered examiner proficiency tests, such as those administered by the FBI to its own employees, (2) results of external tests, such as the examinations administered for certification by the International Association for Identification (IAI), and (3) the outcome of a single overt attempt by the FBI to verify the zero error rate. First, the FBI's internal exams unsurprisingly show very low error rates, and will be considered in the discussion below of Llera Plaza II, where they were entered into evidence. Second, there are two sets of external exams. One has been administered by the Collaborative Testing Services beginning in 1981. The second are the IAI certification exams, which began in 1993. These exams are not "scientific testing," but they are revealing nevertheless.

The CTS exams are neither proctored nor timed, but are administered by mail, and their results have no consequences to the test-taker. However, presuming that the examiners who take the tests are mostly conscientious, the results can be assumed to have some probative value as to whether the false-positive rate is zero. The aggregate false-positive computed by Cole for all reported external proficiency exams as of 2005 is 0.8%.77 The 1995 test was the first one to be authorized by the IAI.78 It consisted of four suspects' rolled print sets and seven latent prints-four matches and three elimination samples. Out of 156 participants, only 44% correctly categorized all seven samples. Worse, 22% made erroneous matches, and one of the "elimination samples" was misidentified as a match by twenty-nine participants. David Grieve, who is head of Forensic Sciences for the Illinois State Police, called the results of that exam "chilling and mindnumbing."79 He concluded, "By any measure, this represents a profile of practice that is unacceptable and thus demands positive action by the entire community."

The IAI's certification is a purely voluntary accreditation, the lack of which is not a bar to testifying in court.80 Unlike the CTS exam, the IAI publishes only the pass/fail rate, which has stayed constant at about half since its inception in 1993, so no inferences are possible about error rates. However, most failures reportedly occur on the portion which requires the examinee to correctly attribute twelve of fifteen latent test prints without any false positives.81 Even if it the IAI made the specific error rates public, it would still be no better than suggestive of what happens in practice, and not a substitute for a controlled simulation.

Finally, during the first formal fingerprint challenge under Daubert, in the Mitchell trial, the prosecution attempted to verify the zero error rate rather dramatically.82 The two prints in evidence, which had been positively identified as belonging to Mitchell, were sent by the FBI to fifty-three law enforcement agencies, along with Mitchell's rolled prints. Without explaining the reason, the FBI asked the agencies to have a "court qualified" examiner determine whether they were a match. Of thirty-four responses, nine failed to identify one or both of the latent prints as belonging to the rolled print donor. Following this unexpected result, the FBI sent an additional mailing to the nine errant agencies containing enlarged photographs with the purported matched areas marked for comparison. In response, all nine agencies produced the correct answers.

Standards: Regarding objective standards, there is no uniform quantitative standard for making identifications, only a subjective and undocumented decision. However, practitioners testify in court that they do have a "standard" for practice: Analyze, Compare, Evaluate, and Verify (ACE-V). In reality, this is little more than a descriptive label applied to actions which are theoretically taken since verification by a second examiner is not standard practice and the first three verbs do not refer to independent steps. A standard that would meet Daubert would be a method for computing the probability of a match.

Daubert Factor 3: Has the theory or technique been subjected to peer review and publication?

The third Daubert factor is ill-suited to the practice of fingerprint identification because the community is a closed one. The self-selected members have implicitly agreed not to disagree as a condition of admission, so there is none of the internal competition that exists in scientific and technical fields. This alliance allows a plethora of methods and standards to peacefully co-exist, with no need to evolve. This is not the case for methods of lifting latent prints from surfaces, which make up the bulk of articles contained in fingerprint-methodology journals.83 That is, unlike print-reading, which has been frozen in time for a century, print-lifting technology is a healthy, evolving, science-based field. Technicians who lift prints, however, are not burdened with the need to look a fact finder in the eye in a court of law and swear to have made an absolute identification.

In court, examiners have successfully testified that the "verification" stage of ACE-V satisfies Daubert peer review. This process rests on a trivial interpretation of the phrase without context (i.e., if a "peer" had the opportunity to tell you your work was okay, that equates to "peer review."). In the Daubert amicus brief for the AAAS and NAS, peer review is explained as one of several institutional checks which may prove useful to courts in assessing scientific research:

Peer review, in its broadest sense, represents the scientific community's effort to police itself and to assure a certain minimum level of quality so that scientists and others can rely on the results of reported scientific research. Moreover, peer review contributes to the advancement of science not merely through the screening of scientific work, but also by helping proponents of new hypotheses to improve their research and interpretations.84

In light of the context of "peer review" as it is used in Daubert, it is disingenuous at best to suggest that having your boss okay your work is positive evidence that Daubert is satisfied. It is the technique, not the practitioner, that is the target of peer review as a safeguard.

Daubert Factor 4: Does the theory or technique enjoy "general acceptance " within the "relevant scientific community "?

Applied to fingerprint examiners, this query is akin to asking whether the central tenets of a church find "general acceptance" within the community of parishioners. The presence of a maverick like David Grieve in no way challenges the "general acceptance" standard, so the only open issue is, what is the "relevant" community? Courts have largely found that, when a forensic identification technique is practiced by a closed community such as hair analysts, practitioners are not the only relevant members.85 Not so for fingerprints. And a more expansive definition of the "relevant community" is unlikely to be embraced by courts sans scientific validation. As more mainstream scientists are drawn into the debate, however, this may change.

E. So What?

Before turning to a specific discussion of how the courts have treated Daubert challenges to latent prints, we should be convinced that there is, in fact, a problem to be solved. Most courts have concluded there is not, pointing to the small number of exposed errors. This reasoning is simply wrong, however, because there is no way to compute the fraction of errors which go unreported or undetected.86 They could be estimated from simulations, but this approach has been resisted by those who could undertake it. Cole's rough estimate based on external proficiency exams indicates the potential scope of the problem. His estimate of 0.8% false positives in 2002 would have resulted in 1,905 total for that year alone.87

To understand how this plays out, it is instructive to consider the Cowans case. In early 2004, Stephan Cowans was released after spending six years in prison, becoming the first person convicted by fingerprint and exonerated by DNA.88 A Boston Police Department (BPD) examiner had identified him as the source of a print left in a home where a family was briefly held hostage. The hostage-taker was believed to be the same man who had, shortly beforehand, shot a police officer while fleeing another crime. A second member of the BPD fingerprint unit verified the match, and so did two experts retained by Cowans, who were former members of the same unit. Fortunately for Cowans, there was enough DNA evidence left at the scene-and preserved for six years-to exonerate him. Eventually, the crime scene print was matched to a resident of the house. Following his release, Cowans poignantly remarked that, after he heard the fingerprint examiner's trial testimony, "he would have voted to convict himself."

Cowans is complicated by the fact that the police officer who made the original match eventually admitted that he noticed his mistake before trial and deliberately failed to report it. Both officers then faced criminal charges, and were eventually reassigned. Most extraordinarily, Boston's police commissioner disbanded the entire unit and turned all fingerprint work over to state police. Yet, the original match was presumably an honest mistake, assuming the defense experts weren't in on a conspiracy. Mistaken false positives based on borderline-quality information are as reproducible as correct matches, particularly when the second examiner knows a match has already been made by a professional colleague. Thus, the adversarial is not well-suited to detect them in the absence of a genuinely independent pool of skeptical examiners. In fact, there is no reliable way to uncover honest erroneous matches within the ACE-V framework so such errors will nearly always go undetected.

Fingerprint examiners need not be incompetent or dishonest to make errors. Vast swaths of experimental methodology in the sciences are specifically designed to eliminate the possibility of observer bias from reported results. This problem has been reviewed comprehensively elsewhere,90 but individuals cannot rid themselves of bias through an effort of will and good intentions; it is intrinsic to our psychological makeup. However, bias can be significantly reduced by "blind" determinations, i.e., by keeping the fingerprint examiner from knowing what those investigating the crime think, or what a previous examiner found. Unfortunately, "blind" testing is rarely practiced in crime labs.91

It is important to note that Brandon Mayfield's misidentification was not merely a fluke; it was a manifestation of using giant databases to search for matches. The database against which the FBI ran the Madrid train bombing prints contained more than 470 million individual rolled fingerprints.93 The Office of Inspector General ("OIG) report on the Mayfield case identifies as the first contributing cause of error, "the unusual similarity in the pattern of Level 2 details within the friction ridges on the fingers of Mayfield and [Daoud]."94 The report concludes that there were, in fact, ten genuine points of similarity, which misled FBI examiners. These features were not identical between the two men's rolled prints, but they were similar enough to mislead four experienced fingerprint specialists as to the source of the latent print in question.

IV. The Judicial System's Response to Daubert Challenges

The judiciary has been singularly united in concluding that fingerprint identification evidence is admissible under Daubert. Individual esteemed members of the bench may perhaps be excused for not wishing to be first to suggest that the emperor is less than fully clothed, and that not one of them has noticed it for over a century. However, that outcome has effectively eviscerated Daubert/Kumho as it applies to fingerprint testimony. Table 2 shows by year federal appeals court challenges to admission of fingerprint identifications under Daubert. Only a few circuits have failed to weigh in, and some are more willing than others to create controlling precedent. Even in unpublished opinions, however, the Fourth Circuit has expressed in strongly worded dicta that fingerprint evidence is presumed valid and reliable. A few of the key decisions which have shaped the legal landscape on this topic are considered below.

A. An Early Dissenting Voice: U.S. v. Parks(1991)

In 1991, just two years before Daubert, a federal judge excluded fingerprint testimony after raising sua sponte the issue of its scientific validity, in United States v. Parks?6 This judge had noticed a fishy pattern in fingerprint identifications. The examiner had testified that the prints at issue had ten or twelve points of agreement, and that she considered eight to be enough for a positive identification. The judge expressed his misgivings:

I've had a lot of fingerprinting testimony, and it's been from the same group of people by and large, and my impression [is] that it slides up and down, that if you have only 10 points, you're comfortable with 8; if you have 12, you're comfortable with 10; if you have 50, you're comfortable with 20. . . . [You are] probably the most junior [examiner] I've ever permitted to testify as an expert, [and] you are comfortable with fewer than anybody that has ever testified before me before. And, as it happens, you also have fewer than anybody that's ever testified before me; that makes me very uncomfortable.97

The judge further questioned his witness, then her supervisor, and ultimately a third examiner-a board certified member of the IAI-regarding published standards in the field, and was aghast to discover there were none. The judge commented,

You don't have any standards ... It's just an ipse dixit ... So far as I've heard from you, and so far I've heard from anybody, those kinds of studies that would turn this into a bona fide science simply haven't been done. This thing could be turned into a science, but it isn't now ....98

This is apparently the only time fingerprint testimony has been excluded by a federal court as not adequately scientific.

B. Bootstrapping Validity: Sherwood & Salameh (1996,1998)

In 2004 a New York district judge cited two pre-Kumho cases as precedents upholding fingerprint reliability.9 The first was Sherwood, which was the first Ninth Circuit opinion to consider fingerprint testimony. 00 Because no challenge was raised in the trial court, Sherwood's value as a precedent for reviewing a lower court's decision is questionable, although it did explicitly rule that there was no error in allowing the examiner to testify to having "no doubt" as to the identification. Salameh was a second Circuit case involving a suspect in the 1993 World Trade Center bombing, and hundreds of fingerprints on various pieces of evidence were identified as belonging to the defendant.101 The identification was not challenged. Only testimony as to how Mr. Salameh might have been holding a book was challenged, making this a peculiarly inappropriate case to cite in this way. It also illustrates an unfortunate fact of Daubert challenges on the whole; no distinction has been made by courts among various types of fingerprint testimony either on the basis of quality or quantity.

C. Excluding Scientific Criticism: Mitchell I (E.D.Pa. 1999)

Mitchell I involved the first formal Daubert hearing on admissibility of fingerprint identification testimony.102 Curiously, the prosecution initially raised a Daubert challenge to a defense witness. James Starrs, a Professor of Law and Forensic Science at George Washington University, had been called to testify to the limitations of fingerprint evidence. The prosecution then moved to hold a Daubert hearing on the admissibility of the defense testimony. The ensuing two-way Daubert hearing lasted five days, and was the occasion for the government's two attempts, discussed supra, to generate evidence that fingerprints satisfy the Daubert factors.

Following the hearing, the judge issued a bench ruling in which he took judicial notice of the uniqueness and permanence of small sections of "friction ridge" skin, but declined to take judicial notice that identifications can be made from latent prints. More significantly, he admitted the government's fingerprint testimony and excluded expert testimony for the defense regarding general reliability of fingerprint identification under Daubert.103 This amounted to a ruling that Daubert simply does not apply to fingerprint identification testimony.

The Third Circuit agreed to hear an appeal (opinion was filed on April 29, 2004). Of the four fingerprint-related grounds for the appeal, the most intriguing involved testimony by Dr. Richard Rau, who coordinated drafting of the NIJ solicitation for validation studies. 104 Dr. Rau claimed that high FBI and NIJ officials had agreed in a September, 1999 meeting to withhold the solicitation until the Mitchell trial had concluded. The named officials denied it flatly, testifying that the delay was purely for budgetary reasons. One of the experts whose testimony was excluded from Mitchell's trial, Dr. Starrs, described the solicitation as "a bolt out of the blue" in its implication that basic validation studies are actually needed.105 The trial court differed in its assessment. In a second trial following remand on other grounds, it held that the solicitation would not have been admissible, and that any claimed impeachment value was questionable.

The Daubert hearing in Mitchell was, in effect, a trial-within-a-trial in which the real parties in interest were not the defendant and the government. Instead, they were the FBI's fingerprint examiners and the three scientists who challenged the very foundations of their profession. The dynamics of this situation have been elegantly characterized by Professor Saks: "A vote to admit fingerprints is a rejection of conventional science as the criterion for admission. A vote for science is a vote to exclude fingerprint expert opinions."106 The certainty of the identification is problematic to those on the "science" side of the debate, but fingerprint experts have no graceful way to abandon the certainty that makes fingerprint identifications so powerful. Judges have thus far failed to distinguish the baby from the bath water, as the subsequent cases show.

D. Side-stepping Daubert: U.S. v. Malveaux (9th Cir. 2000)

United States v. Malveaux was the first post-Kumho Circuit court appeal, and the first of eight unpublished 9th Circuit opinions on fingerprint admissibility. As in Sherwood, no challenge to the fingerprint testimony had been raised at trial. The court first noted that Daubert was controlling on the question of fingerprint admissibility. It followed this with a onesentence analysis: "Malveaux's contention [that the district court erred in admitting fingerprint testimony] is without merit because the record reflects that the expert's testimony was based on scientific techniques and advanced a material aspect of the government's case."107 A footnote to this declaration clarifies, "To the extent that Malveaux questions the validity of the fingerprint comparison, this is a question of weight and credibility that properly went to the jury." But validity of the method is precisely the determination that Daubert requires judges to make in exercising their gatekeeping function. The point of Daubert was to relieve juries of this task. Thus, like the Mitchell court's conclusion, this effectively takes fingerprint identifications outside of Daubert altogether.

E. Co-opting Daubert: U.S. v. Havvard (7th Cir. 2001)108

This five-page opinion was the first published appeals court review of a Daubert ruling on fingerprint admissibility. In marked contrast to Mitchell, the defense did not call any experts, but rather moved to exclude the prosecution's identification testimony altogether on the ground that its scientific reliability had not been established. The prosecution, for its part, called FBI fingerprint expert Stephen Meager, who provided a detailed tutorial on the theory and practice of fingerprint methodology. The appeals court characterized his testimony on error rates: "Meager also testified that the error rate for fingerprint comparison is essentially zero. Though conceding that a small margin of error exists because of differences in individual examiners, he opined that this risk is minimized because print identifications are typically confirmed through peer review."10

Unfortunately, the opinion "betrays a fundamental misunderstanding of the Daubert decision."110 It declares that "the peer-reviewed results of fingerprint comparisons ... have been used in adversarial testing for roughly 100 years."1 ' ' The appeals court also swallowed whole the notion that the defendant had the burden of disproving the technique's reliability, noting that "establishing the reliability of fingerprint analysis was made easier by its 100 years of successful use in criminal trials ... ." The defendant attempted to use the NIJ solicitation and the 1995 CTS test results as evidence of the need for validation and to rebut the notion of a zero error rate, but the court simply excluded them because they were not part of the record in the district court.

The Howard opinion has been widely cited by subsequent courts for the propositions that courtroom adversarial testing is interchangeable with "testing" under Daubert, and that the burden is properly on the defendant to prove absence of reliability. But in science, the burden is on the proponent of a new method to prove it is valid. Howard thus effectively cast challenges to fingerprint testimony as challenges to the adversarial courtroom epistemology itself.

F. The Case that Nearly Opened the Floodgates: Llera Plaza I & II

On January 7, 2002 a district judge in Pennsylvania issued the first opinion excluding conclusory statements of fingerprint individuation from admission.112 Both parties had stipulated that they would not conduct a new hearing on the topic, but rather the Mitchell hearing transcript would be the basis of the judge's opinion. The resulting twenty-page opinion has been widely praised by fingerprint critics as the embodiment of how fingerprint testimony should be analyzed under Daubert. The conclusion stated:

A. This court will take judicial notice of the uniqueness and permanence of fingerprints.

B. The parties will be able to present expert fingerprint testimony (1) describing how any latent and rolled prints at issue in this case were obtained, (2) identifying, and placing before the jury, such fingerprints and any necessary magnifications, and (3) pointing out any observed similarities and differences between a particular latent print and a particular rolled print alleged by the government to be attributable to the same persons. But the parties will not be permitted to present testimony expressing an opinion of an expert witness that a particular latent print matches, or does not match, the rolled print of a particular person and hence is, or is not, the fingerprint of that person."113

In Llera Plaza II, the government moved for a rehearing, arguing that its "prosecutorial effectiveness . . . would be seriously compromised by the preclusion of fingerprint testimony at the 'evaluation' stage . . . ."114 This time there was plenty of expert testimony on both sides. Judge Pollack wrote, "In short, I have changed my mind."115 Vacating the original order, the new order specified that presentation of testimony by either side would now be "subject to the court's oversight . . . with a view to insuring that any proposed expert witness possesses the appropriate expert qualifications and that fingerprints offered in evidence will be of a quality arguably susceptible of responsible analysis, comparison, and evaluation." In other words, he fully accepted the ACE-V acronym as a technical method that meets Daubert.

It appears that the judge was convinced fingerprints ought not to be required to meet a strict Daubert analysis, but under Kumho could be held to a lower standard.116 To the extent that new information was introduced, Judge Pollak tended to discount it. For example, the government called Stephen Meager to reprise his successful Howard performance, supplemented by proficiency exam results with stunningly low error rates, most of them internal FBI exams.117 The defense called Allan Bayle, a twenty-five-year veteran of Scotland Yard and member of the IAI. Mr. Bayle testified as to the level of technical challenge presented by the exams, "if I gave my experts these tests, they'd fall about laughing."118 The court noted, "the FBI examiners got very high proficiency grades, but the tests they took did not."119 The court then adopted a burden-shifting approach to the error prong in declaring, "It has been open to defense counsel to present examples of erroneous identifications attributable to FBI examiners, and no such examples have been forthcoming. I conclude, therefore, . . . that there is no evidence that the error rate of certified FBI fingerprint examiners is unacceptably high."120

Much has been written on the two Llera Plaza decisions. One author called Llera Plaza I "a reasonable accommodation between the criminal justice system's reliance on this type of forensic evidence and the fact that such evidence has blatant shortcomings . . . ."121 Without the crucial element of absolute identification, the impact of fingerprint testimony would be significantly diminished, so it is not surprising the government went to such lengths to obtain the reversal of Llera Plaza II.

G. One Split Decision: U.S. v. Crisp (4th Cir. 2003)

The split panel decision in Crisp was the first published opinion from the 4th Circuit on fingerprint admissibility.122 Unlike many of the fingerprint appeals, forensic identification evidence was central to the government's case. The majority embraced the adversarial-testing-time-honored-tradition approach in a ten-page opinion which also found handwriting analysis admissible. Citing Jennings, it begins its analysis by stating, "Fingerprint identification has been admissible as reliable evidence in criminal trials in this country since at least 1911."123 It cites, among numerous post-Daubert opinions admitting fingerprints, its own unpublished decision in U.S. v. Roberts.124 Roberts contained this curious construction: "virtually every circuit and district court, both before and after Daubert, have a longstanding tradition of allowing fingerprint examiners to state their opinion and conclusions, subject to rigorous cross examination."125 Daubert, it would appear, was simply irrelevant to the "longstanding tradition" of fingerprint admissibility.

Crisp is most notable, however, for judge Michael's vigorous ten-page dissent, which adopted the mainstream science arguments in their entirety. His analysis begins bluntly, "the government's fingerprint identification evidence failed to satisfy any of the Daubert requirements for establishing scientific reliability."126 He also notes that "Professions of absolute certainty by an expert witness . . . seem to be out of place in today's courtroom." Concluding that both the fingerprint and handwriting evidence should have been excluded, he notes that, because "the government conceded at oral argument that this evidence was necessary to prove Crisp's guilt beyond a reasonable doubt," he would also reverse the conviction.

H. Mitchell II (2004)

Mitchell I involved the first formal Daubert hearing challenging fingerprint admissibility. Five years and two trials later, the Third Circuit issued a forty-five-page opinion on his second appeal.12 The appeals panel held that (1) admitting the government's fingerprint testimony was not abuse of discretion, (2) testimony as to whether fingerprint identification is "science" was properly excluded because it was immaterial, (3) the district judge's taking judicial notice of uniqueness of small areas of friction ridge skin was clearly erroneous but the error was harmless, and (4) the NIJ solicitation would not have altered the verdict had it been available to the defense. Moreover any bad faith on the government's part (which it declined to reach) would have been immaterial.

I. Post-Mayfield: Still Waiting for Change

One might expect that after the Brandon Mayfield fiasco, courts might take a more critical view of fingerprint experts' claims of near-infallibility, but that is not the case. No court has failed to admit any aspect of such testimony, and the First Circuit court decision to address latent print admissibility is hardly reassuring. The 11th Circuit in April, 2005 published its first post-Daubert opinion on the admissibility and reliability of fingerprint identification evidence almost a year after the FBI apologized to Mayfield.128 In a four-page opinion, it curtly noted that other circuits had found such evidence reliable, citing Crisp, Howard, Sherwood, and curiously, U.S. v. Janis,129 which involved matching two sets of rolled prints. After explicitly holding the evidence in the case met Daubert, it went on to determine that any error was harmless.

Four additional circuit decisions have reaffirmed fingerprint reliability since the FBI apologized to Brandon Mayfield. These unfortunately include two failed challenges to comparisons of repeat sets of rolled prints, in published opinions by the Ninth and First Circuits.130 No distinction has yet been made by a court between the vastly different case of such a comparison as opposed to latent print identification, but these opinions will likely provide cover for lower courts to uphold latent identifications. Two other unpublished opinions came from the Second and Third Circuits. The Second Circuit upheld a lower court's decision to refuse to hold a Daubert hearing on the reliability of latent print methods.131 Although the review standard was abuse of discretion, the court chose to quote from Kumho in asserting that "[s]uch a hearing is not required 'in ordinary cases where the reliability of an expert's methods is properly taken for granted.'''132

Finally, an unpublished opinion from the Third Circuit upheld a lower court's decision to restrict the testimony of a defense fingerprint expert.133 The expert was a research scientist who had taken three training courses on fingerprint identification, which she estimated included 6,000 fingerprint comparisons. She was initially qualified by the court to testify as to the validity of fingerprint methods in general, not as to the specific evidence of the case. On cross-exam, in response to a question about the three latent prints in evidence, she began to critique one, which the judge took to be beyond her qualifications. He ruled that she had begun to do an identification while on the stand, which was not admissible. In upholding the decision, the court was at pains to compare her purely academic credentials with the prosecution witness's twenty years of experience doing fingerprint analysis for the Wilmington police. This opinion seems to contain a troubling assumption: the only way to become a "qualified" fingerprint witness is to do so for a living, within the law enforcement community.

The lesson of these recent cases should be "don't overreach," but since it is inexpensive and easy to challenge fingerprint evidence generally, but expensive and difficult to mount a serious defense to evidence that is genuinely questionable, defense attorneys are likely to continue mounting doomed challenges. The result may well be additional precedents upholding the notion that fingerprint validity can be "taken for granted."

J. Inquiring Minds: Commonwealth v. Patterson (2005)134

The Patterson appeal was the first time a court solicited amicus curiae briefs on fingerprint reliability.135 The high court of Massachusetts took an interlocutory appeal on a novel fingerprint method and determined that the evidence was not admissible. The case involved four fingerprints which a state police fingerprint examiner had decided were "simultaneous impressions" (left by one hand in one simultaneous touch). None of the four impressions contained enough detail for a positive identification. Prosecutors argued for the proposition that the details of the four prints could be combined exactly as if they were all one print.

Reversing the trial judge's tentative determination, the court held that the controversial technique could not pass its Lanigan-Daubert test.136 The trial court had accepted the testimony of FBI fingerprint expert Steven Meagher, that the novel procedure "involves the exact process involved in individualizing a single latent print, simply applying ACE-V to ... multiple prints from the same hand."137 The state high court was not impressed. It noted the "testimony is conclusory and unsupported by any evidence ... ."138 In addition, David Ashbaugh had called it a "weird doctrine," the use of which in England had "resulted in a hodgepodge of doctrine that is far removed from the truth."139

The court also made three relevant observations. The first is that "[t]he onediscrepancy rule is, unfortunately, less than it appears. Fingerprint examiners can and often do ignore one or more discrepancies ... ." Second, explicitly contradicting Howard, the court adopted the view that "adversary testing is not what the Supreme Court meant when it discussed testing as an admissibility factor." Third, it agreed with the trial court's conclusion that "the verification process of ACE-V was seriously flawed and did not constitute peer review under Daubert." Presumably, the court was influenced by The Innocence Project's amicus brief, whose participants included numerous scientists and scholars, including Cole, Stoney, and Zabell.

One speculates the extent to which this court's relatively open-minded approach to the question was influenced by the Cowans case, and the subsequent revelations which caused the Boston fingerprint unit to be disbanded altogemer.14 Like Mitchell, Patterson had had one jury conviction reversed on appeal, and the testimony at his first trial was given by a Boston examiner. At the time of his second trial, the unit had been disbanded. The Patterson opinion cites that case in a footnote, noting how the Commonwealth had argued that a LaniganDaubert hearing wasn't even necessary until the false positive of Cowans came to light in early 2001.141

Finally, Patterson was accused of murdering a Boston police detective in a particularly horrific crime where fingerprints were the principal evidence against Patterson.142 The extent to which law enforcement officials feel more pressure to solve high-profile or personal crimes and the potential that has for influencing the judgment of forensic specialists was raised by the Mayfield case.

V. The Brandon Mayfield case: Meteorite or Titanic?

The Mayfield incident was unprecedented in the history of fingerprint analysis, in ways that are not necessarily obvious. The salient question here is whether the events were unfortunate but bizarre and unforeseeable, or whether the events were virtually inevitable. The OIG review suggests they were inevitable, based on its conclusions and its recommendations for the future.143

A. Background of the Fingerprint Identification

The full sequence of events in the Mayfield case and reasons why the story played out as it did are too complex to address in entirety in this article, but the relevant highlights from Chapter 2 of the OIG report, are included below.

The Madrid commuter train bombings shocked the world on March 11, 2004, with approximately 200 people killed and more than 1,400 wounded. The fingerprint in question was one of two recovered by the Spanish National Police ("SNP") from a plastic bag found inside a stolen van linked to the bombings. Images of the prints were sent to the FBI via INTERPOL. Michael Wieners, one of three unit chiefs in the FBI fingerprint lab, took charge of the high priority request. He assigned Terry Green, a supervisor with extensive experience, to run the computerized (IAFIS) search for potential matching fingerprints. After rejecting the three highest-scoring candidates generated by the computer search, Green identified the fourth as a match, then identified the candidate as Brandon Mayfield. Mayfield's prints were in the database as a result of an automobile burglary charge when he was a teenager, which had been dismissed.

In accordance with FBI procedures, Wieners asked John Massey, a retired FBI examiner with thirty-five years of experience, to verify the match "independently." He performed a fresh analysis and determined it was a match. In accordance with FBI procedures, neither examiner documented his findings. Green's formal report was finalized on March 20, after which another Unit Chief performed a "peer review" of the report. The report did not contain the specific basis for the match, which was as yet undocumented. Two days later, the FBI received the news that the SNP had expressed some doubts as to the match, based on their "amazement that an American's fingerprint would show up on the evidence, given that the other suspects were local Moroccans."145 The SNP did not yet have Mayfield's rolled prints, but Green sent copies later that same day, with charted enlargements helpfully showing fifteen points of Level 2 similarity. These were "the earliest written record of the basis of the FBI's identification."146

On April 13, the SNP issued a formal report that the latent fingerprint comparison with Mayfield was inconclusive. However, the conclusion of "negativo" became "negative" in the translation, and FBI examiners were uncertain whether it meant inconclusive or an exclusion. Green and Wieners "took another look" at the comparison, but their confidence was unshaken. Wieners traveled to Madrid with a new set of enlargements which had been embellished with additional Level 3 details. The purpose of this meeting was not to compare notes with the SNP, but to present the FBI's conclusions in person. Fortunately for Mr. Mayfield, the Spanish were not fully won over to the FBI's conclusion, but they agreed to reexamine the prints.

Meanwhile, press leaks about the Mayfield investigation were surfacing, making the FBI investigators nervous that Mayfield would flee, despite the fact that they had found nothing else to tie him to the bombing or any other terrorist activity. On May 6, 2004, Mr. Mayfield was arrested pursuant to a material witness warrant, and his home, office, and cars were searched pursuant to criminal search warrants. It is noteworthy that Mrs. Mayfield is a naturalized citizen, originally from Egypt, and Mr. Mayfield had converted to Islam. The extent to which these facts figured into the overall conduct of the investigation is unclear; however, with regard to the FBI's fingerprint investigation, there was nothing about Mayfield to exclude him in their minds.

On May 17, Mayfield's attorneys argued, as part of a motion for his release, that reports in the press regarding the SNP's doubts raised questions to the fingerprint match. The court appointed a defense expert to review the identification. Kenneth Moses, a highly qualified and experienced examiner, confirmed the identification on May 19, reporting having found sixteen minutiae in agreement. Meanwhile, the SNP continued to review its conclusions, and around May 5, three different teams reported that, although there were seven points of similarity, there were also discrepancies that could not be reconciled.147 Most particularly, the entire upper left portion of the print did not match. The FBI concluded that the upper left portion was actually a separate touch, possibly from another person.

On May 19, the SNP informed the FBI that Ouhnane Daoud had been positively identified as the source of two fingerprints which had been lifted from the plastic bag. Daoud had been connected with the bombings when Spanish police raided an apartment in an attempt to arrest suspects in the bombing. "The suspects blew themselves up," but Daoud's name was found on documents recovered from the scene. His fingerprints were on file in connection with an immigration violation. As a result of this identification, Mayfield was released on May 20, even though the FBI still believed Mayfield was guilty.

The FBI went to Madrid to retrieve the Daoud evidence and to deliver it to Washington, D.C. on May 23. Wieners asked Stephen Meagher, another Unit Chief, to become involved, because he "was the least biased examiner and the most able to conduct an objective examination."148 Meagher worked with Green, Wieners, and a fourth examiner and reported on May 24 that the latent print in contention was of "no value" for identification purposes. The FBI withdrew its positive identification of Mayfield, but did not identify Daoud.

"At this point," according to the OIG Review, "miscommunications ... began to proliferate."149 The FBI delivered various contradictory official explanations for the Mayfield incident during the ensuing days, including (1) that the images were actually of "no value" for identification purposes (because the examiners could not believe that they could be positively identified to two people), (2) that "the image [the FBI] was looking at was not as clear as the original" (a claim which was based on a misunderstanding), and that the mistake could be attributed "to 'practitioner error' as distinguished from a failure of the science.'"150 Either the original evidence was worthless, the version the FBI saw was inferior to what the SNP had, or the examiners did not properly follow the (infallible) technique.

In any case, the Material Witness Proceeding against Brandon Mayfield was dismissed that same day, and the FBI issued its unprecedented public apology. On June 15, Agent Meagher formally identified both prints as belonging to Daoud. On October 4, 2004, "attorneys for Mayfield filed a civil action ... against the FBI and DOJ" as well as the four fingerprint examiners.151 The complaint includes alleged violations of his civil rights and rights under the Privacy Act, and challenges to the constitutionality of provisions of the Patriot Act and FISA. On November 29, 2006, the federal government and Brandon Mayfield reached a partial settlement of his claims.152 The government agreed to pay $2 million in damages while allowing Mayfield to pursue his constitutional challenge to the Patriot Act.

B. What Went Wrong?

In its thorough exploration of the causes for the error, the OIG ruled out every official explanation proposed by FBI personnel, including human error (since the same error was made four times), image quality (the quality was sufficient to identify Daoud), and whether the print was of "no value" for identification purposes "because it could be 'made to work' with either Mayfield or Daoud."153 This last it found especially troubling:

This explanation, if accurate, would raise a difficult question about latent fingerprint identification. . . If LFP 17 could be 'made to work' with the known prints of two different people using accepted fingerprint identification methods and standards, the question arises: are such methods and standards preventing false identifications in other cases? In the Mayfield case, by good fortune, the known prints of both subjects eventually came to the attention of the FBI. There is no way for the Laboratory to determine from an initial analysis of a latent that it might 'work with' more than one person's fingerprints in this scenario. If the known fingerprint from only one potential subject is available for comparison, the potential for an erroneous identification is apparent.

This illustrates how the use of computer database searches interacted with the belief that only one print can be found to match a latent of "sufficient" quality. Suppose that Daoud's prints had been available in me IAFIS, and ranked third by the database search. Mayfield's fourthranked print never would have been examined, because once Green determined he had a "match," the seventeen remaining sets of prints would be discarded. User skill is an integral part of the process, so while there is a logical reason to start at the top, computer ranking does not guarantee that the lower-ranked prints have less in common with the latent.

According to the OIG, the first cause of error was the "unusual" fact that the two men's exemplar (rolled) prints shared ten points of similarity. However, there was sufficient reason to exclude Mayfield's print, especially when it was accepted that the upper left quadrant was indeed part of the same touch. Instead of seeing the discrepancies, however, the examiners used "circular reasoning," preventing them from excluding Mayfield. "[T]he LPU examiners' interpretations of some features in LFP 17 were adjusted or influenced during the comparison phase by reasoning 'backward' from features that are visible in the Mayfield exemplars."154 The OIG identified five Level 2 features which the FBI examiners had charted on Mayfield's print as points of correspondence with LFP 17 for which there was no corresponding feature on Daoud's print. The examiners had simply convinced themselves that five features existed where they did not. Notably, this was not against any policy or standard in existence in the lab. Similarly, none of the five Level 3 details upon which the FBI had placed so much reliance for the April 21 briefing had a corresponding feature on Daoud's print.

The second major problem identified by the OIG was a failure to apply "fair reasoning" to the discrepancies as well as to the Level 3 details. According to standards used by INTERPOL, "[a]s a rule the quality of the difference (e.g., explained by distortion) should not be higher than the quality of the similarities . . . ."155 The SNP examiners had concluded the upper left quadrant was part of the same print because it matched with the rest of the print, both in the pattern of lines, and in the pressure of the touch. To reject that interpretation, the FBI examiners needed "to believe a remarkable set of coincidences."156

Examiners accepted ten nonexistent features and rejected numerous features which did exist, largely because their excessive faith in their own infallibility led them to adopt beliefs about individual pieces of evidence consistent with the initial determination of a match. Arguably, the Mayfield case was not a freak occurrence, but was more like the Titanic. The "unsinkable" ship did not sink because no one thought there would be icebergs in the North Atlantic. It sank in part because the captain failed to observe the normal safety measures which the ship's designers had assumed would be followed. These measures were ignored because the ship was thought to be unsinkable. Similarly, fingerprint identification cannot be rendered infallible through a circular set of beliefs. Human beings cannot eliminate observer bias from fingerprint interpretation by an act of will; however, a few simple precautions could vastly reduce the chances of future failures.

C. Recommendations for Change

Finally, the OIG report recommended five categories of changes to FBI fingerprint practices,157 all of which are consistent with those urged by critics such as Ashbaugh, Cole and Stoney. First, it recommends validation research projects, including those designed to determine accuracy of examiner performance, "a minimum quantitative threshold" of similarities for identification, and the randomness of Level 2 and Level 3 details. Second, revisions to standard operating procedures are urged, including converting the ACE-V methodology into a "step-by-step" set of procedures, and requiring that explanations for discrepancies meet an equivalent degree of certainty as that required for identification. Third, the report endorses "a dramatic expansion of ... documentation requirements for latent print examinations," noting that this would promote reproducibility and transparency of analysis, and facilitate error investigations. Fourth, verification procedures should be blind and independent because the current informal system "may contribute to 'confirmation bias' due to the verifier's knowledge that another examiner in the Laboratory has already made an identification."158 Finally, the report advocates improved training for examiners, including "[incorporating Ashbaugh as required reading in the training module," and increased attendance by examiners at international conferences.159

According to the OIG Review, the FBI intends to implement "all recommendations set forth in the Latent Review Team reports," and has already begun to fund some research projects, although it does not specify what these are. It remains to be seen whether the agency will take these recommendations to heart, or whether it will rely on the courts to continue admitting evidence as to "100% certain" identifications. The past two years have not seen any perceptible increase in skepticism by federal courts about such evidence. Regardless of what standards the FBI adopts, other law enforcement agencies are free to continue their current practices. How attorneys and juries may be reacting to challenges is harder to determine, but the high profile of the Mayfield case will lead to reassessing fingerprint identification processes.

The fingerprint community has not embraced the notion that their profession may be broken. Indeed, even Agent Massey-the agent who originally verified the Mayfield match-was quoted in a story about the Mayfield error saying, "I'll preach fingerprints till I die. They're infallible."160 Indeed, for the most part, it seems quite likely that most fingerprint analyses are correct, and most examiners reasonably competent.161 However, it would be better to know it for a scientific fact.

VI. What's Next in the Fingerprint Saga?

While scientists react with horror to fingerprint examiners' claims of infallibility, the judiciary has reacted negatively to direct assaults on the reliability of fingerprint evidence. The fingerprint identification process is conceptually similar to the process by which a finder of fact, whether judge or jury, arrives at a decision. Individual pieces of evidence weigh on one side or the other. At some point, in aggregate, they cause the fact finder to reach a subjective determination as to the "facts" of the matter. Sometimes, a clear discrepancy will lead to a definite exclusion. However, once the determination is made, additional, more ambiguous evidence can be seen to reinforce that conclusion. But fingerprint examinations, unlike trials, can be made to conform to the rigors of science. The question remains whether they should, and if so, what does that really require.

What's needed is a face-saving way for practitioners and judges to separate the baby from the bath water. The problem lies in the all-or-nothing nature of mistakes when they occur. If the defendant is guilty, of course, an erroneous match creates no mischief. But in cases like Cowans or Mayfield, the price of an error can be terribly high, not just to defendants but to the examiners, as well. The adversarial process is ill-suited to ferret out mistakes honestly made by fingerprint examiners.

That the courtroom adversarial challenge is a blunt instrument for refining fingerprint science is suggested by the courts' failure to distinguish between types of fingerprint evidence, including rolled card comparisons, latent print identifications, and the book-holding testimony challenged in the Salameh case. The last involved drawing conclusions about how a book was held from fingerprint placement. This analysis exceeded the examiner's training and was outside of scientific verifiability. Courts are hardly going to rule that rolled cards of ten prints can't be compared reliably or that proof that every fingerprint is unique is even necessary, but cases like Cowans and Mayfield should spur enhanced scrutiny and skepticism, which will result in increased likelihood that mistakes will be revealed.

Other sources of change are in the pipeline. The FBI now requires fingerprint examiner trainees to have a Bachelor's degree-preferably in a physical science.162 An organized effort to educate judges about science is also underway, in the form of an annual conference which commenced in 2002.163 The 2004 conference included a presentation by Sandy Zabell on fingerprint evidence.164 Given these developments, some form of evolutionary change seems inevitable.

Finally, defense attorneys should educate themselves as to the nuances of fingerprint vulnerabilities rather than mounting doomed challenges to evidence that really is unassailable. Blanket assertions that fingerprint methods are invalid under Daubert are similarly illadvised as being likely to produce only adverse precedents and irritated judges. When retired law-enforcement fingerprint examiners are hired to analyze prints for the defense, their underlying assumptions should be ascertained. Does the examiner believe his role to be limited to insuring against fraud or obvious error, or does he believe honest people can differ about the persuasiveness of less-than-perfect evidence? Challenges should be crafted to fit the specific evidence and methods of the case to avoid judicial backlash against overreaching. A more carefully tailored approach to such challenges is more likely to generate a more sophisticated response from judges. Hopefully, as judges, prosecutors, defense attorneys, and fingerprint examiners themselves become better-informed about the true strengths and weaknesses of fingerprint methods, the justice system as a whole will benefit immeasurably.

IMAGE TABLE1

Table 1. Frequency of federal district court and federal appellate court opinions considering admissibility of fingerprint evidence. *As of April 6, 2007.

IMAGE TABLE2

Table 2. Circuit court appeals decisions considering challenges to admitted fingerprint identification testimony by trial judges. As of April 6, 2007.

IMAGE FORMULA3IMAGE FORMULA4IMAGE FORMULA5IMAGE FORMULA6IMAGE FORMULA7IMAGE FORMULA8IMAGE FORMULA9IMAGE FORMULA10IMAGE FORMULA11IMAGE FORMULA12IMAGE FORMULA13IMAGE FORMULA14IMAGE FORMULA15IMAGE FORMULA16IMAGE FORMULA17IMAGE FORMULA18IMAGE FORMULA19IMAGE FORMULA20IMAGE FORMULA21IMAGE FORMULA22IMAGE FORMULA23IMAGE FORMULA24IMAGE FORMULA25IMAGE FORMULA26IMAGE FORMULA27IMAGE FORMULA28IMAGE FORMULA29IMAGE FORMULA30IMAGE FORMULA31IMAGE FORMULA32IMAGE FORMULA33IMAGE FORMULA34IMAGE FORMULA35IMAGE FORMULA36IMAGE FORMULA37IMAGE FORMULA38IMAGE FORMULA39IMAGE FORMULA40IMAGE FORMULA41IMAGE FORMULA42IMAGE FORMULA43IMAGE FORMULA44IMAGE FORMULA45IMAGE FORMULA46IMAGE FORMULA47IMAGE FORMULA48IMAGE FORMULA49IMAGE FORMULA50IMAGE FORMULA51IMAGE FORMULA52IMAGE FORMULA53IMAGE FORMULA54IMAGE FORMULA55IMAGE FORMULA56IMAGE PHOTOGRAPH57AUTHOR_AFFILIATION

Dorothy Schmidt received a J.D. from Lewis & Clark Law School in May of 2007. Her first degree was a B.S. in engineering, following which she worked as a thermal analyst on the B2 "stealth" bomber among other programs. She later received a Ph.D. in Nutrition Science from The Ohio State University, specializing in comparing metabolic properties of different dietary fats. She later worked in Hawaii on a research study investigating obesity and diabetes among a multiethnic rural population. She and her husband moved to Portland, Oregon in 2002. Dr. Schmidt hopes to put her technical background to good use in her new career in the law. This article is an edited version of the article with which Dr. Schmidt received First Place in the 2007 IADC Student Legal Writing Contest.