The Development and Application of Random Match Probabilities to Firearm and Toolmark Identification

The field of firearms and toolmark analysis has encountered deep scrutiny of late, stemming from a handful of voices, primarily in the law and statistical communities. While strong scrutiny is a healthy and necessary part of any scientific endeavor, much of the current criticism leveled at firearm and toolmark analysis is, at best, misinformed and, at worst, punditry. One of the most persistent criticisms stems from the view that as the field lacks quantified random match probability data (or at least a firm statistical model) with which to calculate the probability of a false match, all expert testimony concerning firearm and toolmark identification or source attribution is unreliable and should be ruled inadmissible. However, this critique does not stem from the hard work of actually obtaining data and performing the scientific research required to support or reject current findings in the literature. Although there are sound reasons (described herein) why there is currently no unifying probabilistic model for the comparison of striated and impressed toolmarks as there is in the field of forensic DNA profiling, much statistical research has been, and continues to be, done to aid the criminal justice system. This research has thus far shown that error rate estimates for the field are very low, especially when compared to other forms of judicial error. The first purpose of this paper is to point out the logical fallacies in the arguments of a small group of pundits, who advocate a particular viewpoint but cloak it as fact and research. The second purpose is to give a balanced review of the literature regarding random match probability models and statistical applications that have been carried out in forensic firearm and toolmark analysis.     

Application of likelihood ratios for firearm and toolmark analysis

Historically firearm and toolmark examiners have rendered categorical or inconclusive opinions and eschewed probabilistic ones, especially in the United States. We suggest this practice may no longer be necessary or desirable, and outline an alternative approach that is within a comprehensive logical/Bayesian paradigm. Hypothetical forensic and non-forensic examples are provided for readers who are practicing firearm and toolmark examiners, and the strengths and weaknesses of both approaches are considered.     

Development and Validation of a Virtual Examination Tool for Firearm Forensics, ,

The transition from 2D imaging to 3D scanning in the discipline of firearms and toolmark analysis is likely to provide examiners an unprecedented view of microscopic surface topography. The digital examination of measured 3D surface topographies has been referred to as virtual microscopy (VM). The approach offers several potential advantages over traditional comparison microscopy. Like any new analytic method, VM must be validated prior to its use in a crime laboratory. This paper describes one of the first validation studies of virtual microscopy. Fifty‐six participants at fifteen laboratories used virtual microscopic tools to complete two proficiency‐style tests for cartridge case identification. All participating trained examiners correctly reported 100% of the identifications (known matches) while reporting no false positives. The VM tools also allowed examiners to annotate compared surfaces. These annotations provide insight into the types of marked utilized in comparative analysis. Overall, the results of the study demonstrate that trained examiners can successfully use virtual microscopy to conduct firearms toolmark examination and support the use of the technology in the crime laboratory.     

Reporting likelihood ratio for casework in firearm evidence identification

Likelihood ratio (LR) plays an important role in estimating the weight of evidence in firearm evidence identifications. LR is computed from a statistical model including the distribution of the known-matching (KM or within) and known-nonmatching (KNM or between) comparison scores. Current LR procedures rely on KM/KNM scores from existing reference firearm toolmark data sets or alternatively from generating a set of test fires using multiple firearms. Both procedures may contain theoretical or practical issues which may hinder the LR procedures from reporting an unbiased LR estimation in casework. In this paper, a reference data set was established from a set of firearms, each test-fired two cartridge cases, resulting in a basic data set and a control data set. The congruent matching cells (CMC) method was used to generate CMC scores that are used to fit in the KM/KNM statistical distributions for LR estimation. In the initial test, 130 firearms from eight manufacturers were used for generating a reference data set consisting of 260 cartridge cases representing 130 KM and 8385 KNM pairwise breech face images. Test results showed that the KM and KNM distribution intersect at CMC = 2, which is equivalent to LR = 1 (equally to support both the prosecutor and the defense propositions). When the CMC threshold is increased to 6 or more, the LR values are higher than a million, which can provide extremely strong support to the conclusion of the same firearm (or the prosecutor's proposition) in the casework of firearm evidence identification.     

Experimental design for acquiring relevant data to address the issue of comparing consecutively manufactured tools and firearms

The comparison of consecutively manufactured tools and firearms has provided much, but not all, of the basis for the profession of firearm and toolmark examination. The authors accept the fundamental soundness of this approach but appeal to the experimental community to close two minor gaps in the experimental procedure. We suggest that “blinding” and attention to appropriateness of other experimental conditions that would consolidate the foundations of our profession. We do not suggest that previous work is unsound.     

The role and duties of the shoeprint/toolmark examiner in forensic laboratories

Shoeprint and toolmark examinations have been carried out for forensic purposes for a great number of years. According to literature, the first case of footwear identification dates back to the year 1786. This particular case was worked out by a local police constable. Who then, are the persons working in crime laboratories and specialising in shoeprint and toolmark examinations today? Are they responsible for some other kind of duties, and if ‘yes’, what kind of duties? To find answers to these questions, a questionnaire was prepared and sent to 78 forensic laboratories, 62 laboratories in 27 European countries and 16 laboratories in six countries outside Europe. Forty-three laboratories (55%) in 23 countries; 35 laboratories in 19 European countries and eight laboratories in four countries outside Europe returned the questionnaire. The conducted study shows that the shoeprint and toolmark examiners are performing a great variety of tasks besides shoeprint and toolmark examinations, and also that the tasks vary a lot from one laboratory to another.     

Adapting the Chumbley Score to Match Striae on Land Engraved Areas (LEAs) of Bullets,

The same‐source problem remains a major challenge in forensic toolmark and firearm examination. Here, we investigate the applicability of the Chumbley method (J Forensic Sci, 2018, 63, 849; J Forensic Sci, 2010, 55, 953) (10,12), developed for screwdriver markings, for same‐source identification of striations on bullet LEAs. The Hamby datasets 44 and 252 measured by NIST and CSAFE (high‐resolution scans) are used here. We provide methods to identify parameters that minimize error rates for matching of LEAs, and a remedial algorithm to alleviate the problem of failed tests, while increasing the power of the test and reducing error rates. For 85,491 land‐to‐land comparisons (84,235 known nonmatches and 1256 known matches), the adapted test does not provide a result in 176 situations (originally more than 500). The Type I and Type II error rates are 7.2% (6105 out of 84,235) and 21.4% (271 out of 1256), respectively. This puts the proposed method on similar footing as other single‐feature matching approaches in the literature.     

Toolmark Identification of a Mattock to a Clod of Soil from a Grave

Abstract: At a recent bushland gravesite that contained a deceased male, a clod of soil displaying a striated toolmark was collected during the excavation of the grave. This clod was preserved, and the mark was cast using Mikrosil?. Shovels and a mattock, which had been discarded by the suspects, were subsequently found at another location. A toolmark comparison identified the hoe end of the mattock head as having produced the striated toolmark.     

Objective Analysis of Impressed Chisel Toolmarks

Historical and recent challenges to the practice of comparative forensic examination have created a driving force for the formation of objective methods for toolmark identification. In this study , fifty sequentially manufactured chisels were used to create impression toolmarks in lead (500 toolmarks total). An algorithm previously used to statistically separate known matching and nonmatching striated screwdriver marks and quasi-striated plier marks was used to evaluate the chisel marks. Impression toolmarks, a more complex form of toolmark, pose a more difficult test for the algorithm that was originally designed for striated toolmarks. Results show in this instance that the algorithm can separate matching and nonmatching impression marks, providing further validation of the assumption that toolmarks are identifiably unique.     

Optimization of a Statistical Algorithm for Objective Comparison of Toolmarks

Due to historical legal challenges, there is a driving force for the development of objective methods of forensic toolmark identification. This study utilizes an algorithm to separate matching and nonmatching shear cut toolmarks created using fifty sequentially manufactured pliers. Unlike previously analyzed striated screwdriver marks, shear cut marks contain discontinuous groups of striations, posing a more difficult test of algorithm applicability. The algorithm compares correlation between optical 3D toolmark topography data, producing a Wilcoxon rank sum test statistic. Relative magnitude of this metric separates the matching and nonmatching toolmarks. Results show a high degree of statistical separation between matching and nonmatching distributions. Further separation is achieved with optimized input parameters and implementation of a “leash” preventing a previous source of outliers—however complete statistical separation was not achieved. This paper represents further development of objective methods of toolmark identification and further validation of the assumption that toolmarks are identifiably unique.     

Consecutive matching striation criteria: a general critique

In the forensic science of firearms and toolmark identification, examiners traditionally have drawn conclusions of identity from subjective criteria. This paper critically explores the general validity of one proposed objective-criteria regime--that of counting consecutive matching striations on fired bullets. Practical considerations are discussed, as well as theoretical ones, with both discussions viewed from the perspective of Bayesian logic. It is concluded that drawbacks exist for this particular objective-criteria regime, but that research and logical analysis should continue.     

Confocal microscopy analysis of breech face marks on fired cartridge cases from 10 consecutively manufactured pistol slides

Recent publications from the National Academy of Sciences have called for additional foundational research in the field of firearm and toolmark analysis. We examined test fires from 10 pistol slides with consecutively manufactured breech faces. A total of nine test fires from each pistol slide, for a total of 90 test fired cartridge cases, were compared using confocal microscopy combined with three-dimensional cross-correlation analysis algorithms. A total of 8010 comparisons were performed (720 matches and 7290 nonmatches). The average score for matches was 0.82 with a standard deviation of 0.06. The average score for nonmatches was 0.20 with a standard deviation of 0.03. Additionally, subclass toolmarks were observed on the breech faces, but the presence of subclass was not detected in the correlation analysis. There was no overlap of scores between matching and nonmatching test fires. This provides objective data that support the AFTE (Association of Firearms and Tool Mark Examiners) theory of identification.     

A Validation Study of Bullet and Cartridge Case Comparisons Using Samples Representative of Actual Casework

The foundation of firearm and tool mark identification is that no two tools should produce the same microscopic marks on two separate objects that they would be inaccurately or wrongly identified. Studies addressing the validity of identification infrequently employ tests that mirror realistic casework scenarios. This study attempted to do so using a double‐blind process, reducing test‐taking bias. Test kits including bullets and cartridge cases but not the associated firearms were completed by 31 analysts from 22 agencies. Analysis of the results demonstrated an overall error rate of 0.303%, sensitivity of 85.2%, and specificity of 86.8%. Variability in performance across examiners is addressed, and the effect of examiners’ years of experience on identification accuracy is explored. Finally, the article discusses the importance of studies using realistic case work scenarios when validating the field's performance and in providing courts with usable indicators of the accuracy of firearm and tool mark identification.     

A Historical,Detailed Toolmark Examination from 1916

Marc Bischoff received his PhD from the University of Lausanne, Institute of Police Science and Criminology, and later became the head of the Institute. His final examination paper for his doctorate degree constitutes a toolmark analysis and comparison between a suspected axe and questioned cuttings of wood trunks. Bischoff uses the scientific method to test the hypothesis he formulated. Dated 1916, this report includes his notes, measurements, photomicrographs of the questioned toolmarks and the test impressions, detailed sketches, technical discussion, results, and conclusions. Recognizing the historical value of Bischoff's work, it seemed important to get it translated. Photocopies of Bischoff's original work were obtained. The case report, translated from French, is presented here along with a brief history of the Firearm and Toolmark Identification field and Bischoff's role in the beginning of this field. Bischoff's work stands as testimony to the long tradition of the field of forensic toolmark analysis.     

Characteristics of firearms and gunshot wounds as markers of suicide.

Data from a total of 1,200 cases that the Medical Examiner of Dallas County, Texas, ruled as suicide with a firearm have been compared with the results of forensic laboratory examinations and tests; 75% are with handguns and 25% are with long guns. The incidence of suicides by gender yields 82% males and 18% females, with the most common entrance site being the head. With regard to racial distribution of persons committing suicide, > 88% are white in the study population group, which is 76% white. Data are presented for the incidence of blood detected inside and on the muzzle end of the barrel of the firearm, and for the persistence of blood inside the barrel even after the weapon was discharged to obtain test bullets. The analysis of handwipings for primer residues shows that "positive" results are obtained for approximately 50% of the revolvers and approximately 32% of the pistols when the caliber is > 22. Other projects such as transfer of gunshot residues to the hand by simply handling the weapon, the incidence of single-contact wounds in homicide versus suicide cases, and the percent of positive identification of bullets with weapons submitted are described.     

Global forensic anthropology in the 21st century

While it has a long history, the last 30 years have brought considerable advances to the discipline of forensic anthropology worldwide. Every so often it is essential that these advances are noticed and trends assessed. It is also important to identify those research areas that are needed for the forthcoming years. The purpose of this special issue is to examine some of the examples of research that might identify the trends in the 21st century. Of the 14 papers 5 dealt with facial features and identification such as facial profile determination and skull-photo superimposition. Age (fetus and cranial thickness), sex (supranasal region, arm and leg bones) and stature (from the arm bones) estimation were represented by five articles. Others discussed the estimation of time since death, skull color and diabetes, and a case study dealing with a mummy and skeletal analysis in comparison with DNA identification. These papers show that age, sex, and stature are still important issues of the discipline. Research on the human face is moving from hit and miss case studies to a more scientifically sound direction. A lack of studies on trauma and taphonomy is very clear. Anthropologists with other scientists can develop research areas to make the identification process more reliable. Research should include the assessment of animal attacks on human remains, factors affecting decomposition rates, and aging of the human face. Lastly anthropologists should be involved in the education of forensic pathologists about osteological techniques and investigators regarding archaeology of crime scenes.     

A model for estimating the number of glass fragments transferred when breaking a pane: experiments with firearms and hammer.

Numerous publications have shown the importance of transfer in the interpretation of glass evidence. As this phenomenon is also highly variable, it was decided to test the hypothesis that there exists a means to predict the number of fragments recovered at time t = 0. Panes of float glass-of different types and thickness-were broken using either a firearm or a hammer. It was decided to choose a firearm as the main breaking device, as it allowed not only to have more reproducible conditions but also to acquire knowledge in a field where little has been published. Despite the inherent variation in the breaking process, the results show that using a statistical model it is possible to predict the number of fragments transferred onto a garment from the number of fragments transferred to the ground. This research also indicates the size and number of particles transferred onto a person, when breaking window panes of different types (float, laminated or toughened) with different breaking procedures.     

Asian Criminology’s Expansion and Advancement of Research and Crime Control Practices

Inspired by Professor Jianhong Liu’s article entitled “Asian Criminology—Challenges, Opportunities, and Directions,” published in 2009 in the Asian Journal of Criminology, I attempt to document the expansion of Asian criminology in terms of the numbers of journal articles published, but also to encapsulate some of the unique contributions of original articles published in the Asian Journal of Criminology. To document the expansion of Asian criminology in terms of journal articles, I used the academic search engine, the ISI Web of Science, to identify criminology articles across a variety of Asian countries and time. This led to distinguishing between more traditional criminology and crime control (TCCC) articles and articles focusing on violence against women/children and on trafficking (VAWCT). Second, because it is not yet part of the ISI Web of Science, I read all of the original articles in the Asian Journal of Criminology to account for how Asian criminology is advancing the larger discipline of criminology. The ISI Web of Science data document the rapid expansion of Asian criminology. Some of the specific Asian Journal of Criminology articles are used to exemplify specific examples of how Asian criminology is advancing the field of criminology worldwide through theoretical, methodological and framing designs, and crime control practices.     

An ideal material for the preparation of known toolmark test impressions

Traditionally, toolmark test exemplars are produced by applying a tool's working surface to a piece of soft metal such as lead. Soft, pliable metals are primarily used for this purpose because they will replicate the microscopic grooves present on a tool's working surface without damaging the tool. In this paper the authors present an alternative material for the preparation of test toolmarks. Jewelry modeling or carving waxes are utilized in this study. These waxes are designed for the jewelry modeling industry to create very fine, highly detailed wax models of jewelry pieces that will be cast in various metals utilizing the lost wax casting method. Jeweler's waxes have been found to be ideal for preparing test toolmarks from exemplar tools. The test tool's working surface is applied to a piece of the appropriate wax in a manner consistent with the tool's design. The replicas obtained are exact, highly detailed, 1:1, negative impressions of the exemplar tools working surface, have a long shelf-life, and are suitable for use in toolmark examination and comparison cases.