Repeatability and reproducibility of comparison decisions by firearms examiners

In a comprehensive study to assess various aspects of the performance of qualified forensic firearms examiners, volunteer examiners compared both bullets and cartridge cases fired from three different types of firearms. They rendered opinions on each comparison according to the Association of Firearm & Tool Mark Examiners (AFTE) Range of Conclusions, as Identification, Inconclusive (A, B, or C), Elimination, or Unsuitable. In this part of the study, comparison sets used previously to characterize the overall accuracy of examiners were blindly resubmitted to examiners to assess the repeatability (105 examiners; 5700 comparisons of bullets and cartridge cases) and reproducibility (191 examiners of bullets, 193 of cartridge cases; 5790 comparisons) of firearms examinations. Data gathered using the prevailing AFTE Range were also recategorized into two hypothetical scoring systems. Consistently positive differences between observed agreement and expected agreement indicate that the repeatability and reproducibility of examiners exceed chance agreement. When averaged over bullets and cartridge cases, the repeatability of comparison decisions (involving all five levels of the AFTE Range) was 78.3% for known matches and 64.5% for known nonmatches. Similarly averaged reproducibility was 67.3%% for known matches and 36.5% for known nonmatches. For both repeatability and reproducibility, many of the observed disagreements were between a definitive and inconclusive category. Examiner decisions are reliable and trustworthy in the sense that identifications are unlikely when examiners are comparing non-matching items, and eliminations are unlikely when they are comparing matching items.     

Visualisation by confocal microscopy of traces on bullets and cartridge cases

The capabilities of confocal scanning laser microscopy for the visualisation of marks on bullets and cartridge cases were investigated. Confocal microscopy provides solutions to important limitations of conventional comparison microscopy with grazing light incidence, as generally used for the examination of these marks. It is expected that confocal microscopy, thanks to its broad applicability within the field of firearms investigation and its capability of non-destructively gathering quantitative three-dimensional information, will lead to a more complete and objective forensic examination of bullets and cartridge cases.     

Statistical Approaches to Type Determination of the Ejector Marks on Cartridge Cases

While type determination on bullets has been performed for over a century, type determination on cartridge cases is often overlooked. Presented here is an example of type determination of ejector marks on cartridge cases from Glock and Smith & Wesson Sigma series pistols using Naïve Bayes and Random Forest classification methods. The shapes of ejector marks were captured from images of test‐fired cartridge cases and subjected to multivariate analysis. Naïve Bayes and Random Forest methods were used to assign the ejector shapes to the correct class of firearm with success rates as high as 98%. This method is easily implemented with equipment already available in crime laboratories and can serve as an investigative lead in the form of a list of firearms that could have fired the evidence. Paired with the FBI's General Rifling Characteristics (GRC) database, this could be an invaluable resource for firearm evidence at crime scenes.     

79式微型冲锋枪子弹射击顺序的研究

目的研究79式微型冲锋枪发射子弹的排序。方法对已知顺序实弹射击取样,扩大观察弹头、弹壳上发射过程中形成的痕迹,通过统计、归纳,从中发现与排序有关的痕迹特征。结果壳体肩部圆点痕,弹底边缘压擦痕、拉壳钩痕与发射顺序有关。结论壳体肩部圆点痕、弹底边缘压擦痕等痕迹的有无、位置及轻重程度,对判断79式微型冲锋枪弹壳发射顺序有较可靠的参考价值。     

Three-Dimensional Analysis of Cartridge Case Double-Casts

Due to the shot-to-shot variability in tool mark reproduction on fired cartridge cases, a method of replication is needed for the creation of training and testing sets. Double-casting is one method that has been used for this application, but the accuracy and variability of this method needs to be characterized. Three firearms were used to fire 25 cartridges each to create the master cartridge cases. The double-casting method consists of creating a silicone mold of the master cartridge case. A plastic resin mix is then poured into the mold to create the double-cast reproduction. Fifteen double-casts of each of the 75 fired cartridge cases were created across different silicone molds to analyze within- and between-mold variability. The master cartridge cases and double-casts were scanned with a confocal microscope (Sensofar® S neox) to create three-dimensional representations of the surfaces. Two similarity metrics were used for the objective comparison of the double-casts to their master cartridge cases: the areal correlation coefficient (ACCF_MAX) and the number of congruent matching cells (CMC). The ACCF_MAX and CMC data, along with visual examinations, showed that the double-casting method produces accurate reproductions. Within-mold variability was found to be minimal, and between-mold variability was low. These results illustrate that double-casting can be applied for training and testing purposes.     

Accuracy of comparison decisions by forensic firearms examiners

This black box study assessed the performance of forensic firearms examiners in the United States. It involved three different types of firearms and 173 volunteers who performed a total of 8640 comparisons of both bullets and cartridge cases. The overall false-positive error rate was estimated as 0.656% and 0.933% for bullets and cartridge cases, respectively, while the rate of false negatives was estimated as 2.87% and 1.87% for bullets and cartridge cases, respectively. The majority of errors were made by a limited number of examiners. Because chi-square tests of independence strongly suggest that error probabilities are not the same for each examiner, these are maximum-likelihood estimates based on the beta-binomial probability model and do not depend on an assumption of equal examiner-specific error rates. Corresponding 95% confidence intervals are (0.305%, 1.42%) and (0.548%, 1.57%) for false positives for bullets and cartridge cases, respectively, and (1.89%, 4.26%) and (1.16%, 2.99%) for false negatives for bullets and cartridge cases, respectively. The results of this study are consistent with prior studies, despite its comprehensive design and challenging specimens.     

A Validation Study of Bullet and Cartridge Case Comparisons Using Samples Representative of Actual Casework

The foundation of firearm and tool mark identification is that no two tools should produce the same microscopic marks on two separate objects that they would be inaccurately or wrongly identified. Studies addressing the validity of identification infrequently employ tests that mirror realistic casework scenarios. This study attempted to do so using a double‐blind process, reducing test‐taking bias. Test kits including bullets and cartridge cases but not the associated firearms were completed by 31 analysts from 22 agencies. Analysis of the results demonstrated an overall error rate of 0.303%, sensitivity of 85.2%, and specificity of 86.8%. Variability in performance across examiners is addressed, and the effect of examiners’ years of experience on identification accuracy is explored. Finally, the article discusses the importance of studies using realistic case work scenarios when validating the field's performance and in providing courts with usable indicators of the accuracy of firearm and tool mark identification.     

Automated detection of regions of interest in cartridge case images using deep learning

This paper explores a deep-learning approach to evaluate the position of circular delimiters in cartridge case images. These delimiters define two regions of interest (ROI), corresponding to the breech face and the firing pin impressions, and are placed manually or by an image-processing algorithm. This positioning bears a significant impact on the performance of the image-matching algorithms for firearm identification, and an automated evaluation method would be beneficial to any computerized system. Our contribution consists in optimizing and training U-Net segmentation models from digital images of cartridge cases, intending to locate ROIs automatically. For the experiments, we used high-resolution 2D images from 1195 samples of cartridge cases fired by different 9MM firearms. Our results show that the segmentation models, trained on augmented data sets, exhibit a performance of 95.6% IoU (Intersection over Union) and 99.3% DC (Dice Coefficient) with a loss of 0.014 for the breech face images; and a performance of 95.9% IoU and 99.5% DC with a loss of 0.011 for the firing pin images. We observed that the natural shapes of predicted circles reduce the performance of segmentation models compared with perfect circles on ground truth masks suggesting that our method provide a more accurate segmentation of the real ROI shape. In practice, we believe that these results could be useful for firearms identification. In future work, the predictions may be used to evaluate the quality of delimiters on specimens in a database, or they could determine the region of interest on a cartridge case image.     

Objective Evaluation of Subclass Characteristics on Breech Face Marks

Subclass characteristics can be found on the breech face marks left on spent cartridge cases. Even if they are assumed to be rare and their reported number is small, they can potentially lead to false associations. Subclass characteristics have been studied empirically allowing examiners to recognize them and to understand in which conditions they are produced. Until now, however, their influence on the identification process has not been studied from a probabilistic point of view. In this study, we aim at measuring the effect of these features on the strength of association derived from examinations involving subclass characteristics. The study takes advantage of a 3D automatic comparison system allowing the calculation of likelihood ratios (LRs). The similarities between cartridge case specimens fired by thirteen S&W .40S&W Sigma pistols are quantified, and their respective LRs are computed. The results show that the influence of subclass characteristics on the LRs is limited, even when these features are prevalent among the potential sources considered in a case. We show that the proportion of firearms sharing subclass characteristics should be larger than 40% of the pool of potential firearms for the effect to be significant.     

The effect of electrostatic fingerprint visualization on integrated ballistic identification systems

Visualization of fingerprint corrosion on spent brass cartridge cases by the application of a high electrical potential and conducting carbon powder is becoming an accepted method of fingerprint enhancement. However, to date, no examination has been made of any effect this technique has on ballistic identification. To resolve this, images of the breech face and firing pin marks were captured on six plated nickel and six brass primer cup spent cartridge cases. Three nickel and three brass cases were then subjected to the application of a potential of +2500 V for a period of 1 min. The remaining cases were additionally subjected to the application of carbon powder. These latter cases were then washed to remove all traces of powder. Each case was recaptured with the same ballistic identification apparatus and imaging procedure. None of the twelve cases showed any visual difference after the application of the potential or conducting powder.     

Image matching algorithms for breech face marks and firing pins in a database of spent cartridge cases of firearms

On the market several systems exist for collecting spent ammunition data for forensic investigation. These databases store images of cartridge cases and the marks on them. Image matching is used to create hit lists that show which marks on a cartridge case are most similar to another cartridge case. The research in this paper is focused on the different methods of feature selection and pattern recognition that can be used for optimizing the results of image matching.The images are acquired by side light images for the breech face marks and by ring light for the firing pin impression. For these images a standard way of digitizing the images used. For the side light images and ring light images this means that the user has to position the cartridge case in the same position according to a protocol. The positioning is important for the sidelight, since the image that is obtained of a striation mark depends heavily on the angle of incidence of the light. In practice, it appears that the user positions the cartridge case with +/-10 degrees accuracy.We tested our algorithms using 49 cartridge cases of 19 different firearms, where the examiner determined that they were shot with the same firearm. For testing, these images were mixed with a database consisting of approximately 4900 images that were available from the Drugfire database of different calibers.In cases where the registration and the light conditions among those matching pairs was good, a simple computation of the standard deviation of the subtracted gray levels, delivered the best-matched images. For images that were rotated and shifted, we have implemented a "brute force" way of registration. The images are translated and rotated until the minimum of the standard deviation of the difference is found. This method did not result in all relevant matches in the top position. This is caused by the effect that shadows and highlights are compared in intensity. Since the angle of incidence of the light will give a different intensity profile, this method is not optimal.For this reason a preprocessing of the images was required. It appeared that the third scale of the "à trous" wavelet transform gives the best results in combination with brute force. Matching the contents of the images is less sensitive to the variation of the lighting.The problem with the brute force method is however that the time for calculation for 49 cartridge cases to compare between them, takes over 1 month of computing time on a Pentium II-computer with 333MHz. For this reason a faster approach is implemented: correlation in log polar coordinates. This gave similar results as the brute force calculation, however it was computed in 24h for a complete database with 4900 images.A fast pre-selection method based on signatures is carried out that is based on the Kanade Lucas Tomasi (KLT) equation. The positions of the points computed with this method are compared. In this way, 11 of the 49 images were in the top position in combination with the third scale of the à trous equation. It depends however on the light conditions and the prominence of the marks if correct matches are found in the top ranked position. All images were retrieved in the top 5% of the database. This method takes only a few minutes for the complete database if, and can be optimized for comparison in seconds if the location of points are stored in files.For further improvement, it is useful to have the refinement in which the user selects the areas that are relevant on the cartridge case for their marks. This is necessary if this cartridge case is damaged and other marks that are not from the firearm appear on it.     

Confocal microscopy analysis of breech face marks on fired cartridge cases from 10 consecutively manufactured pistol slides

Recent publications from the National Academy of Sciences have called for additional foundational research in the field of firearm and toolmark analysis. We examined test fires from 10 pistol slides with consecutively manufactured breech faces. A total of nine test fires from each pistol slide, for a total of 90 test fired cartridge cases, were compared using confocal microscopy combined with three-dimensional cross-correlation analysis algorithms. A total of 8010 comparisons were performed (720 matches and 7290 nonmatches). The average score for matches was 0.82 with a standard deviation of 0.06. The average score for nonmatches was 0.20 with a standard deviation of 0.03. Additionally, subclass toolmarks were observed on the breech faces, but the presence of subclass was not detected in the correlation analysis. There was no overlap of scores between matching and nonmatching test fires. This provides objective data that support the AFTE (Association of Firearms and Tool Mark Examiners) theory of identification.     

Reporting likelihood ratio for casework in firearm evidence identification

Likelihood ratio (LR) plays an important role in estimating the weight of evidence in firearm evidence identifications. LR is computed from a statistical model including the distribution of the known-matching (KM or within) and known-nonmatching (KNM or between) comparison scores. Current LR procedures rely on KM/KNM scores from existing reference firearm toolmark data sets or alternatively from generating a set of test fires using multiple firearms. Both procedures may contain theoretical or practical issues which may hinder the LR procedures from reporting an unbiased LR estimation in casework. In this paper, a reference data set was established from a set of firearms, each test-fired two cartridge cases, resulting in a basic data set and a control data set. The congruent matching cells (CMC) method was used to generate CMC scores that are used to fit in the KM/KNM statistical distributions for LR estimation. In the initial test, 130 firearms from eight manufacturers were used for generating a reference data set consisting of 260 cartridge cases representing 130 KM and 8385 KNM pairwise breech face images. Test results showed that the KM and KNM distribution intersect at CMC = 2, which is equivalent to LR = 1 (equally to support both the prosecutor and the defense propositions). When the CMC threshold is increased to 6 or more, the LR values are higher than a million, which can provide extremely strong support to the conclusion of the same firearm (or the prosecutor's proposition) in the casework of firearm evidence identification.     

Comparing the performance of IBIS and BulletTRAX-3D technology using bullets fired through 10 consecutively rifled barrels

This study evaluates the abilities of the Integrated Ballistics Identification System (IBIS) and BulletTRAX-3D electronic imaging systems to identify bullets fired by the same weapon in a large database of images. Ten consecutively rifled handgun barrels were test fired to obtain reference sample and known match sample pairs for upload onto both bullet acquisition systems. Both copper-jacketed and lead bullets were uploaded, to account for variations in the manner in which markings are reproduced on the different metal compositions. Ranked correlation lists were examined and evaluated. For copper-jacketed bullet correlations, both IBIS and BulletTRAX-3D identified all reference samples to their known matches within the top 10 positions. For lead bullets, BulletTRAX-3D identified all reference samples to their known match in the top 10 positions while IBIS identified only 30%. For inter composition comparisons, BulletTRAX-3D was more successful than IBIS, identifying 100% of reference samples to their known match in the top 20 for copper-jacketed to lead comparisons and 90% for lead to copper-jacketed comparisons. These results suggest that BulletTRAX-3D is more effective than IBIS in the analysis of a wider range of bullet types and it was also found to produce images of superior quality.     

The Use of Near‐Infrared Photography to Image Fired Bullets and Cartridge Cases

An imaging technique that is capable of reducing glare, reflection, and shadows can greatly assist the process of toolmarks comparison. In this work, a camera with near‐infrared (near‐IR) photographic capabilities was fitted with an IR filter, mounted to a stereomicroscope, and used to capture images of toolmarks on fired bullets and cartridge cases. Fluorescent, white light‐emitting diode (LED), and halogen light sources were compared for use with the camera. Test‐fired bullets and cartridge cases from different makes and models of firearms were photographed under either near‐IR or visible light. With visual comparisons, near‐IR images and visible light images were comparable. The use of near‐IR photography did not reveal more details and could not effectively eliminate reflections and glare associated with visible light photography. Near‐IR photography showed little advantages in manual examination of fired evidence when it was compared with visible light (regular) photography.     

Gunshot wounds caused by Fiocchi Anticrime cartridges (plastic bullets).

The Fiocchi Anticrime cartridge (caliber 12/70) used for shotguns contains 15 plastic bullets that have a total weight (mass) of 15.2 g and feature a V4 of 302 m/s and an E4 of 694.7 J. Hence the bullets, which are claimed to be harmless, are able to penetrate the skin at close range. This article presents four cases of injuries. According to our tests, the Fiocchi Anticrime cartridge may well cause fatal injuries within firing distances of up to approximately 4-5 m.     

The National Ballistics Imaging Comparison (NBIC) project

In response to the guidelines issued by the American Society of Crime Laboratory Directors/Laboratory Accreditation Board (ASCLD/LAB-International) to establish traceability and quality assurance in U.S. crime laboratories, a NIST/ATF joint project entitled National Ballistics Imaging Comparison (NBIC) was initialized in 2008. The NBIC project aims to establish a National Traceability and Quality System for ballistics identifications in crime laboratories within the National Integrated Ballistics Information Network (NIBIN) of the U.S. NIST Standard Reference Material (SRM) 2460 bullets and 2461 cartridge cases are used as reference standards. 19 ballistics examiners from 13 U.S. crime laboratories participated in this project. They each performed 24 periodic image acquisitions and correlations of the SRM bullets and cartridge cases over the course of a year, but one examiner only participated in Phase 1 tests of SRM cartridge case. The correlation scores were collected by NIST for statistical analyses, from which control charts and control limits were developed for the proposed Quality System and for promoting future assessments and accreditations for firearm evidence in U.S. forensic laboratories in accordance with the ISO 17025 Standard.     

Classification of firing pin impressions using HOG-SVM

Crimes, such as robbery and murder, often involve firearms. In order to assist with the investigation into the crime, firearm examiners are asked to determine whether cartridge cases found at a crime scene had been fired from a suspect's firearm. This examination is based on a comparison of the marks left on the surfaces of cartridge cases. Firing pin impressions can be one of the most commonly used of these marks. In this study, a total of nine Ruger model 10/22 semiautomatic rifles were used. Fifty cartridges were fired from each rifle. The cartridge cases were collected, and each firing pin impression was then cast and photographed using a comparison microscope. In this paper, we will describe how one may use a computer vision algorithm, the Histogram of Orientated Gradient (HOG), and a machine learning method, Support Vector Machines (SVMs), to classify images of firing pin impressions. Our method achieved a reasonably high accuracy at 93%. This can be used to associate a firearm with a cartridge case recovered from a scene. We also compared our method with other feature extraction algorithms. The comparison results showed that the HOG-SVM method had the highest performance in this classification task.     

A Fully Automatic Method for Comparing Cartridge Case Images,

When a gun is fired, it leaves marks on cartridge cases that are thought to be unique to the gun. In current practice, firearms examiners inspect cartridge cases for “sufficient agreement,” in which case they conclude that they come from the same gun, testifying in courts as such. A 2016 President's Council of Advisors on Science and Technology report questioned the scientific validity of such analysis (President's Committee of Advisors on Science and Technology, Washington, DC, Executive Office of the President). One recommendation was to convert firearms analysis to an objective method. We propose a fully automated, open‐source method for comparing breechface marks on cartridge cases using 2D optical images. We improve on existing methodology by automating the selection of marks, and removing the effects of circular symmetry. We propose an empirical computation of a “random match probability” given a known database, which can be used to quantify the weight of evidence. We demonstrate an improvement in accuracy on images from controlled test fires.     

Ricochet Behavior on Glass—Critical Ricochet Angles,Ricochet Angles,and Deflection Angles

For shooting scene reconstruction purposes, knowledge about the ricochet behavior of bullets provides valuable information. In this study, the critical ricochet angles of four cartridge types were established on plain float glass. The estimates of the critical ricochet angles varied between cartridge types and were 21.0° for .32 Auto FMJ bullets, 15.8° for 9 mm Luger FMJ bullets, 17.6° for .45 Auto FMJ bullets, and 21.3° for 9 mm Luger, Action NP bullets. The corresponding ricochet and deflection angles per incidence angle varied depending on the state of the ricocheted bullets. The mean ricochet angles are always lower than the corresponding angles of incidence, and the mean ricochet angles for the FMJ bullets with undamaged jackets are lower than those of bullets where the jacket is either damaged or the bullet partially ricocheted and partially perforated. Mean ricochet angles are lower for undamaged FMJ bullets than for undamaged Action NP bullets.