Using subsampling to estimate the strength of handwriting evidence via score-based likelihood ratios

The likelihood ratio paradigm has been studied as a means for quantifying the strength of evidence for a variety of forensic evidence types. Although the concept of a likelihood ratio as a comparison of the plausibility of evidence under two propositions (or hypotheses) is straightforward, a number of issues arise when one considers how to go about estimating a likelihood ratio. In this paper, we illustrate one possible approach to estimating a likelihood ratio in comparative handwriting analysis. The novelty of our proposed approach relies on generating simulated writing samples from a collection of writing samples from a known source to form a database for estimating the distribution associated with the numerator of a likelihood ratio. We illustrate this approach using documents collected from 432 writers under controlled conditions.     

Uncertainty about the true source. A note on the likelihood ratio at the activity level

This paper focuses on likelihood ratio based evaluations of fibre evidence in cases in which there is uncertainty about whether or not the reference item available for analysis - that is, an item typically taken from the suspect or seized at his home - is the item actually worn at the time of the offence. A likelihood ratio approach is proposed that, for situations in which certain categorical assumptions can be made about additionally introduced parameters, converges to formula described in existing literature. The properties of the proposed likelihood ratio approach are analysed through sensitivity analyses and discussed with respect to possible argumentative implications that arise in practice.     

Bayesian networks and the value of the evidence for the forensic two-trace transfer problem*

Forensic scientists face increasingly complex inference problems for evaluating likelihood ratios (LRs) for an appropriate pair of propositions. Up to now, scientists and statisticians have derived LR formulae using an algebraic approach. However, this approach reaches its limits when addressing cases with an increasing number of variables and dependence relationships between these variables. In this study, we suggest using a graphical approach, based on the construction of Bayesian networks (BNs). We first construct a BN that captures the problem, and then deduce the expression for calculating the LR from this model to compare it with existing LR formulae. We illustrate this idea by applying it to the evaluation of an activity level LR in the context of the two-trace transfer problem. Our approach allows us to relax assumptions made in previous LR developments, produce a new LR formula for the two-trace transfer problem and generalize this scenario to n traces.     

Combining evidence in complex cases - a practical approach to interdisciplinary casework

Activity level evaluations, although still a major challenge for many disciplines, bring a wealth of possibilities for a more formal approach to the evaluation of interdisciplinary forensic evidence. This paper proposes a practical methodology for combining evidence from different disciplines within the likelihood ratio framework. Evidence schemes introduced in this paper make the process of combining evidence more insightful and intuitive thereby assisting experts in their interdisciplinairy evaluation and in explaining this process to the courts.When confronted with two opposing scenarios and multiple types of evidence, the likelihood ratio approach allows experts to combine this evidence in a probabilistic manner. Parts of the prosecution and defence scenarios for which forensic science is expected to be informative are identified. For these so called core elements, activity level propositions are formulated. Afterwards evidence schemes are introduced to assist the expert in combining the evidence in a logical manner. Two types of evidence relations are identified: serial and parallel evidence. Practical guidelines are given on how to deal with both types of evidence relations when combining the evidence.     

Assessment of forensic findings when alternative explanations have different likelihoods—“Blame-the-brother”-syndrome

Assessment of forensic findings with likelihood ratios is for several cases straightforward, but there are a number of situations where contemplation of the alternative explanation to the evidence needs consideration, in particular when it comes to the reporting of the evidentiary strength. The likelihood ratio approach cannot be directly applied to cases where the proposition alternative to the forwarded one is a set of multiple propositions with different likelihoods and different prior probabilities. Here we present a general framework based on the Bayes' factor as the quantitative measure of evidentiary strength from which it can be deduced whether the direct application of a likelihood ratio is reasonable or not. The framework is applied on DNA evidence in forms of an extension to previously published work. With the help of a scale of conclusions we provide a solution to the problem of communicating to the court the evidentiary strength of a DNA match when a close relative to the suspect has a non-negligible prior probability of being the source of the DNA.     

Learning about Bayesian networks for forensic interpretation: an example based on the 'the problem of multiple propositions'

Both, Bayesian networks and probabilistic evaluation are gaining more and more widespread use within many professional branches, including forensic science. Notwithstanding, they constitute subtle topics with definitional details that require careful study. While many sophisticated developments of probabilistic approaches to evaluation of forensic findings may readily be found in published literature, there remains a gap with respect to writings that focus on foundational aspects and on how these may be acquired by interested scientists new to these topics. This paper takes this as a starting point to report on the learning about Bayesian networks for likelihood ratio based, probabilistic inference procedures in a class of master students in forensic science. The presentation uses an example that relies on a casework scenario drawn from published literature, involving a questioned signature. A complicating aspect of that case study - proposed to students in a teaching scenario - is due to the need of considering multiple competing propositions, which is an outset that may not readily be approached within a likelihood ratio based framework without drawing attention to some additional technical details. Using generic Bayesian networks fragments from existing literature on the topic, course participants were able to track the probabilistic underpinnings of the proposed scenario correctly both in terms of likelihood ratios and of posterior probabilities. In addition, further study of the example by students allowed them to derive an alternative Bayesian network structure with a computational output that is equivalent to existing probabilistic solutions. This practical experience underlines the potential of Bayesian networks to support and clarify foundational principles of probabilistic procedures for forensic evaluation.     

Bayesian analysis of fingerprint, face and signature evidences with automatic biometric systems

The Bayesian approach provides a unified and logical framework for the analysis of evidence and to provide results in the form of likelihood ratios (LR) from the forensic laboratory to court. In this contribution we want to clarify how the biometric scientist or laboratory can adapt their conventional biometric systems or technologies to work according to this Bayesian approach. Forensic systems providing their results in the form of LR will be assessed through Tippett plots, which give a clear representation of the LR-based performance both for targets (the suspect is the author/source of the test pattern) and non-targets. However, the computation procedures of the LR values, especially with biometric evidences, are still an open issue. Reliable estimation techniques showing good generalization properties for the estimation of the between- and within-source variabilities of the test pattern are required, as variance restriction techniques in the within-source density estimation to stand for the variability of the source with the course of time. Fingerprint, face and on-line signature recognition systems will be adapted to work according to this Bayesian approach showing both the likelihood ratios range in each application and the adequacy of these biometric techniques to the daily forensic work.     

Likelihood ratio methods for forensic comparison of evaporated gasoline residues

In the investigation of arson, evidence connecting a suspect to the fire scene may be obtained by comparing the composition of ignitable liquid residues found at the crime scene to ignitable liquids found in possession of the suspect. Interpreting the result of such a comparison is hampered by processes at the crime scene that result in evaporation, matrix interference, and microbial degradation of the ignitable liquid.Most commonly, gasoline is used as a fire accelerant in arson. In the current scientific literature on gasoline comparison, classification studies are reported for unevaporated and evaporated gasoline residues. In these studies the goal is to discriminate between samples of several sources of gasoline, based on a chemical analysis. While in classification studies the focus is on discrimination of gasolines, for forensic purposes a likelihood ratio approach is more relevant.In this work, a first step is made towards the ultimate goal of obtaining numerical values for the strength of evidence for the inference of identity of source in gasoline comparisons. Three likelihood ratio methods are presented for the comparison of evaporated gasoline residues (up to 75% weight loss under laboratory conditions). Two methods based on distance functions and one multivariate method were developed. The performance of the three methods is characterized by rates of misleading evidence, an analysis of the calibration and an information theoretical analysis.The three methods show strong improvement of discrimination as compared with a completely uninformative method. The two distance functions perform better than the multivariate method, in terms of discrimination and rates of misleading evidence.     

Evidence evaluation in fingerprint comparison and automated fingerprint identification systems--modelling within finger variability

Recent challenges and errors in fingerprint identification have highlighted the need for assessing the information content of a papillary pattern in a systematic way. In particular, estimation of the statistical uncertainty associated with this type of evidence is more and more called upon. The approach used in the present study is based on the assessment of likelihood ratios (LRs). This evaluative tool weighs the likelihood of evidence given two mutually exclusive hypotheses. The computation of likelihood ratios on a database of marks of known sources (matching the unknown and non-matching the unknown mark) allows an estimation of the evidential contribution of fingerprint evidence. LRs are computed taking advantage of the scores obtained from an automated fingerprint identification system and hence are based exclusively on level II features (minutiae). The AFIS system attributes a score to any comparison (fingerprint to fingerprint, mark to mark and mark to fingerprint), used here as a proximity measure between the respective arrangements of minutiae. The numerator of the LR addresses the within finger variability and is obtained by comparing the same configurations of minutiae coming from the same source. Only comparisons where the same minutiae are visible both on the mark and on the print are therefore taken into account. The denominator of the LR is obtained by cross-comparison with a database of prints originating from non-matching sources. The estimation of the numerator of the LR is much more complex in terms of specific data requirements than the estimation of the denominator of the LR (that requires only a large database of prints from an non-associated population). Hence this paper addresses specific issues associated with the numerator or within finger variability. This study aims at answering the following questions: (1) how a database for modelling within finger variability should be acquired; (2) whether or not the visualisation technique or the choice of different minutiae arrangements may influence that modelling and (3) what is the magnitude of LRs that can be expected from such a model. Results show that within finger variability is affected by the visualisation technique used on the mark, the number of minutiae and the minutiae configuration. They also show that the rates of misleading evidence in the likelihood ratios obtained for one of the configurations examined are low.     

Assessing the Impact of Community-oriented Policing on Arrest

Community-oriented policing’s (COP’s) influence and effect on policing has been significant in recent years with evidence suggesting that it increases police satisfaction, partially improves police legitimacy and reduces citizen’s fear of crime. It has also been hypothesized to influence the likelihood of an arrest; however, this possibility presents competing theoretical mechanisms and limited empirical assessment. The current study uses a multilevel analytic approach to merge data from the National Incident Based Reporting System data base with information from the Law Enforcement Management and Administrative Statistics survey to investigate this possibility. Results indicate that agencies engaged in greater and specific COP activities experience an increased likelihood of arrest in violent crime incidents. This effect varies by the type of violent crime and the amount of COP activity undertaken.     

Understanding of forensic expert reports by judges,defense lawyers and forensic professionals

Forensic scientists strongly advocate the use of likelihood ratios for expressing the diagnostic value of evidence in technical forensic reports. They call this the logically correct approach. The correct comprehension of such likelihood ratios by jurists, however, appears to be particularly problematic. The present research has empirically investigated this issue for defense lawyers and criminal judges. For comparative purposes forensic professionals, many of whom use the logically correct approach, were included in the study as well.

Using fictitious forensic reports, it was shown that proper understanding of likelihood ratios by jurists is quite poor, due mainly, but not exclusively, to the prosecutor's fallacy. Forensic professionals outperformed jurists to a large extent but made many mistakes themselves. It is further shown that participants’ self-expressed supposed level of understanding of logically correct conclusions is quite high and thus not justified by their levels of proper understanding. Suggestions for how to improve the situation are presented.     

Consideration of the probative value of single donor 15-plex STR profiles in UK populations and its presentation in UK courts

The adoption of new 15 locus STR multiplex systems into UK forensic science would be facilitated by agreed guidelines for reporting the strength of DNA evidence using likelihood ratios. To facilitate such an agreement, we present an analysis of previously published UK allele frequencies for white Caucasian, Afro-Caribbean and Indo-Pakistani populations and investigate their effect on likelihood ratios for single donor profiles. We consider the implication of the five additional loci and suggest a procedure for reporting likelihood ratios for 15-plex STR profiles.     

The likelihood approach to compare populations: a study on DNA evidence and pitfalls of intuitions

The paper follows on from earlier work [Taroni F and Aitken CGG. Probabilistic reasoning in the law, Part 1: assessment of probabilities and explanation of the value of DNA evidence. Science & Justice 1998; 38: 165-177]. Different explanations of the value of DNA evidence were presented to students from two schools of forensic science and to members of fifteen laboratories all around the world. The responses were divided into two groups; those which came from a school or laboratory identified as Bayesian and those which came from a school or laboratory identified as non-Bayesian. The paper analyses these responses using a likelihood approach. This approach is more consistent with a Bayesian analysis than one based on a frequentist approach, as was reported by Taroni F and Aitken CGG. [Probabilistic reasoning in the law, Part 1: assessment of probabilities and explanation of the value of DNA evidence] in Science & Justice 1998.     

Advanced Framework for Digital Forensic Technologies and Procedures*

Abstract: Recent trends in global networks are leading toward service‐oriented architectures and sensor networks. On one hand of the spectrum, this means deployment of services from numerous providers to form new service composites, and on the other hand this means emergence of Internet of things. Both these kinds belong to a plethora of realms and can be deployed in many ways, which will pose serious problems in cases of abuse. Consequently, both trends increase the need for new approaches to digital forensics that would furnish admissible evidence for litigation. Because technology alone is clearly not sufficient, it has to be adequately supported by appropriate investigative procedures, which have yet become a subject of an international consensus. This paper therefore provides appropriate a holistic framework to foster an internationally agreed upon approach in digital forensics along with necessary improvements. It is based on a top‐down approach, starting with legal, continuing with organizational, and ending with technical issues. More precisely, the paper presents a new architectural technological solution that addresses the core forensic principles at its roots. It deploys so‐called leveled message authentication codes and digital signatures to provide data integrity in a way that significantly eases forensic investigations into attacked systems in their operational state. Further, using a top‐down approach a conceptual framework for forensics readiness is given, which provides levels of abstraction and procedural guides embellished with a process model that allow investigators perform routine investigations, without becoming overwhelmed by low‐level details. As low‐level details should not be left out, the framework is further evaluated to include these details to allow organizations to configure their systems for proactive collection and preservation of potential digital evidence in a structured manner. The main reason behind this approach is to stimulate efforts on an internationally agreed “template legislation,” similarly to model law in the area of electronic commerce, which would enable harmonized national implementations in the area of digital forensics.     

Resolving the so-called “probabilistic paradoxes in legal reasoning” with Bayesian networks

Examples of reasoning problems such as the twins problem and poison paradox have been proposed by legal scholars to demonstrate the limitations of probability theory in legal reasoning. Specifically, such problems are intended to show that use of probability theory results in legal paradoxes. As such, these problems have been a powerful detriment to the use of probability theory – and particularly Bayes theorem – in the law. However, the examples only lead to ‘paradoxes’ under an artificially constrained view of probability theory and the use of the so-called likelihood ratio, in which multiple related hypotheses and pieces of evidence are squeezed into a single hypothesis variable and a single evidence variable. When the distinct relevant hypotheses and evidence are described properly in a causal model (a Bayesian network), the paradoxes vanish. In addition to the twins problem and poison paradox, we demonstrate this for the food tray example, the abuse paradox and the small town murder problem. Moreover, the resulting Bayesian networks provide a powerful framework for legal reasoning.     

Investigation Delayed Is Justice Denied: Proposals for Expediting Forensic Examinations of Digital Evidence*

Abstract: There is an urgent need to reduce the growing backlog of forensic examinations in Digital Forensics Laboratories (DFLs). Currently, DFLs routinely create forensic duplicates and perform in‐depth forensic examinations of all submitted media. This approach is rapidly becoming untenable as more cases involve increasing quantities of digital evidence. A more efficient and effective three‐tiered strategy for performing forensic examinations will enable DFLs to produce useful results in a timely manner at different phases of an investigation, and will reduce unnecessary expenditure of resources on less serious matters. The three levels of forensic examination are described along with practical examples and suitable tools. Realizing that this is not simply a technical problem, we address the need to update training and establish thresholds in DFLs. Threshold considerations include the likelihood of missing exculpatory evidence and seriousness of the offense. We conclude with the implications of scaling forensic examinations to the investigation.     

Confidence interval of the likelihood ratio associated with mixed stain DNA evidence

Likelihood ratios are necessary to properly interpret mixed stain DNA evidence. They can flexibly consider alternate hypotheses and can account for population substructure. The likelihood ratio should be seen as an estimate and not a fixed value, because the calculations are functions of allelic frequency estimates that were estimated from a small portion of the population. Current methods do not account for uncertainty in the likelihood ratio estimates and are therefore an incomplete picture of the strength of the evidence. We propose the use of a confidence interval to report the consequent variation of likelihood ratios. The confidence interval is calculated using the standard forensic likelihood ratio formulae and a variance estimate derived using the Taylor expansion. The formula is explained, and a computer program has been made available. Numeric work shows that the evidential strength of DNA profiles decreases as the variation among populations increases.     

A probabilistic approach to the joint evaluation of firearm evidence and gunshot residues

The present paper addresses issues that affect both the separate as well as the joint evaluation of firearm evidence (i.e., marks) and gunshot residues (GSR). Mark evidence will be used as a basis to discriminate among barrels through which a bullet in question might have been shot whereas GSR will be used to draw inferences about the distance of firing. Particular attention is drawn to the coherent handling of uncertainties associated with the various parameters considered within each item of evidence. The proposed analysis relies on a probabilistic viewpoint that uses graphical models (i.e., Bayesian networks) as an aid to cope with the complexity induced by the number of variables considered. The paper discusses how an approach based on a probabilistic network environment can be used for the formal analysis and construction of arguments. Emphasis is made on the gain of insight into structural dependencies that may be uncovered when the evaluative process is extended beyond single items of scientific evidence.     

Biological and DNA evidence in 1000 sexual assault cases

Using a Filemaker-based database (DNA Pro-FILES, Synchrone Infosystème Inc.), we have conducted a large-scale study on 1000 sexual assault (SA) cases where a standardized kit was submitted to our laboratory alone or with other types of exhibits. We looked at the likelihood of obtaining good quality DNA evidence, allegedly from the assailant, according to a number of parameters.The overall proportion of SA cases with DNA evidence is nearly 50%. A little more than 30% of SA kits provided DNA evidence while for 16% of cases DNA evidence could be obtained only from other exhibits.The likelihood of obtaining DNA evidence is approximately 50% in teenager and adult SA cases, but much lower for children 10 years old or younger (15%). In children cases, profiles were found mostly on clothing or skin swabs.The likelihood of obtaining DNA evidence from vaginal swabs remains good for up to 3 days after the assault (from 35% on the first day to 23% on the third day). A DNA profile was obtained from approximately 22% of anal/rectal swabs and 41% of skin swabs taken less than 1 day after the assault. Less than 10% of oral washes provided DNA evidence, all having been collected within 24 h of the assault.We found that in bodily samples, a negative result for acid phosphate (AP) is a poor predictor of the likelihood of obtaining good quality DNA evidence. Approximately 15% of vaginal swabs and 8% of anal swabs negative for AP nevertheless provided good quality DNA evidence.     

Evaluation and presentation of forensic DNA evidence in European laboratories.

The first part of this article describes Bayes' theorem and its practical application to scientific evidence. Secondly, statements proposed by European DNA laboratories are presented, analysed and commented upon. Finally, the authors reiterate their proposal of adopting a likelihood ratio framework justified by the usefulness of the approach in practical context of DNA cases. Note that this framework underlines the main questions the experts have to answer at trial: to what degree does the evidence support the prosecutor's hypothesis? and to what degree does the evidence support the defence's hypothesis?