Development and Implementation of an Effective Blind Proficiency Testing Program

Blind proficiency testing is ideal for testing crime laboratory personnel because the elements of analyst bias and anticipation are removed. However, sending proficiency tests through the laboratory system as real casework is difficult. The substantial challenges with preparing and administering blind tests may prevent laboratory managers from initiating blind testing. In 2015, the Harris County Institute of Forensic Sciences committed to improving its crime laboratory’s proficiency testing program by adding blind tests. The goal was to test the whole system, from evidence receipt to report release. With careful planning, trial-and-error, and ongoing assessment of available resources, not only was the program proven to be feasible, but there was also clear understanding of how to optimize our program. In this article, we share our experiences, lessons learned, and program details to assist other forensic service providers with developing their own blind testing programs, which would ultimately lead to improved quality assurance.     

Perceptions of blind proficiency testing among latent print examiners

In recent years, scholars have levied multiple criticisms against traditional proficiency testing procedures in forensic laboratories. Consequently, on several occasions, authorities have formally recommended that laboratories implement blind proficiency testing procedures. Implementation has been slow, but laboratory management has increasingly expressed interest in initiating blind testing in at least some forensic disciplines, with some laboratories conducting blind testing in almost all disciplines. However, little is known about how a key population perceives blind proficiency testing, i.e., forensic examiners. We surveyed active latent print examiners (N = 338) to explore perceptions of blind proficiency testing and determine whether beliefs varied between examiners who work for laboratories with and without blind proficiency testing. Results suggest that examiners do not hold particularly strong beliefs about such procedures, but that examiners who work in laboratories with blind proficiency testing procedures view them significantly more positively than those who do not. Further, examiner responses provide insight into potential obstacles to continued implementation.     

The feasibility of external blind DNA proficiency testing. II. Experience with actual blind tests

The background and goals of a national study to determine the feasibility of blind proficiency testing in U.S. forensic DNA laboratories are discussed. Part of the project involved designing and executing a series of fifteen blind proficiency tests. Execution included biological specimen donor recruitment and case evidence manufacturing. Simulated cases were submitted to DNA laboratories by law enforcement agencies and in some cases by other forensic-science laboratories. Replicate-manufactured evidence was submitted to reference laboratories to simulate the workings of a larger-scale program. Ten tests were straightforward, and essentially tested analytical ability. Five tests involved selecting on the basis of case facts appropriate bloodstains for typing from a bloodstain pattern. We describe in detail our experience in designing and conducting these blind proficiency test trials, and relate those experiences to the overall issue of blind proficiency testing as a quality-assurance tool in forensic DNA laboratories. In this feasibility test series, one blind test was detected by a laboratory, a second one was shown to the lab by law enforcement, and a third was never completed because of lapses in communication. Turnaround times were relatively fast in the independent/commercial labs and relatively slow in the larger public laboratories. Two cross-state case-to-case CODIS "hits" were "planted" among the first series of ten blind tests. One pair was detected. One member of the second pair went to a lab that was not CODIS-ready.     

The Importance of Blind Forensic Odontology Analyses

Forensic odontology cases examined from 2012 to 2019 at the Defense POW/MIA Accounting Agency (DPAA) laboratory were reviewed. Five cases from World War II and the Korea War were selected. Three of the cases presented involve dental assemblages, which were built by previous analysts. Postmortem radiographic analyses at the DPAA laboratory of these cases were complicated by the use of an older version of a digital x-ray program and the assumption that teeth placed in the parent bone (maxilla and mandible) did not fully articulate due to the presence of debris at their root apices. Conflicting mitochondrial DNA test results for the submitted teeth and previously believed to be parent bone indicated these elements were not from the same individual. The remaining two cases are examples of how knowledge of findings from other disciplines may sway an opinion and could possibly lead to the rendering of erroneous opinions by the forensic odontologist. Having knowledge of previous examinations or results from other scientists, disciplines, and lines of evidence may lead to a bias in findings or opinions. If not careful, even the best-intended scientist may fall victim to a bias in their opinion/analysis. The forensic odontologist should perform all analysis in the blind, not knowing any information which may bias their opinion, and utilize current versions of digital x-ray software available and their tools when performing their examinations.     

Results of a proposed breath alcohol proficiency test program

Although proficiency test programs have long been used in both clinical and forensic laboratories, they have not found uniform application in forensic breath alcohol programs. An initial effort to develop a proficiency test program appropriate to forensic breath alcohol analysis is described herein. A total of 11 jurisdictions participated in which 27 modern instruments were evaluated. Five wet bath simulator solutions with ethanol vapor concentrations ranging from 0.0254 to 0.2659 g/210 L were sent to participating programs, instructing them to perform n = 10 measurements on each solution using the same instrument. Four of the solutions contained ethanol only and one contained ethanol mixed with acetone. The systematic errors for all instruments ranged from -11.3% to +11.4% while the coefficient of variations ranged from zero to 6.1%. A components-of-variance analysis revealed at least 79% of the total variance as being due to the between-instrument component for all concentrations. Improving proficiency test program development should consider: (1) clear protocol instructions, (2) frequency of proficiency testing, (3) use lower concentrations for determining limits-of-detection and -quantitation, etc. Despite the lack of a biological component, proficiency test participation should enhance the credibility of forensic breath test programs.     

法医精神病学能力验证相关问题探讨

能力验证是利用实验室间比对,按照预先制定的准则评价参加者的能力,是世界各国用以评价和保障实验室/机构能力所采用的重要手段.在我国司法鉴定领域,能力验证已逐渐成为执业管理、监督管理的重要手段.根据管理部门的要求,司法鉴定机构应通过参加能力验证活动证明其具有相应的技术能力,并作为内部质量控制、发现问题和持续改进的重要途径.作为公认的主观性较强、风险较高的司法鉴定行业,在法医精神病专业开展能力验证活动更具必要性与迫切性.近几年的运作尝试也表明,在本专业开展能力验证活动是可行的,评价结果亦是行业技术水平的真实反映.但在项目设计与运作上还有完善空间,如样本与物品制备可真实、完备些,可争取更大的技术指导与示范引导作用,应合理控制考核难度,并增强专家公议结果的可靠程度、可接受性,且运作方式应更灵活、多样.参加机构应提高对能力验证的认识,加强业务培训,科学对待;作业时严格遵守作业要求,通读懂材料,寻找关键点,并注重要点与逻辑性,反馈结论要尽量明确、唯一,争取较满意评价结果.     

开展司法鉴定能力验证 确保持续鉴定能力

本文论述了能力验证活动监控司法鉴定机构和司法鉴定人的持续鉴定能力在司法鉴定质量控制和司法鉴定行业管理中的重要价值,并从理论和实践上阐明了能力验证活动的类型、组织和实施方法。     

The feasibility of external blind DNA proficiency testing. I. Background and findings

We describe the origins, purposes, and findings of a national study to determine whether a large-scale program of blind proficiency testing in U.S. DNA laboratories is feasible and/or practical. Proficiency testing in clinical laboratories is reviewed, particularly as mandated by the Clinical Laboratory Improvement Acts and its role in the regulation of those laboratories. Proficiency testing in forensic urine drug testing labs is also briefly reviewed. Studies involving comparisons between open and blind proficiency testing are discussed. The clinical laboratory proficiency testing and regulation landscape provides the background for the DNA Act of 1994, and the congressional mandate to investigate blind proficiency testing in forensic DNA laboratories. Four models of blind proficiency testing are defined and discussed, along with the advantages and disadvantages of each and estimates of the costs of a large-scale program. The purposes of proficiency testing in a quality-assurance context are likewise discussed and related to the models and the arguments generally proffered for and against blind vs. open proficiency testing.     

电子数据司法鉴定能力验证的设计与分析——以2021年电子数据功能性鉴定为例

电子数据鉴定作为司法程序中的重要环节,对于确保鉴定流程的规范性和鉴定意见的正确性,以及维护司法公正具有重要意义.针对电子数据司法鉴定实验室,能力验证是一种有效的质量控制与改进手段.电子数据功能性鉴定作为电子数据鉴定的重要组成部分,有必要利用能力验证进行质量控制.通过梳理2021年电子数据功能性鉴定能力验证(2021SF...     

Organizing a Proficiency Testing Program on Stamp Impressions Examination in Accordance with ISO/IEC 17043 Requirements

Stamps and stamp impressions examinations are based on matching defects and design details on the questioned and control samples. These examinations are routinely carried out by document examiners around the world. International proficiency tests for questioned documents examination have been available for decades while similar programs specifically focusing on stamp impressions examination are rare. This study reported a recent proficiency testing program on stamp impressions examination organized by an accredited provider in accordance with ISO/IEC 17043 requirements. Twenty‐four forensic laboratories registered for the program. Apart from giving details on the design and operation of the program, this study also aimed to provide the limitations and difficulties encountered in sample preparations, homogeneity, and stability tests of the testing materials. Various comments and feedbacks received from the participants, particularly in respect of examination approaches, challenges faced by the participants in forming conclusions and their suggestions for further improvement would be evaluated.     

The FAA's postmortem forensic toxicology self-evaluated proficiency test program: the first seven years

Existing proficiency-testing (PT) programs do not address the complexity of postmortem forensic toxicology. These programs do not include decomposed samples and solid tissues. Therefore, the Federal Aviation Administration in July 1991 started such a needed PT program. This program is used to: (i) professionally develop and maintain technical currency on a voluntary, interlaboratory, and self-evaluation basis, and (ii) quantifiably assess methods in the absence and presence of interfering substances. There are currently about 30 laboratories in the program. Functioning under various governmental/non-governmental agencies and academic institutions, these laboratories represent a broad cross-section of the country. PT samples are distributed quarterly, and result summaries are sent to the participants, while maintaining their anonymity. Since the inception of the program, 28 PT samples encompassing whole blood, plasma, urine, kidney, or liver, with (or without) drugs, metabolites, and common chemicals (nicotine, caffeine, beta-phenylethylamine, etc.) have been analyzed by the participants. Analytical findings were generally consistent with the anticipated values, but they were dependent on the nature and conditions of the specimens and types of the added analytes. Some incidences of false positives of concern were noted, as well. This PT program is one of the few programs recommended by the American Board of Forensic Toxicology in which laboratories may participate for their accreditation by the Board. It is anticipated that this PT program will continue to play a critical part in supporting the quality assurance/quality control (QA/QC) component of forensic toxicology, thereby enhancing operational performance.     

An American proficiency testing program

In 1975 the Forensic Sciences Foundation Inc. (FSF) under a grant from the Law Enforcement Assistance Administration (LEAA) in the United States carried out a research project on the design and execution of a proficiency testing program for crime laboratories. Following completion of that research, FSF Inc. affiliated with Collaborative Testing Services Inc. (CTS) to maintain an operational program on a cost recoverable basis and with the assistance of a professional advisory committee (PAC) appointed by the American Society of Crime Laboratory Directors (ASCLD).This paper discusses the problems of running a proficiency testing program of this type on a national/international basis for a large number of laboratories and covering a variety of evidence categories. Problems of confidentiality of results, test design and production, results analysis and reporting are emphasized. Some evaluation is made of the general types of results reported in this program.     

论司法鉴定人资格认证框架的构建

通过对我国司法鉴定人资格管理中存在的问题分析,在借鉴相关行业管理模式的基础上,以国际标准化组织发布的ISO/IEC17024准则为指导.提出将人员认证与鉴定人职业资格相结合,将资格考试和能力验证作为鉴定人认证的主要评价手段,尝试在现有法律制度和管理模式下构建我国司法鉴定人资格认证的基本框架.     

Histopathological findings of cardiac conduction system of 150 Finns

One hundred and fifty heart specimens were collected from the cases submitted for autopsy in the Department of Forensic Medicine in the University of Turku in March-May 1995 and May-July 1996, respectively. The cardiac conduction system (CCS) of these hearts were examined in order to find out the histopathological changes in the CCS of Finnish persons and their forensic pathological significance. There were 94 males and 56 females. Almost all age groups were included. The results revealed that in most of the persons the fibrous tissue and fatty tissue in the CCS increased with aging. In about half of the persons, there was deposition of calcium in the central fibrous body, pars membranacea, and the top of the musculature in the interventricular septum. In seven cases, the atrioventricular node (AVN), His bundle (HB) or bundle branches (RBB, LBB) were compressed by the calcium deposition. Hemorrhage, inflammation, amyloidosis, tumor, fatty infiltration and developmental malformations were observed in 31 cases. Twenty-eight cases died of myocardial infarct, among them, no involvement of the CCS could be observed. The authors concluded that routine examination of the CCS is helpful for revealing diseases of the CCS and improving the quality of forensic pathological diagnosis.     

Latent Print Proficiency Testing: An Examination of Test Respondents,Test-Taking Procedures,and Test Characteristics

Proficiency testing is a key component of quality assurance programs within crime laboratories and can help improve laboratory practices. However, current proficiency testing procedures contain significant limitations and can be misinterpreted by examiners and court personnel (Garrett & Mitchell, 2018). To evaluate some of these limitations, we surveyed latent print examiners (n = 198) after they completed a Collaborative Testing Services, Inc. proficiency test. Additionally, we evaluated test performance and used a quality metric algorithm to evaluate the quality of test prints. Results do not suggest that respondents are dissimilar to the broader examiner population, although they may engage in different behaviors when completing tests versus casework. Findings show that proficiency testing contains prints of high quality and is perceived as both relatively easy and representative of casework. The test discriminated between inexperienced and experienced respondents, and verification procedures were largely ineffective in reducing errors. Objective quality metrics may provide a path forward to improving proficiency testing in a measurable manner.     

比较法视野下的司法鉴定制度的反思

司法鉴定是依据科学的基础原理由专业的鉴定人员做出结论的实证技术活动。它是科学的产物,但是不一定等于科学。如果不能对错误司法鉴定进行区分、鉴别的话,其结果只能是一个盲人领着一群盲人。司法鉴定制度有其独特的运行规律,也有其异域的可以借鉴之处,这些也是保证司法鉴定科学、合法、准确必须抓住的要素。     

Forensic laboratory backlog: The impact of inconclusive results of marijuana analysis and the implication on analytical routine

Forensic laboratories worldwide are struggling to keep up with the increasing number of cases submitted for analysis, regardless of the reasons, backlog of controlled substances cases is a reality in many countries. In this paper we analyse the number of petitioned examinations (from 2016 to 2020) and the data from 11,655 marijuana TLC results from the Forensic Laboratory in the Federal District Civil Police in Brazil. Data demonstrates that backlog increases inconclusive results, with storage and light playing a crucial role in the process. Additionally we explored the repercussions of delayed forensic results for controlled substances and propose an approach to overcome waiting time in this context.     

人牙咬痕同一认定数字化分析的盲法评定

目的评定人牙咬痕同一认定数字化分析方法的精确性。方法 采用数字化分析法对人牙咬痕及8名“疑犯”牙列模型进行双盲法分析实验,用Photoshop 5.5将获得的咬痕扫描图像生成overlay,用AutoCAD Rl4工程测量软件分析各项参数,比较人牙咬痕和“疑犯”牙列参数的匹配性。结果人牙咬痕及“疑犯”牙列模型数据化分析的各项参数分析结果一致。结论对于试验性咬痕,数字化分析是一种可行的认定方法,对法医学实践具有可行性及良好的应用前景。     

法医精神病鉴定内容架构及评定要点解析

法医精神病鉴定在刑事、民事和行政三大诉讼领域中,以鉴定意见的证据形式发挥着重要作用。然而,与其重要性不相匹配的是,法医精神病鉴定学科发展尚不完善、专业发育尚不成熟,尚不能满足社会和民众的期盼和要求,甚至引发负面评价。为进一步促进法医精神病鉴定规范化、标准化建设。以法医精神病鉴定人的视角,结合法医精神病鉴定的内容架构,重点阐述法医精神病鉴定主要项目及其作用、评定要点,尤其聚焦刑事责任能力评定,阐明当前存在分歧和困难;简要介绍法医精神病鉴定人执业要求、法医精神病鉴定质量控制;以及简述法医精神病鉴定与临床精神医学实践的区别与联系。