Implementation of a Blind Quality Control Program in a Forensic Laboratory

A blind quality control (QC) program was successfully developed and implemented in the Toxicology, Seized Drugs, Firearms, Latent Prints (Processing and Comparison), Forensic Biology, and Multimedia (Digital and Audio/Video) sections at the Houston Forensic Science Center (HFSC). The program was put into practice based on recommendations set forth in the 2009 National Academy of Sciences report and is conducted in addition to accreditation required annual proficiency tests. The blind QC program allows HFSC to test its entire quality management system and provides a real-time assessment of the laboratory’s proficiency. To ensure the blind QC cases mimicked real casework, the workflow for each forensic discipline and their evidence submission processes were assessed prior to implementation. Samples are created and submitted by the HFSC Quality Division to whom the expected answer is known. Results from 2015 to 2018 show that of the 973 blind samples submitted, 901 were completed, and only 51 were discovered by analysts as being blind QC cases. Implementation data suggests that this type of program can be employed at other forensic laboratories.     

The feasibility of external blind DNA proficiency testing. II. Experience with actual blind tests

The background and goals of a national study to determine the feasibility of blind proficiency testing in U.S. forensic DNA laboratories are discussed. Part of the project involved designing and executing a series of fifteen blind proficiency tests. Execution included biological specimen donor recruitment and case evidence manufacturing. Simulated cases were submitted to DNA laboratories by law enforcement agencies and in some cases by other forensic-science laboratories. Replicate-manufactured evidence was submitted to reference laboratories to simulate the workings of a larger-scale program. Ten tests were straightforward, and essentially tested analytical ability. Five tests involved selecting on the basis of case facts appropriate bloodstains for typing from a bloodstain pattern. We describe in detail our experience in designing and conducting these blind proficiency test trials, and relate those experiences to the overall issue of blind proficiency testing as a quality-assurance tool in forensic DNA laboratories. In this feasibility test series, one blind test was detected by a laboratory, a second one was shown to the lab by law enforcement, and a third was never completed because of lapses in communication. Turnaround times were relatively fast in the independent/commercial labs and relatively slow in the larger public laboratories. Two cross-state case-to-case CODIS "hits" were "planted" among the first series of ten blind tests. One pair was detected. One member of the second pair went to a lab that was not CODIS-ready.     

Perceptions of blind proficiency testing among latent print examiners

In recent years, scholars have levied multiple criticisms against traditional proficiency testing procedures in forensic laboratories. Consequently, on several occasions, authorities have formally recommended that laboratories implement blind proficiency testing procedures. Implementation has been slow, but laboratory management has increasingly expressed interest in initiating blind testing in at least some forensic disciplines, with some laboratories conducting blind testing in almost all disciplines. However, little is known about how a key population perceives blind proficiency testing, i.e., forensic examiners. We surveyed active latent print examiners (N = 338) to explore perceptions of blind proficiency testing and determine whether beliefs varied between examiners who work for laboratories with and without blind proficiency testing. Results suggest that examiners do not hold particularly strong beliefs about such procedures, but that examiners who work in laboratories with blind proficiency testing procedures view them significantly more positively than those who do not. Further, examiner responses provide insight into potential obstacles to continued implementation.     

Latent Print Proficiency Testing: An Examination of Test Respondents,Test-Taking Procedures,and Test Characteristics

Proficiency testing is a key component of quality assurance programs within crime laboratories and can help improve laboratory practices. However, current proficiency testing procedures contain significant limitations and can be misinterpreted by examiners and court personnel (Garrett & Mitchell, 2018). To evaluate some of these limitations, we surveyed latent print examiners (n = 198) after they completed a Collaborative Testing Services, Inc. proficiency test. Additionally, we evaluated test performance and used a quality metric algorithm to evaluate the quality of test prints. Results do not suggest that respondents are dissimilar to the broader examiner population, although they may engage in different behaviors when completing tests versus casework. Findings show that proficiency testing contains prints of high quality and is perceived as both relatively easy and representative of casework. The test discriminated between inexperienced and experienced respondents, and verification procedures were largely ineffective in reducing errors. Objective quality metrics may provide a path forward to improving proficiency testing in a measurable manner.     

The feasibility of external blind DNA proficiency testing. I. Background and findings

We describe the origins, purposes, and findings of a national study to determine whether a large-scale program of blind proficiency testing in U.S. DNA laboratories is feasible and/or practical. Proficiency testing in clinical laboratories is reviewed, particularly as mandated by the Clinical Laboratory Improvement Acts and its role in the regulation of those laboratories. Proficiency testing in forensic urine drug testing labs is also briefly reviewed. Studies involving comparisons between open and blind proficiency testing are discussed. The clinical laboratory proficiency testing and regulation landscape provides the background for the DNA Act of 1994, and the congressional mandate to investigate blind proficiency testing in forensic DNA laboratories. Four models of blind proficiency testing are defined and discussed, along with the advantages and disadvantages of each and estimates of the costs of a large-scale program. The purposes of proficiency testing in a quality-assurance context are likewise discussed and related to the models and the arguments generally proffered for and against blind vs. open proficiency testing.     

An American proficiency testing program

In 1975 the Forensic Sciences Foundation Inc. (FSF) under a grant from the Law Enforcement Assistance Administration (LEAA) in the United States carried out a research project on the design and execution of a proficiency testing program for crime laboratories. Following completion of that research, FSF Inc. affiliated with Collaborative Testing Services Inc. (CTS) to maintain an operational program on a cost recoverable basis and with the assistance of a professional advisory committee (PAC) appointed by the American Society of Crime Laboratory Directors (ASCLD).This paper discusses the problems of running a proficiency testing program of this type on a national/international basis for a large number of laboratories and covering a variety of evidence categories. Problems of confidentiality of results, test design and production, results analysis and reporting are emphasized. Some evaluation is made of the general types of results reported in this program.     

How do latent print examiners perceive proficiency testing? An analysis of examiner perceptions,performance, and print quality

Proficiency testing has the potential to serve several important purposes for crime laboratories and forensic science disciplines. Scholars and other stakeholders, however, have criticized standard proficiency testing procedures since their implementation in laboratories across the United States. Specifically, many experts label current proficiency tests as non-representative of actual casework, at least in part because they are not sufficiently challenging (e.g., [1], [2], [3], [4]. In the current study, we surveyed latent print examiners (n = 322) after they completed a Collaborative Testing Services proficiency test about their perceptions of test items. We also evaluated respondents’ test performance and used a quality metric algorithm (LQMetrics) to obtain objective indicators of print quality on the test. Results were generally consistent with experts’ concerns about proficiency testing. The low observed error rate, examiner perceptions of relative ease, and high objective print quality metrics together suggest that latent print proficiency testing is not especially challenging. Further, examiners indicated that the test items that most closely resembled real-world casework were also the most difficult and contained prints of the lowest quality. Study findings suggest that including prints of lower quality may increase both the difficulty and representativeness of proficiency testing in latent print examination.     

物证鉴定的能力验证

能力验证是物证鉴定质量保障体系的重要组成部分,可以非常有效的评估、监控和提高实验室物证鉴定能力和质量。本文论述了物证鉴定能力验证的主要方法、形式、类型和特点,以及在物证鉴定实验室质量管理中的具体应用,介绍了国外物证鉴定实验室参加能力验证活动的历史和现状,并提出在我国物证鉴定实验室广泛开展能力验证活动的思考。     

Organizing a Proficiency Testing Program on Stamp Impressions Examination in Accordance with ISO/IEC 17043 Requirements

Stamps and stamp impressions examinations are based on matching defects and design details on the questioned and control samples. These examinations are routinely carried out by document examiners around the world. International proficiency tests for questioned documents examination have been available for decades while similar programs specifically focusing on stamp impressions examination are rare. This study reported a recent proficiency testing program on stamp impressions examination organized by an accredited provider in accordance with ISO/IEC 17043 requirements. Twenty‐four forensic laboratories registered for the program. Apart from giving details on the design and operation of the program, this study also aimed to provide the limitations and difficulties encountered in sample preparations, homogeneity, and stability tests of the testing materials. Various comments and feedbacks received from the participants, particularly in respect of examination approaches, challenges faced by the participants in forming conclusions and their suggestions for further improvement would be evaluated.     

HAIRVEQ 2006: evolution of laboratories' performance after different educational actions

HAIRVEQ is a proficiency testing program for hair analysis of illicit drugs organized by the Istituto Superiore di Sanità (Rome, Italy) and the Institut Municipal d'Investigació Mèdica (Barcelona, Spain). The aim of the three exercises performed in 2006 was the evaluation of 32 laboratories' performance when analyzing the same hair sample containing opiates, cocaine and methadone, after carrying out some specific educational interventions. In the first round, the sample was sent to be analyzed following laboratory routine methodology. In the second round, standard operating procedures (SOP) for hair testing including sample preparation, method validation and qualitative and quantitative data evaluation, and an open hair sample for SOP training were also sent together with other hair samples including the one used for performance evaluation. After the second round, a workshop was held with participant laboratories to discuss methodological issues and interpretation of obtained results. An additional amount of open samples was distributed to the laboratories for implementing the SOPs. In the third round, the same unknown sample containing opiates, cocaine and methadone was resent for the final evaluation of laboratory performance. In the first round, 11 incorrect qualitative results (10 false negative and 1 false positive) were reported by seven laboratories (22%), in the second round, a reduction in the number of incorrect results was observed (4 false negatives and 1 false positive were reported by four laboratories, 13%) and in the third round, 5 false positives and 5 false negatives were reported by seven laboratories (22%). Concerning quantitative results, the scatter was similar between the three rounds and similar to the ones reported by other proficiency tests in hair analysis. More educational actions should be addressed to a group of laboratories, which did not yet show satisfying qualitative and quantitative results.     

Medicare, Medicaid and CLIA programs; extension of certain effective dates for clinical laboratory requirements and personnel requirements for cytologists--HCFA. Final rule with comment period

This final rule extends certain effective dates for clinical laboratory requirements in regulations published on February 28, 1992, which implemented provisions of the Clinical Laboratory Improvement Amendments of 1988 (CLIA) and announces our approval of a certifying organization for qualifying cytotechnologists. This rule extends the date by which an individual must enroll in an HCFA-approved cytology proficiency testing (PT) program and the date by which an individual with a doctoral degree must possess board certification to qualify as a director of a laboratory that performs high complexity testing. In addition, we are extending the phase-in of the quality control requirements applicable to unmodified, moderate complexity tests cleared for commercial distribution by the Food and Drug Administration (FDA). We are extending the date to meet applicable CLIA QC requirements for laboratories using commercial, nonmodified tests to fulfill certain quality control (QC) requirements. These effective date extensions do not reduce the current requirements for quality test performance. The date extensions are necessary due to the limited number and scope of currently operating cytology PT programs, resource constraints that have prevented commencement of the substantial number of quality control reviews, and inability of many laboratory directors to complete certification requirements within the time period originally specified.     

Proficiency Testing as a Basis for Estimating Uncertainty of Measurement: Application to Forensic Alcohol and Toxicology Quantitations

Abstract: While forensic laboratories will soon be required to estimate uncertainties of measurement for those quantitations reported to the end users of the information, the procedures for estimating this have been little discussed in the forensic literature. This article illustrates how proficiency test results provide the basis for estimating uncertainties in three instances: (i) For breath alcohol analyzers the interlaboratory precision is taken as a direct measure of uncertainty. This approach applies when the number of proficiency tests is small. (ii) For blood alcohol, the uncertainty is calculated from the differences between the laboratory’s proficiency testing results and the mean quantitations determined by the participants; this approach applies when the laboratory has participated in a large number of tests. (iii) For toxicology, either of these approaches is useful for estimating comparability between laboratories, but not for estimating absolute accuracy. It is seen that data from proficiency tests enable estimates of uncertainty that are empirical, simple, thorough, and applicable to a wide range of concentrations.     

开展司法鉴定能力验证 确保持续鉴定能力

本文论述了能力验证活动监控司法鉴定机构和司法鉴定人的持续鉴定能力在司法鉴定质量控制和司法鉴定行业管理中的重要价值,并从理论和实践上阐明了能力验证活动的类型、组织和实施方法。     

Development and Validation of a Virtual Examination Tool for Firearm Forensics, ,

The transition from 2D imaging to 3D scanning in the discipline of firearms and toolmark analysis is likely to provide examiners an unprecedented view of microscopic surface topography. The digital examination of measured 3D surface topographies has been referred to as virtual microscopy (VM). The approach offers several potential advantages over traditional comparison microscopy. Like any new analytic method, VM must be validated prior to its use in a crime laboratory. This paper describes one of the first validation studies of virtual microscopy. Fifty‐six participants at fifteen laboratories used virtual microscopic tools to complete two proficiency‐style tests for cartridge case identification. All participating trained examiners correctly reported 100% of the identifications (known matches) while reporting no false positives. The VM tools also allowed examiners to annotate compared surfaces. These annotations provide insight into the types of marked utilized in comparative analysis. Overall, the results of the study demonstrate that trained examiners can successfully use virtual microscopy to conduct firearms toolmark examination and support the use of the technology in the crime laboratory.     

Functional MRI Detection of Deception After Committing a Mock Sabotage Crime*

Abstract: Using Blood Oxygen Level Dependent (BOLD) functional MRI (fMRI) to detect deception is feasible in simple laboratory paradigms. A mock sabotage scenario was used to test whether this technology would also be effective in a scenario closer to a real‐world situation. Healthy, nonmedicated adults were recruited from the community, screened, and randomized to either a Mock‐crime group or a No‐crime group. The Mock‐crime group damaged and stole compact discs (CDs), which contained incriminating video footage, while the No‐crime group did not perform a task. The Mock‐crime group also picked up an envelope from a researcher, while the No‐crime group did not perform this task. Both groups were instructed to report that they picked up an envelope, but did not sabotage any video evidence. Participants later went to the imaging center and were scanned while being asked questions regarding the mock crime. Participants also performed a simple laboratory based fMRI deception testing (Ring‐Watch testing). The Ring‐Watch testing consisted of “stealing” either a watch or a ring. The participants were instructed to report that they stole neither object. We correctly identified deception during the Ring‐Watch testing in 25 of 36 participants (Validated Group). In this Validated Group for whom a determination was made, computer‐based scoring correctly identified nine of nine Mock‐crime participants (100% sensitivity) and five of 15 No‐crime participants (33% specificity). BOLD fMRI presently can be used to detect deception concerning past events with high sensitivity, but low specificity.     

从亲子鉴定的问题看标准化规范化管理的重要性

近年来,亲子鉴定技术在我国发展较快,越来越多的人应用这一鉴定手段维护了自己的合法权益.但我们发现,我国目前对鉴定部门的管理却相对滞后,主要表现为实验室缺乏规范化管理、鉴定机构良莠不齐、技术标准不够完备以及缺乏实验室的质量检测体系等.这些问题如不解决,将不适应司法体制改革的趋势,阻碍鉴定技术科学、公正地为司法实践服务.因此,我们呼吁有关部门对这一问题加以重视、加强协作,使科学鉴定早日走上标准化、规范化管理的道路.     

HLA血型亲权鉴定系统分析软件的开发及应用

应用FoxPro2．5系统开发了HLA亲权鉴定软件,程序运行中有汉字提示,使用方便。对63例HLA亲权鉴定案例进行复核计算,所得结果与人工计算结果一致,证明程序计算结果可靠,可在实际检案中应用。     

Forensic science does not start in the lab: the concept of diagnostic field tests

An attempt to improve an analytical system can focus either on the actual processing or on the input. In forensic science, much emphasis has been placed on improving laboratory procedures, as though the input is already the best that can be obtained. Means of improving the basic input have gained much less attention. Yet, it must be agreed that even the best laboratory cannot gain from an item more than has been contained in it when it arrived from the field. The detection of latent materials at the crime scene by physical or chemical techniques and the diagnostic examination of material already discovered belong to the concept of diagnostic field tests. This group also includes "mapping" for the presence of certain materials, such as latent fingerprints through the distribution of amino acids on the surface. These tests are conducted outside the laboratory, without sophisticated instrumentation, at the crime scene, the suspect's home, or elsewhere. A significant advantage of the use of diagnostic field tests is the ability to deal with "dissipating evidence" such as gunshot residue or explosive traces on the hands of suspects. If time is lost, there is a risk of losing such evidence, which is liable to deteriorate rapidly. In my presentation, I will discuss older and some newly developed forensic field tests, with specific emphasis on the Israeli experience.     

Role of chemical tests and scene investigation in determination of range of fire

Forensic pathologists have historically found several characteristics of the entrance wound invaluable in determining range of fire in gunshot fatalities. Among these characteristics are the pattern and constituents of any deposited material such as soot and/or gun powder residues. We describe a case in which the application of previously described characteristics, in the absence of laboratory testing and examination of the crime scene, would have led to an erroneous conclusion with potentially grave consequences. We suggest that all attempts be made to use available laboratory tests and to perform detailed examination of crime scenes in determining the circumstances surrounding fatal gunshot injuries.     

Workplace drug testing in a military organization: Results and experiences from the testing program in the Finnish Defence Forces

In the military environment drug abuse is a particular risk for occupational safety. In the Finnish Defence Forces a drug testing program was conducted in 2002–2005; soldiers, professional civilians, and military students were tested when applying for a work or right to study; furthermore, annually 5% of the personnel were subjected to random testing. In total, over 2000 urine samples were analyzed in an accredited laboratory for cannabis, opiates, amphetamines, or cocaine. In this article, the drug testing program as a part of the anti-drug strategy of the Finnish Defence Forces is described, and the findings including practical experiences and financial expenses are reported. Only one person applying for a civilian post tested positive for amphetamine and cannabis. In seven other samples codeine and morphine were detected; these were, however, due to prescribed medication, not drug abuse. In the execution of the program, no particular difficulties were reported. In conclusion, it seems that the use of illicit drugs in the Finnish military is extremely rare, at least partly due to the successful anti-drug strategy. After an elaborate planning, even an extensive drug testing program can be executed without substantial setbacks. In the future, the effectiveness of drug testing programs as a means of improving occupational safety needs to be investigated in controlled studies using comparative design.