Shoeprint image retrieval and crime scene shoeprint image linking by using convolutional neural network and normalized cross correlation

A shoeprint image retrieval process aims to identify and match images of shoeprints found at crime scenes with shoeprint images from a known reference database. It is a challenging problem in the forensic discipline of footwear analysis because a shoeprint found at the crime scene is often imperfect. Recovered shoeprints may be partial, distorted, left on surfaces that do not mark easily, or perhaps come from shoes that do not transfer marks easily. In this study, we present a shoeprint retrieval method by using a convolutional neural network (CNN) and normalized cross-correlation (NCC). A pre-trained CNN was used to extract features from the pre-processed shoeprint images. We then employed NCC to compute a similarity score based on the extracted image features. We achieved a retrieval accuracy of 82% in our experiments, where a “successful” retrieval means that the ground truth image was returned in the top 1% of returned images. We also extend our shoeprint retrieval method to the problem of linking shoeprints recovered from crime scenes. This new method can provide a linkage between two crime scenes if the two recovered shoeprints originated from the same shoe. This new method achieved a retrieval accuracy of 88.99% in the top 20% of returned images.     

Developing a Spatial-Temporal Method for the Geographic Investigation of Shoeprint Evidence

Abstract:  This article examines the potential of a spatial-temporal method for analysis of forensic shoeprint data. The large volume of shoeprint evidence recovered at crime scenes results in varied success in matching a print to a known shoe type and subsequently linking sets of matched prints to suspected offenders. Unlike DNA and fingerprint data, a major challenge is to reduce the uncertainty in linking sets of matched shoeprints to a suspected serial offender. Shoeprint data for 2004 were imported from the Greater London Metropolitan Area Bigfoot database into a geographic information system, and a spatial-temporal algorithm developed for this project. The results show that by using distance and time constraints interactively, the number of candidate shoeprints that can implicate one or few suspects can be substantially reduced. It concludes that the use of space-time and other ancillary information within a geographic information system can be quite helpful for forensic investigation.     

Survey of 1276 shoeprint impressions and development of an automatic shoeprint pattern matching facility.

1276 shoeprints were collected at a scientific exhibition. Details regarding the age groups of the participants, style, size and manufacturer/brand of their shoes were recorded. The impressions were assigned to pattern groups showing that the most common pattern was present in only 1% of the population studied and most patterns were much less common. The impressions were digitized and a system developed for automatically sorting a database of images of outsole patterns in response to a reference image. The database images are ranked so that those from the same pattern group as the reference shoeprint are likely to be at the start of the list. A database of 486 complete shoeprint images belonging to 142 pattern groups was established with each group containing two or more examples. Tests of the system have shown that the first-ranked database image belongs to the same pattern group as the reference image 60% of the time and that a correct match appears within the first 5% of the ranked images 88% of the time. The system has translational and rotational invariance so that the spatial positioning of the reference shoeprint images does not have to correspond with the spatial positioning of the shoeprint images of the database. The performance of the system for matching partial shoeprints was also determined.     

Robust Shoeprint Retrieval Method Based on Local‐to‐Global Feature Matching for Real Crime Scenes

In this study, an automatic and robust crime scene shoeprint retrieval method is proposed. As most shoeprints left at crime scenes are randomly partial and noisy, crime scene shoeprint retrieval is a challenging task. To handle partial, noisy shoeprint images, we employ denoising deep belief network (DBN) to extract local features and use spatial pyramid matching (SPM) to obtain a local‐to‐global matching score. In this study, 536 query shoeprint images from crime scenes and a large scale database containing 34,768 shoeprint images are used to evaluate the retrieval performance. Experimental results show that the proposed method outperforms other state‐of‐the‐art methods in terms of retrieval accuracy, feature dimension, and retrieval speed. The proposed method achieves a cumulative match score (CMS) of 65.67% at top 10 which is 5.60% higher than the second best performing method.     

The role and duties of the shoeprint/toolmark examiner in forensic laboratories

Shoeprint and toolmark examinations have been carried out for forensic purposes for a great number of years. According to literature, the first case of footwear identification dates back to the year 1786. This particular case was worked out by a local police constable. Who then, are the persons working in crime laboratories and specialising in shoeprint and toolmark examinations today? Are they responsible for some other kind of duties, and if ‘yes’, what kind of duties? To find answers to these questions, a questionnaire was prepared and sent to 78 forensic laboratories, 62 laboratories in 27 European countries and 16 laboratories in six countries outside Europe. Forty-three laboratories (55%) in 23 countries; 35 laboratories in 19 European countries and eight laboratories in four countries outside Europe returned the questionnaire. The conducted study shows that the shoeprint and toolmark examiners are performing a great variety of tasks besides shoeprint and toolmark examinations, and also that the tasks vary a lot from one laboratory to another.     

模糊图像的最大熵恢复

首先介绍了由Hollis等人提出的最大熵图像恢复算法,该方法经过DavidS．C．Biggs等提出的方法加速后可以大大缩短处理时间。列举运用该加速算法恢复一些实际模糊图像的例子,其中包括运动模糊图像和离焦模糊图像,这些类型的模糊图像都是司法实践中常见的。良好的恢复结果证明了这一方法的有效性及其在司法领域中应用的可行性。     

Likelihood Ratios for Deep Neural Networks in Face Comparison

In this study, we aim to compare the performance of systems and forensic facial comparison experts in terms of likelihood ratio computation to assess the potential of the machine to support the human expert in the courtroom. In forensics, transparency in the methods is essential. Consequently, state-of-the-art free software was preferred over commercial software. Three different open-source automated systems chosen for their availability and clarity were as follows: OpenFace, SeetaFace, and FaceNet; all three based on convolutional neural networks that return a distance (OpenFace, FaceNet) or similarity (SeetaFace). The returned distance or similarity is converted to a likelihood ratio using three different distribution fits: parametric fit Weibull distribution, nonparametric fit kernel density estimation, and isotonic regression with pool adjacent violators algorithm. The results show that with low-quality frontal images, automated systems have better performance to detect nonmatches than investigators: 100% of precision and specificity in confusion matrix against 89% and 86% obtained by investigators, but with good quality images forensic experts have better results. The rank correlation between investigators and software is around 80%. We conclude that the software can assist in reporting officers as it can do faster and more reliable comparisons with full-frontal images, which can help the forensic expert in casework.     

The Use of Scale‐Invariance Feature Transform Approach to Recognize and Retrieve Incomplete Shoeprints

Shoeprints left at the crime scene provide valuable information in criminal investigation due to the distinctive patterns in the sole. Those shoeprints are often incomplete and noisy. In this study, scale‐invariance feature transform is proposed and evaluated for recognition and retrieval of partial and noisy shoeprint images. The proposed method first constructs different scale spaces to detect local extrema in the underlying shoeprint images. Those local extrema are considered as useful key points in the image. Next, the features of those key points are extracted to represent their local patterns around key points. Then, the system computes the cross‐correlation between the query image and each shoeprint image in the database. Experimental results show that full‐size prints and prints from the toe area perform best among all shoeprints. Furthermore, this system also demonstrates its robustness against noise because there is a very slight difference in comparison between original shoeprints and noisy shoeprints.     

Proposing Problem-Based Learning for teaching future forensic speech scientists

In the last 10–15 years, Masters programmes and undergraduate modules have emerged in the UK that teach forensic speech science. Forensic speech science is the forensic subdiscipline concerned with analysing speech recordings, such as telephone calls of unknown speakers, when they arise as evidence. In order to answer questions surrounding the identity of the speakers in these recordings, forensic speech analysts draw on their expertise in phonetics and acoustics. Even though existing UK forensic speech science programmes do not claim to train students to a level where they are in a position to carry out real-life forensic casework, a proportion of the graduates from these programmes do go on to fill discipline-specific roles in security organisations or for private providers of forensic speech analysis. It is therefore surely in the community’s interests to review educational approaches to capitalise on the current training opportunities. This paper specifically proposes to explore the potential of a Problem-Based Learning (PBL) approach to forensic speech science teaching. PBL is a student-centred learning approach that heavily relies on the students’ independence in the solving of ill-structured problems. PBL has shown to be beneficial to programmes that directly lead on to discipline-specific professional roles, and has even become the standardised teaching approach in some of those areas (medicine being the flagship example). Given its reported success in other disciplines, the question arises as to whether PBL could bring similar benefits to prospective forensic speech practitioners and to forensic speech science as a whole.     

A New Anti‐forensic Scheme—Hiding the Single JPEG Compression Trace for Digital Image

To prevent image forgeries, a number of forensic techniques for digital image have been developed that can detect an image's origin, trace its processing history, and can also locate the position of tampering. Especially, the statistical footprint left by JPEG compression operation can be a valuable source of information for the forensic analyst, and some image forensic algorithm have been raised based on the image statistics in the DCT domain. Recently, it has been shown that footprints can be removed by adding a suitable anti‐forensic dithering signal to the image in the DCT domain, this results in invalid for some image forensic algorithms. In this paper, a novel anti‐forensic algorithm is proposed, which is capable of concealing the quantization artifacts that left in the single JPEG compressed image. In the scheme, a chaos‐based dither is added to an image's DCT coefficients to remove such artifacts. Effectiveness of both the scheme and the loss of image quality are evaluated through the experiments. The simulation results show that the proposed anti‐forensic scheme can verify the reliability of the JPEG forensic tools.     

Contextual information management: An example of independent-checking in the review of laboratory-based bloodstain pattern analysis

This article describes a New Zealand forensic agency's contextual information management protocol for bloodstain pattern evidence examined in the laboratory. In an effort to create a protocol that would have minimal impact on current work-flow, while still effectively removing task-irrelevant contextual information, the protocol was designed following an in-depth consultation with management and forensic staff. The resulting design was for a protocol of independent-checking (i.e. blind peer-review) where the checker's interpretation of the evidence is conducted in the absence of case information and the original examiner's notes or interpretation(s). At the conclusion of a ten-case trial period, there was widespread agreement that the protocol had minimal impact on the number of people required, the cost, or the time to complete an item examination. The agency is now looking to adopt the protocol into standard operating procedures and in some cases the protocol has been extended to cover other laboratory-based examinations (e.g. fabric damage, shoeprint examination, and physical fits). The protocol developed during this trial provides a useful example for agencies seeking to adopt contextual information management into their workflow.     

High-speed search using Tarari content processor in digital forensics

Recently, “Speed” is one of the hot issues in digital forensics. Thanks to a recent advanced technology, today we can get bigger hard drive disks at a lower price than previously. But unfortunately, it means for forensic investigators that they need tremendous time and effort in the sequence of process of creating forensic images, searching into them and analyzing them. In order to solve this problem, some methods have been proposed to improve performance of forensic tools. One of them getting attention is a hardware-based approach. However, such a way is limited in the field of evidence cloning or password cracking while it is rarely used in searching and analysis of the digital evidence. In this paper, we design and implement a high-speed search engine using a Tarari content processor. Furthermore, we show feasibility of our approach by comparing its performance and features to those of a popular forensic tool currently on the market.     

High-speed search using Tarari content processor in digital forensics

Recently, “Speed” is one of the hot issues in digital forensics. Thanks to a recent advanced technology, today we can get bigger hard drive disks at a lower price than previously. But unfortunately, it means for forensic investigators that they need tremendous time and effort in the sequence of process of creating forensic images, searching into them and analyzing them. In order to solve this problem, some methods have been proposed to improve performance of forensic tools. One of them getting attention is a hardware-based approach. However, such a way is limited in the field of evidence cloning or password cracking while it is rarely used in searching and analysis of the digital evidence. In this paper, we design and implement a high-speed search engine using a Tarari content processor. Furthermore, we show feasibility of our approach by comparing its performance and features to those of a popular forensic tool currently on the market.     

A novel file carving algorithm for National Marine Electronics Association (NMEA) logs in GPS forensics

Globe positioning system (GPS) devices are an increasing importance source of evidence, as more of our devices have built-in GPS capabilities. In this paper, we propose a novel framework to efficiently recover National Marine Electronics Association (NMEA) logs and reconstruct GPS trajectories. Unlike existing approaches that require file system metadata, our proposed algorithm is designed based on the file carving technique without relying on system metadata. By understanding the characteristics and intrinsic structure of trajectory data in NMEA logs, we demonstrate how to pinpoint all data blocks belonging to the NMEA logs from the acquired forensic image of GPS device. Then, a discriminator is presented to determine whether two data blocks can be merged. And based on the discriminator, we design a reassembly algorithm to re-order and merge the obtained data blocks into new logs. In this context, deleted trajectories can be reconstructed by analyzing the recovered logs. Empirical experiments demonstrate that our proposed algorithm performs well when the system metadata is available/unavailable, log files are heavily fragmented, one or more parts of the log files are overwritten, and for different file systems of variable cluster sizes.     

Quantitative evaluation of the results of digital forensic investigations: a review of progress

Unlike conventional forensics, digital forensics does not at present generally quantify the results of its investigations. It is suggested that digital forensics should aim to catch up with other forensic disciplines by using Bayesian and other numerical methodologies to quantify its investigations’ results. Assessing the plausibility of alternative hypotheses (or propositions, or claims) which explain how recovered digital evidence came to exist on a device could assist both the prosecution and the defence sides in criminal proceedings: helping the prosecution to decide whether to proceed to trial and helping defence lawyers to advise a defendant how to plead. This paper reviews some numerical approaches to the goal of quantifying the relative weights of individual items of digital evidence and the plausibility of hypotheses based on that evidence. The potential advantages enabling the construction of cost-effective digital forensic triage schemas are also outlined.

Key points

• The absence of quantified results from digital forensic investigations, unlike those of conventional forensics, is highlighted.
• A number of approaches towards quantitative evaluation of the results of digital forensic investigations are reviewed.
• The significant potential benefits accruing from such approaches are discussed.

     

Automatic forensic face recognition from digital images.

Digital image evidence is now widely available from criminal investigations and surveillance operations, often captured by security and surveillance CCTV. This has resulted in a growing demand from law enforcement agencies for automatic person-recognition based on image data. In forensic science, a fundamental requirement for such automatic face recognition is to evaluate the weight that can justifiably be attached to this recognition evidence in a scientific framework. This paper describes a pilot study carried out by the Forensic Science Service (UK) which explores the use of digital facial images in forensic investigation. For the purpose of the experiment a specific software package was chosen (Image Metrics Optasia). The paper does not describe the techniques used by the software to reach its decision of probabilistic matches to facial images, but accepts the output of the software as though it were a 'black box'. In this way, the paper lays a foundation for how face recognition systems can be compared in a forensic framework. The aim of the paper is to explore how reliably and under what conditions digital facial images can be presented in evidence.     

The suitability of visual taphonomic methods for digital photographs: An experimental approach with pig carcasses in a tropical climate

In the context of increased scrutiny of the methods in forensic sciences, it is essential to ensure that the approaches used in forensic taphonomy to measure decomposition and estimate the postmortem interval are underpinned by robust evidence-based data. Digital photographs are an important source of documentation in forensic taphonomic investigations but the suitability of the current approaches for photographs, rather than real-time remains, is poorly studied which can undermine accurate forensic conclusions. The present study aimed to investigate the suitability of 2D colour digital photographs for evaluating decomposition of exposed human analogues (Sus scrofa domesticus) in a tropical savanna environment (Hawaii), using two published scoring methods; Megyesi et al., 2005 and Keough et al., 2017. It was found that there were significant differences between the real-time and photograph decomposition scores when the Megyesi et al. method was used. However, the Keough et al. method applied to photographs reflected real-time decomposition more closely and thus appears more suitable to evaluate pig decomposition from 2D photographs. The findings indicate that the type of scoring method used has a significant impact on the ability to accurately evaluate the decomposition of exposed pig carcasses from photographs. It was further identified that photographic taphonomic analysis can reach high inter-observer reproducibility. These novel findings are of significant importance for the forensic sciences as they highlight the potential for high quality photograph coverage to provide useful complementary information for the forensic taphonomic investigation. New recommendations to develop robust transparent approaches adapted to photographs in forensic taphonomy are suggested based on these findings.     

Performance Study of a Score‐based Likelihood Ratio System for Forensic Fingermark Comparison

In this article, the performance of a score‐based likelihood ratio (LR) system for comparisons of fingerprints with fingermarks is studied. The system is based on an automated fingerprint identification system (AFIS) comparison algorithm and focuses on fingerprint comparisons where the fingermarks contain 6–11 minutiae. The hypotheses under consideration are evaluated at the level of the person, not the finger. The LRs are presented with bootstrap intervals indicating the sampling uncertainty involved. Several aspects of the performance are measured: leave‐one‐out cross‐validation is applied, and rates of misleading evidence are studied in two ways. A simulation study is performed to study the coverage of the bootstrap intervals. The results indicate that the evidential strength for same source comparisons that do not meet the Dutch twelve‐point standard may be substantial. The methods used can be generalized to measure the performance of score‐based LR systems in other fields of forensic science.     

基于水印技术的司法数码影像防伪信息系统

数码影像作为证据使用存在一定的争议,针对此提出并建立了基于水印技术的司法数码影像防伪信息系统,该系统由加密防伪子系统、信息管理子系统和加密信息验证子系统组成,为确保司法数码影像的证据效力提供了一种技术方案。