FRASH: A framework to test algorithms of similarity hashing

Automated input identification is a very challenging, but also important task. Within computer forensics this reduces the amount of data an investigator has to look at by hand. Besides identifying exact duplicates, which is mostly solved using cryptographic hash functions, it is necessary to cope with similar inputs (e.g., different versions of a file), embedded objects (e.g., a JPG within a Word document), and fragments (e.g., network packets), too. Over the recent years a couple of different similarity hashing algorithms were published. However, due to the absence of a definition and a test framework, it is hardly possible to evaluate and compare these approaches to establish them in the community.The paper at hand aims at providing an assessment methodology and a sample implementation called FRASH: a framework to test algorithms of similarity hashing. First, we describe common use cases of a similarity hashing algorithm to motivate our two test classes efficiency and sensitivity & robustness. Next, our open and freely available framework is briefly described. Finally, we apply FRASH to the well-known similarity hashing approaches ssdeep and sdhash to show their strengths and weaknesses.     

On the database lookup problem of approximate matching

Investigating seized devices within digital forensics gets more and more difficult due to the increasing amount of data. Hence, a common procedure uses automated file identification which reduces the amount of data an investigator has to look at by hand. Besides identifying exact duplicates, which is mostly solved using cryptographic hash functions, it is also helpful to detect similar data by applying approximate matching.Let x denote the number of digests in a database, then the lookup for a single similarity digest has the complexity of O(x). In other words, the digest has to be compared against all digests in the database. In contrast, cryptographic hash values are stored within binary trees or hash tables and hence the lookup complexity of a single digest is O(log₂(x)) or O(1), respectively.In this paper we present and evaluate a concept to extend existing approximate matching algorithms, which reduces the lookup complexity from O(x) to O(1). Therefore, instead of using multiple small Bloom filters (which is the common procedure), we demonstrate that a single, huge Bloom filter has a far better performance. Our evaluation demonstrates that current approximate matching algorithms are too slow (e.g., over 21 min to compare 4457 digests of a common file corpus against each other) while the improved version solves this challenge within seconds. Studying the precision and recall rates shows that our approach works as reliably as the original implementations. We obtain this benefit by accuracy–the comparison is now a file-against-set comparison and thus it is not possible to see which file in the database is matched.     

An evaluation of forensic similarity hashes

The fast growth of the average size of digital forensic targets demands new automated means to quickly, accurately and reliably correlate digital artifacts. Such tools need to offer more flexibility than the routine known-file filtering based on crypto hashes. Currently, there are two tools for which NIST has produced reference hash sets–ssdeep and sdhash. The former provides a fixed-sized fuzzy hash based on random polynomials, whereas the latter produces a variable-length similarity digest based on statistically-identified features packed into Bloom filters.This study provides a baseline evaluation of the capabilities of these tools both in a controlled environment and on real-world data. The results show that the similarity digest approach significantly outperforms in terms of recall and precision in all tested scenarios and demonstrates robust and scalable behavior.     

Automated evaluation of approximate matching algorithms on real data

Bytewise approximate matching is a relatively new area within digital forensics, but its importance is growing quickly as practitioners are looking for fast methods to screen and analyze the increasing amounts of data in forensic investigations. The essential idea is to complement the use of cryptographic hash functions to detect data objects with bytewise identical representation with the capability to find objects with bytewise similar representations.Unlike cryptographic hash functions, which have been studied and tested for a long time, approximate matching ones are still in their early development stages and evaluation methodology is still evolving. Broadly, prior approaches have used either a human in the loop to manually evaluate the goodness of similarity matches on real world data, or controlled (pseudo-random) data to perform automated evaluation.This work's contribution is to introduce automated approximate matching evaluation on real data by relating approximate matching results to the longest common substring (LCS). Specifically, we introduce a computationally efficient LCS approximation and use it to obtain ground truth on the t5 set. Using the results, we evaluate three existing approximate matching schemes relative to LCS and analyze their performance.     

Forensic discovery auditing of digital evidence containers

Current digital forensics methods capture, preserve, and analyze digital evidence in general-purpose electronic containers (typically, plain files) with no dedicated support to help establish that the evidence has been properly handled. Auditing of a digital investigation, from identification and seizure of evidence through duplication and investigation is, essentially, ad hoc, recorded in separate log files or in an investigator's case notebook. Auditing performed in this fashion is bound to be incomplete, because different tools provide widely disparate amounts of auditing information – including none at all – and there is ample room for human error. The latter is a particularly pressing concern given the fast growth of the size of forensic targets.Recently, there has been a serious community effort to develop an open standard for specialized digital evidence containers (DECs). A DEC differs from a general purpose container in that, in addition to the actual evidence, it bundles arbitrary metadata associated with it, such as logs and notes, and provides the basic means to detect evidence-tampering through digital signatures. Current approaches consist of defining a container format and providing a specialized library that can be used to manipulate it. While a big step in the right direction, this approach has some non-trivial shortcomings – it requires the retooling of existing forensic software and, thereby, limits the number of tools available to the investigator. More importantly, however, it does not provide a complete solution since it only records snapshots of the state of the DEC without being able to provide a trusted log of all data operations actually performed on the evidence. Without a trusted log the question of whether a tool worked exactly as advertised cannot be answered with certainty, which opens the door to challenges (both legitimate and frivolous) of the results.In this paper, we propose a complementary mechanism, called the Forensic Discovery Auditing Module (FDAM), aimed at closing this loophole in the discovery process. FDAM can be thought of as a ‘clean-room’ environment for the manipulation of digital evidence, where evidence from containers is placed for controlled manipulation. It functions as an operating system component, which monitors and logs all access to the evidence and enforces policy restrictions. This allows the immediate, safe, and verifiable use of any tool deemed necessary by the examiner. In addition, the module can provide transparent support for multiple DEC formats, thereby greatly simplifying the adoption of open standards.     

Evaluating detection error trade-offs for bytewise approximate matching algorithms

Bytewise approximate matching is a relatively new area within digital forensics, but its importance is growing quickly as practitioners are looking for fast methods to analyze the increasing amounts of data in forensic investigations. The essential idea is to complement the use of cryptographic hash functions to detect data objects with bytewise identical representation with the capability to find objects with bytewise similar representations.Unlike cryptographic hash functions, which have been studied and tested for a long time, approximate matching ones are still in their early development stages, and have been evaluated in a somewhat ad-hoc manner. Recently, the FRASH testing framework has been proposed as a vehicle for developing a set of standardized tests for approximate matching algorithms; the aim is to provide a useful guide for understanding and comparing the absolute and relative performance of different algorithms.The contribution of this work is twofold: a) expand FRASH with automated tests for quantifying approximate matching algorithm behavior with respect to precision and recall; and b) present a case study of two algorithms already in use–sdhash and ssdeep.     

Multiplex-direct PCR assay for foodborne pathogen identification: An application in forensic investigation

Foodborne pathogens present serious concerns to human health and can even lead to fatalities. Microbial forensic science thus plays an important role in consumer protection, food security, and even in litigation. The gold standard for pathogen identification – bacterial culture – is costly and time-consuming. A cheaper and quicker alternative will benefit both forensic science and medical diagnosis. In this study, we developed and validated a molecular-based method termed ‘multiplex-direct PCR assay’ to simultaneously detect three common foodborne pathogens – Escherichia coli O157:H7, Campylobacter jejuni, and Listeria monocytogenes. Three previously reported species-specific primer pairs were modified and used to directly amplify samples without DNA extraction. The assay was also validated for its specificity, sensitivity, and applied to test several samples obtained from a local market and clinical samples. The results showed the expected PCR fragments of approximately 490, 343, and 209 bp for E. coli O157:H7, C. jejuni, and L. monocytogenes, respectively. The assay was specific to the targeted pathogens and was sufficiently sensitive and robust to effectively analyze market samples. The whole process took less than 1 h to complete indicating that the assay is suitable for reliable, rapid, and inexpensive identification of these three foodborne pathogens, which could be useful in microbial forensic investigation.     

Population genetic data of 38 autosomal InDels in San Basilio de Palenque,the first free town in America

In order to increase the information about Indels, we report allele frequencies and statistical parameters of forensic efficiency obtained typing a sample of 114 unrelated healthy individuals living in San Basilio de Palenque – Colombia using a panel of 38 autosomal InDels. No significant deviations from Hardy–Weinberg expectations were found except in the marker rs10629077 (p = 0.0002). The present database will be useful for forensic and paternity purposes for the region studied. Moreover, these additional markers can help forensic laboratories to solve parentage testing as well as to improve the analysis of degraded DNA samples.     

The Decisional Juridical Discourse of the Appellate Body of the WTO: Among Treaties and Dictionaries as Referents

This present paper is devoted to the analysis of the decisional juridical discourses of the Appellate Body of the World Trade Organization. For this end, we decided to develop the research around two poles which shall be approached in an interweaving manner: the first concerns an examination of the methods of interpretation adopted by the Appellate Body and the second, which is a consequence of the former, devotes itself to the problem derived from the interpretation of authentic international treaties in more than one language. In the light of these two approaches we can verify that the interpretation of the Appellate Body is highly influenced by the search for the purpose of the text and the construction of the juridical discourse in question is made with reference to the linguistic system analyzed as a dictionary. It was established, that the Appellate Body carries out a dictionary interpretation with a tendency, even incipient, to consider the linguistic versions of the World Trade Organization Agreements. Finally, the task is structured having as a backdrop two interdependent concepts which should not be neglected in an analysis of international juridical discourses. They are the following: ?juridical culture’ and ?language’. Both will be dealt with from a semiotic perspective since the central element of our study – and of the intersection between these two concepts – is the linguistic sign.     

Association between copy number variations in the OCA2-HERC2 locus and human eye colour

Human eye colour variation is strongly associated with single nucleotide polymorphisms (SNPs) in the OCA2-HERC2 locus, especially rs12913832 that is found in an enhancer element of OCA2. In a previous study we found that 43 out of 166 individuals in a Norwegian population with the brown eye colour genotype HERC2 rs12913832:AA or AG, did not have the expected brown eye colour. To investigate if duplications or deletions in the OCA2-HERC2 locus could explain the blue eye colour in these individuals, we analysed massively parallel sequencing (MPS) data for copy number variations (CNVs) in the OCA2-HERC2 region. The ∼500 kb long OCA2-HERC2 locus was sequenced in 94 individuals with the rs12913832:AG and AA genotypes. Of these, 43 were observed to have blue eye colour and 51 were observed to have brown eye colour. CNVs were analysed using R and the R-package panelcn.MOPS - CNV detection tool for targeted NGS panel data. In rs12913832:AG individuals, CNVs in 32 regions were significantly associated with blue eye colour (Benjamini-Hochberg adjusted p-value ≤ 0.05). In rs12913832:AA individuals, CNVs in 14 regions were associated with blue eye colour using raw p-values (p ≤ 0.05). The functional effects of these CNVs on OCA2 expression are yet to be investigated. However, this study suggests that CNVs in the OCA2-HERC2 locus might explain why some of the rs12913832:AG and AA individuals have unexpectedly blue eyes.     

Hear no evil, see no evil, speak no evil: the unsightly Milosevic case

To ignore evil is to cause it to cease to exist, thought the ancients, and so, perhaps, think those who accuse former leaders of now dismembered countries, no longer in existence, of war crimes, and who would prevent those they accuse of raising the aggression which was committed against their country. Can the evil of aggression be willed out of existence if it goes unmentioned, and if international ad hoc bodies do not consider it a crime within their jurisdiction? And if the defendant is gagged, if judgments permit him to be removed from the courtroom altogether, will we be free from having to see and hear the evil he persistently identifies, and for which he points out there will be no justice? The Milosevic trial has been underreported to the point where “speaking evil” – that is, expressing criticism of the persistent procedural irregularities that have plagued the proceedings, and indeed the outright erosion of fair trial rights (heralded as “progress” in some quarters) – has become a demanding exercise. It is one we attempt here.     

Quadruplex real-time PCR for forensic DNA quantitation

Forensic DNA quantitation is an important initial step preceding PCR amplification of the STR loci even though information concerning the quality of the DNA is not revealed. A quadruplex real-time PCR (qPCR) assay was developed to quantify four DNA targets: (1) the human RB1 gene in nuclear DNA, (2) the DAZ gene present on the human Y chromosome, (3) the ATPase8 gene present in human mitochondrial DNA and (4) an artificial internal positive control to reveal possible PCR inhibition. Primers labeled with four different fluorophores are used together with a single quencher using the antiprimer quenching-based qPCR method in one reaction, in which the resultant amplicons are less than 127 bp in size. Sensitivity was shown to be less than ten copies for all four targets in the absence of amplification inhibition. The amplification remained sensitive in the presence of an excess of non-human DNA.     

Clustering digital forensic string search output

This research comparatively evaluates four competing clustering algorithms for thematically clustering digital forensic text string search output. It does so in a more realistic context, respecting data size and heterogeneity, than has been researched in the past. In this study, we used physical-level text string search output, consisting of over two million search hits found in nearly 50,000 allocated files and unallocated blocks. Holding the data set constant, we comparatively evaluated k-Means, Kohonen SOM, Latent Dirichlet Allocation (LDA) followed by k-Means, and LDA followed by SOM. This enables true cross-algorithm evaluation, whereas past studies evaluated singular algorithms using unique, non-reproducible datasets. Our research shows an LDA + k-Means using a linear, centroid-based user navigation procedure produces optimal results. The winning approach increased information retrieval effectiveness, from the baseline random walk absolute precision rate of 0.04, to an average precision rate of 0.67. We also explored a variety of algorithms for user navigation of search hit results, finding that the performance of k-means clustering can be greatly improved with a non-linear, non-centroid-based cluster and document navigation procedure, which has potential implications for digital forensic tools and use thereof, particularly given the popularity and speed of k-means clustering.     

Validation of image stream hashing: A forensic method for content verification

How do forensic examiners know if they have altered an image stream when converting a digital image from one codec or file container to another for analysis? Forensic standards and best practices recommend avoiding alteration or degradation of multimedia data during transcoding. An image stream hashing method was recently introduced to the forensic science community to answer the question above. This paper offers an initial validation study of image stream hashing method that may answer the question above. The first half of the study's experiments tested the image stream hashing method to measure fitness for use in forensic science while identifying errors and limitations. The study's second phase analyzed the systematic errors detected in initial tests to discover error causation. Causation analysis identified four method limitations subsequently used to develop proposed standard controls of method operations. The final study phase repeated the initial experiments used in the first phase while implementing the proposed standard controls of method operations. Initial test results indicated the method had significant error rates, limiting the effectiveness of the method to only three of the five file types used in the study. The final testing phase revealed that implementing proposed standard controls of method operations reduced the potential systematic errors to a negligible level when using the image stream hashing method for content verification. The validation study concluded that examiners could use the image stream hashing method for forensic science only by implementing error mitigation techniques that utilize the proposed standard controls of method operations.     

Forensic Considerations for Preprocessing Effects on Clinical MDCT Scans

Manipulation of digital photographs destined for medico‐legal inquiry must be thoroughly documented and presented with explanation of any manipulations. Unlike digital photography, computed tomography (CT) data must pass through an additional step before viewing. Reconstruction of raw data involves reconstruction algorithms to preprocess the raw information into display data. Preprocessing of raw data, although it occurs at the source, alters the images and must be accounted for in the same way as postprocessing. Repeated CT scans of a gunshot wound phantom were made using the Toshiba Aquilion 64‐slice multidetector CT scanner. The appearance of fragments, high‐density inclusion artifacts, and soft tissue were assessed. Preprocessing with different algorithms results in substantial differences in image output. It is important to appreciate that preprocessing affects the image, that it does so differently in the presence of high‐density inclusions, and that preprocessing algorithms and scanning parameters may be used to overcome the resulting artifacts.     

Does contextual information bias bitemark comparisons?

A growing body of research suggests that the interpretation of fingerprint evidence is open to contextual bias. While there has been suggestion in the literature that the same might apply to bitemarks – a form of identification evidence in which a degree of contextual information during the comparison phase is generally unavoidable – there have so far been no empirical studies to test this assertion. We explored dental and non-dental students' ability to state whether two bitemarks matched, while manipulating task ambiguity and the presence and emotional intensity of additional contextual information. Provision of the contextual information influenced participants' decisions on the ambiguous bitemarks. Interestingly, when participants were presented with highly emotional images and subliminally primed with the words ‘same’ and ‘guilty’, they made fewer matches relative to our control condition. Dental experience also played a role in decision-making, with dental students making more matches as the experiment progressed, regardless of context or task ambiguity. We discuss ways that this exploratory research can be extended in future studies.     

Predicting the types of file fragments

A problem that arises in computer forensics is to determine the type of a file fragment. An extension to the file name indicating the type is stored in the disk directory, but when a file is deleted, the entry for the file in the directory may be overwritten. This problem is easily solved when the fragment includes the initial header, which contains explicit type-identifying information, but it is more difficult to determine the type of a fragment from the middle of a file.We investigate two algorithms for predicting the type of a fragment: one based on Fisher's linear discriminant and the other based on longest common subsequences of the fragment with various sets of test files. We test the ability of the algorithms to predict a variety of common file types. Algorithms of this kind may be useful in designing the next generation of file-carvers – programs that reconstruct files when directory information is lost or deleted. These methods may also be useful in designing virus scanners, firewalls and search engines to find files that are similar to a given file.     

Lambe v Finch (1626): An Early Seventeenth-Century Expectant Heir Chancery Suit in Context

ABSTRACT

The suit of Lambe v Finch (1626), at first glance, appears to offer evidence that the court of chancery’s jurisdiction to relieve expectant heirs from the consequences of their improvident bargains had at this time not yet developed to the point it was to reach in the latter part of the seventeenth century. However, if a contextual case study approach is taken, the significance of this particular suit to the development of the jurisdiction changes. By going beyond the information contained in the enrolled decree, a clearer – and qualitatively different – picture emerges; one which offers a more nuanced understanding of the jurisdiction to relieve expectant heirs, and sheds light on the involvement of one individual with the court of chancery in the early seventeenth century.     

Organizational factors related to technology transfer and innovation

This paper highlights those organizational factors which the research literature indicates as having an impact upon an organization's likelihood to exhibit innovative behavior. Measures of technical progressiveness, characteristics of firms receptive to outside information, the effect of organizational size, internal structure and procedures, and other factors are investigated, as are policies which encourage innovation. It is concluded that most research offers little of value to one desiring to make agiven organization more innovative (such as through effective internal communication of technical information), but is of value to an outsider attempting to identifywhich organizations are likely to be receptive to new technology.     

Rethinking the one-stop-shop mechanism: Legal certainty and legitimate expectation

This paper aims to contribute to the discussion concerning the one-stop-shop mechanism proposed in the General Data Protection Regulation (hereinafter “GDPR”). The choice of regulation as the instrument to legislate on data protection is already an unmistakable indication that unification and simplification (together with respect of data subjects' interests) shall be the guide for every legal discussion on the matter. The one-stop-shop mechanism (hereinafter “OSS”) clearly reflects the unification and simplification which the reform aims for. We believe that OSS is logically connected with the idea of one Data Protection Authority (hereinafter “DPA”) with an exclusive jurisdiction and that this can only mean that, given one controller, no other DPA can be a competent authority.² In other words, OSS implies a single and comprehensive competent authority of a given controller. In our analysis we argue that such architecture: a) works well with the “consistency mechanism”; b) provides guarantees to data subjects for a clear allocation of powers (legal certainty); and c) is not at odds with the complaint lodging procedure. Our position on fundamental questions is as follows. What is the perimeter of competence of the DPA in charge? We believe that it should have enforcement power on every issue of the controller, including issuing the fines. How to reconcile such dominant role of one DPA with the principle of co-operation among DPAs? We do not consider co-operation at odds with the rule that decisions are taken by just one single authority. Finally, we share some suggestions on how to make the jurisdiction allocation mechanism (the main establishment criterion) more straightforward.