On the database lookup problem of approximate matching

Investigating seized devices within digital forensics gets more and more difficult due to the increasing amount of data. Hence, a common procedure uses automated file identification which reduces the amount of data an investigator has to look at by hand. Besides identifying exact duplicates, which is mostly solved using cryptographic hash functions, it is also helpful to detect similar data by applying approximate matching.Let x denote the number of digests in a database, then the lookup for a single similarity digest has the complexity of O(x). In other words, the digest has to be compared against all digests in the database. In contrast, cryptographic hash values are stored within binary trees or hash tables and hence the lookup complexity of a single digest is O(log₂(x)) or O(1), respectively.In this paper we present and evaluate a concept to extend existing approximate matching algorithms, which reduces the lookup complexity from O(x) to O(1). Therefore, instead of using multiple small Bloom filters (which is the common procedure), we demonstrate that a single, huge Bloom filter has a far better performance. Our evaluation demonstrates that current approximate matching algorithms are too slow (e.g., over 21 min to compare 4457 digests of a common file corpus against each other) while the improved version solves this challenge within seconds. Studying the precision and recall rates shows that our approach works as reliably as the original implementations. We obtain this benefit by accuracy–the comparison is now a file-against-set comparison and thus it is not possible to see which file in the database is matched.     

A reference database of Windows artifacts for file-wiping tool execution analysis

Anti-forensic technology can play an effective role in protecting information, but it can make forensic investigations difficult. Specifically, file-wiping permanently erases evidence, making it challenging for investigators to determine whether a file ever existed and prolonging the investigation process. To address this issue, forensic researchers have studied anti-forensic techniques that detect file-wiping activities. Many previous studies have focused on the effects of file-wiping tools on $MFT, $LogFile, and $DATA, rather than on Windows artifacts. Additionally, previous studies that have examined Windows artifacts have considered different artifacts, making it difficult to study them in a comprehensive manner. To address this, we focused on analyzing traces in 13 Windows artifacts of 10 file-wiping tools' operations in the Windows operating system comprehensively. For our experiments, we installed each file-wiping tool on separate virtual machines and checked the traces that the tools left behind in each artifact. We then organized the results in a database format. Our analysis revealed that most of the tools left traces on other artifacts, except for JumpList, Open&SavePidlMRU, and lnk. There were also some cases where traces remained on the other three artifacts. Based on our research, forensic investigators can quickly identify whether a file-wiping tool has been used, and it can assist in decision-making for evidence collection and forensic triage.     

Introducing the Microsoft Vista event log file format

Several operating systems provide a central logging service which collects event messages from the kernel and applications, filters them and writes them into log files. Since more than a decade such a system service exists in Microsoft Windows NT. Its file format is well understood and supported by forensic software. Microsoft Vista introduces an event logging service which entirely got newly designed. This confronts forensic examiners and software authors with unfamiliar system behavior and a new, widely undocumented file format.This article describes the history of Windows system loggers, what has been changed over time and for what reason. It compares Vista log files in their native binary form and in a textual form. Based on the results, this paper for the first time publicly describes the key-elements of the new log file format and the proprietary binary encoding of XML. It discusses the problems that may arise during daily work. Finally it proposes a procedure for how to recover information from log fragments. During a criminal investigation this procedure was successfully applied to recover information from a corrupted event log.     

Forensic carving of network packets and associated data structures

Using validated carving techniques, we show that popular operating systems (e.g. Windows, Linux, and OSX) frequently have residual IP packets, Ethernet frames, and associated data structures present in system memory from long-terminated network traffic. Such information is useful for many forensic purposes including establishment of prior connection activity and services used; identification of other systems present on the system’s LAN or WLAN; geolocation of the host computer system; and cross-drive analysis. We show that network structures can also be recovered from memory that is persisted onto a mass storage medium during the course of system swapping or hibernation. We present our network carving techniques, algorithms and tools, and validate these against both purpose-built memory images and a readily available forensic corpora. These techniques are valuable to both forensics tasks, particularly in analyzing mobile devices, and to cyber-security objectives such as malware analysis.     

The Linux FAT32 allocator and file creation order reconstruction

The allocation algorithm of the Linux FAT32 file system driver positions files on disk in such a way that their relative positions reveal information on the order in which these files have been created. This provides an opportunity to enrich information from (carved) file fragments with time information, even when such file fragments lack the file system metadata in which time-related information is usually to be found.Through source code analysis and experiments the behaviour of the Linux FAT allocator is examined. How an understanding of this allocator can be applied in practice is demonstrated with a case study involving a TomTom GPS car navigation device. In this case, time information played a crucial role. Large amounts of location records could be carved from this device's flash storage, yielding insight into the locations the device has visited—yet the carved records themselves offered no information on when the device had been at the locations. Still, bounds on the records' time of creation could be inferred when making use of filesystem timestamps related to neighbouring on-disk positions.Finally, we perform experiments which contrast the Linux behaviour with that of Windows 7. We show that the latter differs subtly, breaking the strong relation between creation order and position.     

Reverse engineering of ReFS

File system forensics is an important part of Digital Forensics. Investigators of storage media have traditionally focused on the most commonly used file systems such as NTFS, FAT, ExFAT, Ext2-4, HFS+, APFS, etc. NTFS is the current file system used by Windows for the system volume, but this may change in the future. In this paper we will show the structure of the Resilient File System (ReFS), which has been available since Windows Server 2012 and Windows 8. The main purpose of ReFS is to be used on storage spaces in server systems, but it can also be used in Windows 8 or newer. Although ReFS is not the current standard file system in Windows, while users have the option to create ReFS file systems, digital forensic investigators need to investigate the file systems identified on a seized media. Further, we will focus on remnants of non-allocated metadata structures or attributes. This may allow metadata carving, which means searching for specific attributes that are not allocated. Attributes found can then be used for file recovery. ReFS uses superblocks and checkpoints in addition to a VBR, which is different from other Windows file systems. If the partition is reformatted with another file system, the backup superblocks can be used for partition recovery. Further, it is possible to search for checkpoints in order to recover both metadata and content.Another concept not seen for Windows file systems, is the sharing of blocks. When a file is copied, both the original and the new file will share the same content blocks. If the user changes the copy, new data runs will be created for the modified content, but unchanged blocks remain shared. This may impact file carving, because part of the blocks previously used by a deleted file might still be in use by another file. The large default cluster size, 64 KiB, in ReFS v1.2 is an advantage when carving for deleted files, since most deleted files are less than 64 KiB and therefore only use a single cluster. For ReFS v3.2 this advantage has decreased because the standard cluster size is 4 KiB.Preliminary support for ReFS v1.2 has been available in EnCase 7 and 8, but the implementation has not been documented or peer-reviewed. The same is true for Paragon Software, which recently added ReFS support to their forensic product. Our work documents how ReFS v1.2 and ReFS v3.2 are structured at an abstraction level that allows digital forensic investigation of this new file system. At the time of writing this paper, Paragon Software is the only digital forensic tool that supports ReFS v3.x.It is the most recent version of the ReFS file system that is most relevant for digital forensics, as Windows automatically updates the file system to the latest version on mount. This is why we have included information about ReFS v3.2. However, it is possible to change a registry value to avoid updating. The latest ReFS version observed is 3.4, but the information presented about 3.2 is still valid. In any criminal case, the investigator needs to investigate the file system version found.     

The windows IconCache.db: A resource for forensic artifacts from USB connectable devices

This paper investigates the evidential potential of the IconCache database file when tracking activity from USB connectable devices on Windows systems. It focuses on the artifacts which are created and retained on a Windows host when executable files are either present on or run from a USB connectable device. Artifacts left in the IconCache database as a result of running executables from a DVD drive or the host itself, are also examined.It is shown that the IconCache.db stores numerous artifacts of investigative interest. These are created on system boot and added to, both when using host-based executables and when installing or using executables from other media. Executables present on USB devices, whether invoked or not, will create artifacts in the IconCache.db. file. Findings should therefore be interpreted carefully and corroborated against other evidence.     

File fragment encoding classification—An empirical approach

Over the past decade, a substantial effort has been put into developing methods to classify file fragments. Throughout, it has been an article of faith that data fragments, such as disk blocks, can be attributed to different file types. This work is an attempt to critically examine the underlying assumptions and compare them to empirically collected data. Specifically, we focus most of our effort on surveying several common compressed data formats, and show that the simplistic conceptual framework of prior work is at odds with the realities of actual data. We introduce a new tool, zsniff, which allows us to analyze deflate-encoded data, and we use it to perform an empirical survey of deflate-coded text, images, and executables. The results offer a conceptually new type of classification capabilities that cannot be achieved by other means.     

Bin-Carver: Automatic recovery of binary executable files

File carving is the process of reassembling files from disk fragments based on the file content in the absence of file system metadata. By leveraging both file header and footer pairs, traditional file carving mainly focuses on document and image files such as PDF and JPEG. With the vast amount of malware code appearing in the wild daily, recovery of binary executable files becomes an important problem, especially for the case in which malware deletes itself after compromising a computer. However, unlike image files that usually have both a header and footer pair, executable files only have header information, which makes the carving much harder. In this paper, we present Bin-Carver, a first-of-its-kind system to automatically recover executable files with deleted or corrupted metadata. The key idea is to explore the road map information defined in executable file headers and the explicit control flow paths present in the binary code. Our experiment with thousands of binary code files has shown our Bin-Carver to be incredibly accurate, with an identification rate of 96.3% and recovery rate of 93.1% on average when handling file systems ranging from pristine to chaotic and highly fragmented.     

A comparison of forensic evidence recovery techniques for a windows mobile smart phone

Acquisition, decoding and presentation of information from mobile devices is complex and challenging. Device memory is usually integrated into the device, making isolation prior to recovery difficult. In addition, manufacturers have adopted a variety of file systems and formats complicating decoding and presentation.A variety of tools and methods have been developed (both commercially and in the open source community) to assist mobile forensics investigators. However, it is unclear to what extent these tools can present a complete view of the information held on a mobile device, or the extent the results produced by different tools are consistent.This paper investigates what information held on a Windows Mobile smart phone can be recovered using several different approaches to acquisition and decoding. The paper demonstrates that no one technique recovers all information of potential forensic interest from a Windows Mobile device; and that in some cases the information recovered is conflicting.     

Comments on the Linux FAT32 allocator and file creation order reconstruction [Digit Investig 11(4), 224–233]

Minnaard proposed a novel method that constructs a creation time bound of files recovered without time information. The method exploits a relationship between the creation order of files and their locations on a storage device managed with the Linux FAT32 file system. This creation order reconstruction method is valid only in non-wraparound situations, where the file creation time in a former position is earlier than that in a latter position. In this article, we show that if the Linux FAT32 file allocator traverses the storage space more than once, the creation time of a recovered file is possibly earlier than that of a former file and possibly later than that of a latter file on the Linux FAT32 file system. Also it is analytically verified that there are at most n candidates for the creation time bound of each recovered file where n is the number of traversals by the file allocator. Our analysis is evaluated by examining file allocation patterns of two commercial in-car dashboard cameras.     

Windows系统存储区DPAPI的解密技术研究

目的 在电子数据取证过程中,数据的加解密经常是取证人员关注的重点.数据保护接口(DPAPI)作为Windows系统提供的数据保护接口被广泛使用,目前主要用于保护加密的数据.其特性主要表现在加密和解密必须在同一台计算机上操作,密钥的生成、使用和管理由Windows系统内部完成,如果更换计算机则无法解开DPAPI加密数据....     

Forensic integrity verification of video recordings based on MTS files

Digital video is used in criminal trials as evidence with legal responsibility because video content vividly depicts events occurring at a crime scene. However, using sophisticated video editing software, assailants can easily manipulate visible clues for their own benefit. Therefore, the integrity of digital video files acquired or submitted as evidence must be ensured. Forensic analysis of digital video is key to ensuring the integrity of links with individual cameras. In this study, we analyzed whether it is possible to ensure the integrity of MTS video files. Herein, we propose a method to verify the integrity of MTS files encoded by advanced video coding high definition (AVCHD), which is frequently used for video recording. To verify MTS file integrity, we propose five features. Codec information, picture timing, and camera manufacture/model are modified AVI and MP4-like format video verification features. Group of pictures and Universally Unique Identifier patterns were specifically developed for MTS streams. We analyzed the features of 44 standard files recorded using all recording options of seven cameras. We checked whether integrity can be validated on unmanipulated videos recorded in various environments. In addition, we considered whether manipulated MTS files edited in video editing software could be validated. Experimental results show that all unmanipulated and manipulated MTS files with known recording devices were discriminated only when all five features were checked. These results show that the proposed method verifies the integrity of MTS files, strengthening the validity of MTS file-based evidence in trials.     

Dropout in a Treatment Program for Self-referring Wife Abusing Men

Demographic factors and levels of psychological and physical aggression were assessed as predictors of dropout for those who participated in a group treatment program for maritally discordant couples reporting husband to wife physical aggression. In addition, follow-up assessments with dropouts were conducted to obtain clients' reasons for termination. Demographic variables and levels of physical aggression did not predict dropout; however, higher levels of psychological aggression did. The most frequently cited reasons for dropout by participants were treatment-related issues. Specifically, clients reported that the group format did not enable them to address their individual couple issues. Regarding future treatment planning, our findings indicate that in cases where men are severely psychologically abusive, individual sessions may be warranted that precede or are in conjunction with couple sessions. Also, it is as important to target women's psychological aggression as it is to target men's psychological aggression. Finally, our findings indicate that treatment programs should utilize some mechanism that allows for more personalized attention to each couple, such as an individual couples format and/or an individual supplement to the group format.     

IoT forensics: Exploiting unexplored log records from the HIKVISION file system

CCTV surveillance systems are IoT products that can be found almost everywhere. Their digital forensic analysis often plays a key role in solving crimes. However, it is common for these devices to use proprietary file systems, which frequently hinders a complete examination. HIKVISION is a well-known manufacturer of such devices that typically ships its products with its proprietary file system. The HIKVISION file system has been analyzed before but that research has focused on the recovery of video footage. In this paper, the HIKVISION file system is being revisited regarding the log records it stores. More specifically, these log records are thoroughly examined to uncover both their structure and meaning. These unexplored pieces of evidence remain unexploited by major commercial forensic software, yet they can contain critical information for an investigation. To further assist digital forensic examiners with their analysis, a Python utility, namely the Hikvision Log Analyzer, was developed as part of this study that can automate part of the process.     

Forensic analysis of video file formats

Video file format standards define only a limited number of mandatory features and leave room for interpretation. Design decisions of device manufacturers and software vendors are thus a fruitful resource for forensic video authentication. This paper explores AVI and MP4-like video streams of mobile phones and digital cameras in detail. We use customized parsers to extract all file format structures of videos from overall 19 digital camera models, 14 mobile phone models, and 6 video editing toolboxes. We report considerable differences in the choice of container formats, audio and video compression algorithms, acquisition parameters, and internal file structure. In combination, such characteristics can help to authenticate digital video files in forensic settings by distinguishing between original and post-processed videos, verifying the purported source of a file, or identifying the true acquisition device model or the processing software used for video processing.     

Forensic artefacts left by Windows Live Messenger 8.0

Windows Live Messenger – commonly referred by MSN Messenger – is the most used instant messaging client worldwide, and is mostly used on Microsoft Windows XP.Previous examination into MSN Messenger concludes that few traces reside on the hard disk after MSN usage [Dickson M. An examination into MSN Messenger 7.5 contact identification. Digit Investig 2006;3]. In this article the opposite is concluded based on user settings, contact files and log files. With the use of file signatures and known file structures it is possible to recover useful information when deleted. Programs such as Forensic Box can help to analyse artefacts which are left behind after the use of Windows Live Messenger.     

Predicting the types of file fragments

A problem that arises in computer forensics is to determine the type of a file fragment. An extension to the file name indicating the type is stored in the disk directory, but when a file is deleted, the entry for the file in the directory may be overwritten. This problem is easily solved when the fragment includes the initial header, which contains explicit type-identifying information, but it is more difficult to determine the type of a fragment from the middle of a file.We investigate two algorithms for predicting the type of a fragment: one based on Fisher's linear discriminant and the other based on longest common subsequences of the fragment with various sets of test files. We test the ability of the algorithms to predict a variety of common file types. Algorithms of this kind may be useful in designing the next generation of file-carvers – programs that reconstruct files when directory information is lost or deleted. These methods may also be useful in designing virus scanners, firewalls and search engines to find files that are similar to a given file.     

The Normalised Compression Distance as a file fragment classifier

We have applied the generalised and universal distance measure NCD—Normalised Compression Distance—to the problem of determining the type of file fragments. To enable later comparison of the results, the algorithm was applied to fragments of a publicly available corpus of files. The NCD algorithm in conjunction with the k-nearest-neighbour (k ranging from one to ten) as the classification algorithm was applied to a random selection of circa 3000 512-byte file fragments from 28 different file types. This procedure was then repeated ten times. While the overall accuracy of the n-valued classification only improved the prior probability from approximately 3.5% to circa 32–36%, the classifier reached accuracies of circa 70% for the most successful file types.A prototype of a file fragment classifier was then developed and evaluated on new set of data (from the same corpus). Some circa 3000 fragments were selected at random and the experiment repeated five times. This prototype classifier remained successful at classifying individual file types with accuracies ranging from only slightly lower than 70% for the best class, down to similar accuracies as in the prior experiment.     

Assessing measurement invariance of PCL-R assessments from file reviews of North American and German offenders

Cross-cultural research on psychopathy necessitates assessment methods that are generalizable across linguistic and cultural differences. Multiple-group confirmatory factor analysis was used to compare the factorial structure of Psychopathy Checklist-Revised (PCL-R) assessments obtained from file reviews of North-American (N = 2622) and German (N = 443) male offenders. The analyses indicated that the 18 item, 4-factor model of the PCL-R obtained with the standard PCL-R protocol (interview and file review) also holds for file review data. On a factor-by-factor level, the data are commensurate with strong factorial invariance of factor loadings and item thresholds for the Interpersonal and Lifestyle factors, and with likely metric invariance for the Affective factor. The Antisocial factor showed structural differences between the two samples. The results imply that cultural or environmental factors more strongly influence the judgment and/or expression of antisociality. Based on the results, cross-cultural comparisons between North-American and German offenders in terms of PCL-R psychopathy should be limited to the Interpersonal and Lifestyle factors. Further research using data obtained through the standard protocol (i.e., interview plus file information) is encouraged.