In the eye of the beholder? An examination of the inter-rater reliability of the LSI-R and YLS/CMI in a correctional agency

PurposeTo examine the inter-rater reliability of two risk assessment tools: The Level of Service Inventory-Revised (LSI-R) and the Youth Level of Service/Case Management Inventory (YLS/CMI).MethodsTwo identical experiments are reported. For both studies, a random sample of 10 offenders were interviewed and videotaped with each tool (totaling 20 offenders). The tapes were then shown to a random selection of 20 raters (for a total of 40 raters) employed at a state agency. The fully-crossed design allowed each of the raters to rate the each of the cases, resulting in 200 total risk score observations for each tool. Inter-rater reliability analyses were then conducted.ResultsThe LSI-R demonstrated adequate to fair reliability, with certain domains showing lower reliability. Overall, the LSI-R had an ICC of .65. The YLS/CMI demonstrated higher reliability (ICC of .78). In addition, for the LSI-R study, comparisons were made between staff raters who work in a facility versus those in the community (e.g., probation officers). For the YLS/CMI study, comparisons were made between incarcerated offenders versus probationers. Neither comparison yielded consistent differences.ConclusionsThe YLS/CMI is generally reliable. The LSI-R showed less reliability. However, each study showed certain domains with less than ideal reliability.     

How well do dynamic needs predict recidivism? Implications for risk assessment and risk reduction

Purpose

The current study assessed the predictive validity of nine dynamic risk factors in two samples of justice-involved individuals (n = 24,972) to identify promising targets for correctional programming. The study also tested the incremental predictive validity of dynamic risk relative to static indicators of recidivism risk (i.e., criminal history, age and sex).

Methods

The study relied on bivariate correlations and stepwise multivariate logistic regression analyses to test the predictive and incremental validity of dynamic risk constructs measured by the Level of Service Inventory-Revised (LSI-R).

Results

Study findings revealed that while several dynamic risk constructs were significantly related to recidivism (antisocial attitudes, antisocial peers, education/employment, and substance abuse) over a 36-month follow-up period, the inclusion of dynamic risk items on the LSI-R did little to improve the overall predictive validity of the instrument. Across both study samples, static criminal history risk emerged as the most robust predictor of recidivism.

Conclusions

The advantages of third and fourth generation risk-need assessment tools are noted for classification purposes, but the study findings imply that risk prediction is better served by static risk factors. Implications for risk assessment and offender case management are discussed.     

Using the level of service inventory-revised to improve assessment and treatment in drug court

More than 2,000 drug courts in the United States provide supervision and substance-abuse treatment to thousands of offenders. Yet the treatment continuum from assessment to aftercare is underexplored. The effectiveness of the Level of Service Inventory-Revised (LSI-R) as a risk assessment tool is well established. However, fewer studies have considered its use in guiding treatment strategies. In using the LSI-R, the drug court program relied on the structured interview protocol (not the risk classification scores) to identify criminogenic needs that then helped determine placement in a high- or low-needs treatment track. To evaluate the effectiveness of these treatment placement decisions, this research used the LSI-R scores to examine individual and group differences (N = 182). Significant and substantive differences at the individual and group levels were found thus providing empirical support for using the LSI-R as a link between assessment and treatment. Implications for developing standards and practice protocols for drug courts are discussed.     

Ethnicity, gender, and the Level of Service Inventory-Revised

Offender assessment and classification is becoming an increasingly important part of correctional supervision and intervention. Over the last several decades, several different methods and “generations” of offender classification have emerged. Of most value appear to be third-generation, actuarial, dynamic risk/need assessments. With any new correctional strategy, however, there is a need to investigate the use of these risk/need assessments on offender subgroups in order to inform issues, such as reliability and prevalence of criminogenic risk factors. The present study utilized data that were gathered using the Level of Service Inventory-Revised (LSI-R). Results and comparisons of these assessments were investigated and presented for Native American and non-Native American offenders as well as male and female offenders. Discussion of the results, implications, and recommendations for further research are presented.     

Social ecology, individual risk, and recidivism: A multilevel examination of main and moderating influences

Purpose

Despite a longstanding tradition in criminology to consider the impact of neighborhood context on crime-related outcomes, criminologists have largely ignored the influence of social ecology on recidivism until recently. The purpose of the present study was to examine the main and moderating influences of social ecology on recidivism.

Materials and Methods

The present study used hierarchical nonlinear modeling to estimate the effects of concentrated disadvantage, immigrant concentration, and residential stability on recidivism for a sample of offenders released from custody/supervision in 2006 and nested within Iowa counties. We controlled for individual-level risk for recidivism using the Level of Service Inventory-Revised (LSI-R), a validated risk assessment instrument. We also examined whether the relationship between LSI-R score and recidivism varied across counties, and if so, whether this variation can be explained by social structural characteristics.

Results

Results indicate that residential stability was the only contextual variable significantly related to recidivism. The relationship between individual-level risk and recidivism did not vary across contexts.

Conclusions

The findings suggest that the social structural context has limited influence on recidivism, while the LSI-R is a robust predictor of recidivism across contexts. We discuss the implications of our findings for theory, practice, and future research.     

Exploring the validity of the Level of Service Inventory-Revised with Native American offenders

The use of actuarial risk/need assessment tools is an increasingly important part of the correctional landscape. Actuarial tools ideally will provide a valid, dynamic assessment of an offender's overall risk/need level, and will identify their most prevalent criminogenic needs. What results is typically a number or score that can be used to assign an offender to a risk level that is associated with an assumed likelihood of recidivism. Testing the predictive validity of actuarial risk/need assessment tools is of paramount concern, particularly when they are utilized with new (and under-researched) populations. The current study assessed the predictive validity of the Level of Service Inventory-Revised using a sample of Native American and White offenders in a northern midwestern state. Results showed the instrument to have modest predictive validity utilizing the entire sample of offenders, with varying results for subsequent subgroups.     

Utility of the Revised Level of Service Inventory (LSI-R) in Predicting Recidivism After Long-Term Incarceration

Assessing an inmate’s risk for recidivism may become more challenging as the length of incarceration increases. Although the population of Long-Term Inmates (LTIs) is burgeoning, no risk assessment tools have been specifically validated for this group. Based on a sample of 1,144 inmates released in a state without parole, we examine the utility of the Level of Service Inventory-Revised (LSI-R) in assessing risk of general and violent felony recidivism for LTIs (n = 555). Results indicate that (a) the LSI-R moderately predicts general, but not necessarily violent, recidivism, and (b) this predictive utility is not moderated by LTI status, and is based in part on ostensibly dynamic risk factors. Implications for informing parole decision-making and risk management for LTIs are discussed.

A Person-centered Approach to Risk and Need Classification in Drug Court

A sample of 265 New York City drug court participants completed the Level of Service Inventory—Revised (LSI-R) and Texas Christian University Drug Screen II (TCUDS). Three participant clusters were identified through a person-centered analysis of their LSI-R and TCUDS responses: low risk (LR), criminogenic risk (CR), and complex behavioral health needs (CBHN). Although CBHN scored higher than CR and LR on the LSI-R and TCUDS, they were no more likely to be re-arrested at 24 months and no higher in their rate of positive drug tests. The CR cluster predicted re-arrest beyond the LSI-R and rate of positive drug tests beyond the LSI-R and TCUDS. CBHN participants placed in a residential (vs. non-residential) setting were disproportionately likely to be re-arrested. Results point to a sub-population of drug court participants not captured in variable-centered summary risk scores, who might require intensive case management or referral to suitable treatment.     

Predicting Recidivism with the Psychological Inventory of Criminal Thinking Styles and Level of Service Inventory-Revised: Screening Version

Recidivism was evaluated in 178 male inmates administered the Psychological Inventory of Criminal Thinking Styles (PICTS) and scored on the Level of Service Inventory-Revised: Screening Version (LSI-R:SV) 1–55 months before their release from prison. Age, prior charges, the LSI-R:SV total score, and the PICTS General Criminal Thinking (GCT), Proactive Criminal Thinking (P), and Reactive Criminal Thinking (R) scores served as predictors of recidivism in follow-ups spanning 1–53 months. Age, prior charges, and the PICTS GCT and R scales consistently and incrementally predicted general recidivism (all charges), whereas prior charges and the PICTS R scale consistently and incrementally predicted serious recidivism (more serious charges). Although these results support the predictive efficacy and incremental validity of content-relevant self-report measures of criminality like the PICTS, they also indicate that the effect is modest and in need of further clarification. One area requiring further investigation is the potential role of the PICTS, particularly the R scale, as a dynamic risk factor.     

The Psychological Inventory of Criminal Thinking Styles and Level of Service Inventory-Revised: Screening Version as Predictors of Official and Self-reported Disciplinary Infractions

A total of 159 male inmates screened with the Psychological Inventory of Criminal Thinking Styles (PICTS) and Level of Service Inventory-Revised: Screening Version (LSI-R:SV) were followed for a period of 24 months for evidence of disciplinary infractions (incident reports). Eighty-three of these inmates also furnished a self-report of disciplinary infractions occurring during the 24-month follow-up. The PICTS General Criminal Thinking (GCT) score and LSI-R:SV total score correlated with and accurately identified the presence of an officially recorded disciplinary infraction, an officially recorded severe disciplinary infraction, and a self-reported disciplinary infraction but only age and the GCT score achieved incremental validity when age, GCT, and LSI-R:SV were included as predictors in the same probit regression or loglinear survival equation.     

Speed Bumps On The Information Superhighway: A Study Of Federal Agency Compliance With The Electronic Freedom Of Information Act Of 1996

The Electronic Freedom of Information Act of 1996 amended the Freedom of Information Act of 1966 to clarify that computerized agency records are subject to disclosure under the FOIA. Congress passed the 1996 amendments because federal agencies for many years had been withholding computerized records and databases from the public. Another feature of the EFOIA requires agencies to publish their policies and rules on the Internet. The rationale behind this provision was to guard against the development of "secret law" known only to agency officials, but not to members of the public who deal with the agencies. The purpose of this article is to examine how federal agencies have complied with the EFOIA in its first few years of implementation. A look at the documented early history of EFOIA compliance reveals agency performance has been marked by slow starts, missed deadlines and general confusion over exactly what kinds of records qualify as disclosable records under the statute.     

Predicting institutional misconduct using the Youth Level of Service/ Case Management Inventory

Offender assessment in corrections has advanced considerably over the last several decades. Currently, it is not uncommon to find correctional professionals using any number of objective standardized assessment instruments. While many of these instruments possess face validity as well as statistical predictive validity, more work is needed to test classification protocol on new populations, and in various correctional environments. The current paper investigates the predictive validity of the Youth Level of Service/Case Management Inventory (YLS/CMI) within an institutional setting. Specifically, the composite score rendered from the YLS/CMI is used to predict institutional misconduct. The YLS/ CMI was found to effectively differentiate between two levels of offender risk/ need, and was significantly related to outcome using several different statistical techniques.     

System Change through Collaboration…Eight Steps for Getting from There to Here

Family courts and child welfare agencies across the country are charged with protecting the safety of our children. That mission has become more challenging with increasing federal legislation and decreasing funding. In Buffalo, N.Y., the Family Court and the Department of Social Services have teamed up to respond to this challenge. With minimal additional staffing and resources, they have led a collaboration of agencies and service providers to change the way business is done in child welfare. By engaging each other in an interagency system change effort, the amount of time children spend in foster care has been reduced. The collaboration has been able to accomplish in a relatively short time what no agency had previously been able to accomplish on its own. The beneficiaries have been the children and families of Erie County.     

Are the major risk/need factors predictive of both female and male reoffending?: a test with the eight domains of the level of service/case management inventory

The Level of Service/Case Management Inventory (LS/CMI) and the Youth version (YLS/CMI) generate an assessment of risk/need across eight domains that are considered to be relevant for girls and boys and for women and men. Aggregated across five data sets, the predictive validity of each of the eight domains was gender-neutral. The composite total score (LS/CMI total risk/need) was strongly associated with the recidivism of males (mean r = .39, mean AUC = .746) and very strongly associated with the recidivism of females (mean r = .53, mean AUC = .827). The enhanced validity of LS total risk/need with females was traced to the exceptional validity of Substance Abuse with females. The intra-data set conclusions survived the introduction of two very large samples composed of female offenders exclusively. Finally, the mean incremental contributions of gender and the gender-by-risk level interactions in the prediction of criminal recidivism were minimal compared to the relatively strong validity of the LS/CMI risk level. Although the variance explained by gender was minimal and although high-risk cases were high-risk cases regardless of gender, the recidivism rates of lower risk females were lower than the recidivism rates of lower risk males, suggesting possible implications for test interpretation and policy.     

Risk assessment for intimate partner violence: how can the police assess risk?

ABSTRACT

The use of risk assessment tools by frontline police for intimate partner violence has the potential to make a difference to policing. In this paper, the key aspects of intimate partner violence risk assessment are outlined critically with a particular emphasis on how they can be used in practice. Two, evidence-based, exemplars are reviewed. These are the Ontario Domestic Abuse Risk Assessment (ODARA), an example of the actuarial approach, and the Brief Spousal Assault Form for the Evaluation of Risk (B-SAFER), an example of the structured professional judgement approach. In addition, the victim giving his or her own appraisal of risk is discussed. All three approaches have some validity when administered properly but practical factors reduce this validity. The content of the risk assessment tools are outlined and practical concerns such as training, time to administer, reliability, validity, and the overlap of intimate partner violence with other forms of offending are discussed. A balanced overview of the strengths, weaknesses and future potential of intimate partner violence risk assessment is provided.     

Using Reasonable Efforts Determinations to Improve Systems and Case Practice in Cases Involving Family Violence and Child Maltreatment

The Adoption and Safe Families Act (ASFA) of 1997 reinforced that the safety, permanency, and well‐being of the child should be the primary concerns when making decisions about child protection interventions, child placement, and efforts at reunification. The court's role in oversight of agency practice in individual cases through the requirement of specific judicial findings as a condition of receipt of certain funding was also maintained and strengthened by ASFA. Based on the recognition of the number of cases where there is a co‐occurrence of domestic violence and child maltreatment, there is a need for communities and agencies to set reasonable expectations of good practice for responding to the issues raised. As the community sets the expectations of good practice through agency policy, training, and service delivery, the judiciary, through the findings regarding “continuation in the home” or “reasonable efforts” in each individual case, provides the oversight of practice required by ASFA. This article will explore the current applications of reasonable efforts, discuss ways that courts and communities are defining the concept, and examine the need for the development of a reasonable efforts protocol.     

Issues in violent risk assessment: lessons learned and future directions

Realizing that the assessment of dangerousness with a yes/no format as a poor form of violent risk assessment has been the most important lesson learned about violence in the last 20 years. Further examining (a) what outcome and (b) how the indicators of the outcome should be measured has resulted in better violent risk assessment. The most promising methodological innovation in violent risk assessment has been the introduction of the receiver operating characteristic to assess the efficiency of risk prediction. In the future, the tension between prediction and explanation of violence will need to be maintained while focusing on a mechanism-driven strategy of risk management.     

Perceptions of Law Enforcement Officers in Seeking Mental Health Treatment in a Right-to-Work State

The high stress environment of law enforcement places officers at risk for a variety of mental and physical health problems; however, officers are reluctant to seek out treatment. The purpose of this study was to identify which factors associated with law enforcement officers had predictive value in the level of stigma perceived in seeking mental health treatment by currently employed, certified peace officers in Arizona. The factors included sex, age, race/ethnicity, years employed as an officer, size of the department, current rank/position within the department, the type of government operating the agency, and payment of union dues. The study included 454 participants. Participation included the completion of demographic data, the Self-Stigma of Seeking Psychological Help (SSOSH), and the Perceptions of Stigmatization by Others for Seeking Help (PSOSH). Using standard multiple regression, the most significant finding was the relationship between the size of the department and levels of stigma (p = .014); such that, the size was inversely related to the levels of perceived stigma. As the agency size increased, the perceptions of stigma decreased. Such a finding has several implications for law enforcement agencies related to preparedness and training.