Evaluation technology transfer: Some problems and solutions

Evaluation of technology transfer is an important part of the total transfer process. Sound results require a practical approach, which avoids use of the research model. The evaluation process is often expensive and time consuming. However, good evaluation will improve:motivation,knowledge,decisions, andaccountability. Evaluation must be based on use of reliable data. Technology transfer evaluation data can be classified — based on accuracy — aslow,medium, orhigh order — data. Evaluation problems can be designated — according to origin — as being related tonew information,human elements, orinstitutional structure. In each area, the more common evaluation problems can be solved with careful attention to detail. Technology transfer evaluation can be a complicated, but rewarding process. There is nosingle, correct way for Technology Transfer evaluation, but rather there is a wide variety of techniques which all have merit, depending on local circumstances.     

Long-term follow-up of a preschool experiment

Objectives

This study was designed to provide experimental evidence of the effects of a preschool program on young children living in poverty. It began as a program evaluation but now, half a century later, serves as a test of the long-term effects and return on investment of high-quality preschool education for young children living in poverty.

Methods

This study was conducted in the U.S., beginning in the 1960s, and has generated data on study participants from birth through 40, with new data now being collected at age 50. The study used random assignment procedures to assign 123 children to a preschool program and a control group who receive no preschool program.

Results

Program participants surpassed non-participants in intellectual performance at school entry, school achievement throughout schooling, commitment to schooling, high school graduation rate, adult employment rate and earnings, reduced childhood antisocial behavior, and reduced adult crime and incarceration. The program's return on investment was at least seven times as great as its operating cost.

Conclusions

While these powerful results have been found not only in this study but in several similar studies, they have not been found in studies of larger preschool programs, such as the Head Start Impact Study. This discrepancy suggests that differences between the two types of programs account for the better results found in studies such as this one. Among these differences are highly qualified teachers, a valid child development curriculum, extensive engagement of parents, and regular assessment of program implementation and children's development     

A decision support system model for technology transfer

Technology transfer is the process by which technology originating at one institutional setting is adapted for use in another. A major impediment to the implementation of new technologies to assist with mangerial decision-making problems is a lack of communication between the technology and management communities. Development of a tool designed to bridge the technology transfer gap was the goal of this research. The result is a prototype software package which may be used on an interactive computer terminal by a manager for assistance in designing a decision support system (DSS). The four primary research tasks were:

1. Develop a conceptual model of the DSS design process.
2. Select and adapt, or create, appropriate software to mechanize the model.
3. Develop a knowledge base to describe the interactiveness of various organization variables and managerial decision-making needs.
4. Collect and analyze interview data and implement resultant production rules on the model.

Tasks 1 and 2 were accomplished first to establish the feasibility of this effort. An interview instrument was developed for Task 3. And, corporate managers from several firms were interviewed to accomplish Task 4. Using this data, a prototype production rule model (called DECAIDS for DECISION AIDS) was constructed which supports managerial decision-making from the EMYCIN production rule system used at Stanford University. The purpose of this article is to introduce the need for a Decision Support System Model. A complete copy of this research can be obtained through University Microfilms International, 300 N. Zeeb Road, Ann Arbor, Michigan 48106, from the Naval Postgraduate School, Monterey, California 93940, or the Defense Technical Information Center, Cameron Station, Alexandria, Virginia 22314. The title is “An Interactive Decision Support System for Technology Transfer Pertaining to Organization and Management”, 1980.     

Adverse Neighborhood Conditions and Sanction Risk Perceptions: Using SEM to Examine Direct and Indirect Effects

Objectives

The present study examines how individuals’ sanction risk perceptions are shaped by neighborhood context.

Methods

Using structural equation modeling on data from waves 6 and 7 of the National Youth Survey, we assess the direct and indirect relationships between adverse neighborhood conditions and two dimensions of sanction risk perceptions: the certainty of punishment and perceived shame. In addition, the role of shame as a mediator between neighborhood context and certainty of punishment is also investigated.

Results

The results indicate that adverse neighborhood conditions indirectly affect both forms of sanction risk perceptions, and additional results show that perceived shame fully mediates the effect of neighborhood conditions on perceptions of the certainty of punishment.

Conclusions

The perceptual deterrence/rational choice perspective will need to be revised to accommodate more explicitly the role of neighborhood context in shaping sanction risk perceptions.     

Sustainability of the effects of multisystemic therapy for juvenile delinquents in The Netherlands: effects on delinquency and recidivism

Objectives

The present study focused on the sustainability of the effects of Multisystemic Therapy (MST) on delinquency and recidivism.

Methods

A sample of 256 juveniles with severe and persistent antisocial behavior were randomly assigned to MST (147) and Treatment As Usual (TAU) (109) condition. Pre-test assessment took place before the start of MST/control group treatment. Post-test assessment took place at 6 months after termination of the program. Delinquency (parent and adolescent reported) was assessed 6 months after termination of the treatment. Official judicial data were collected to assess recidivism, with a mean length of follow-up of 3.06 years. ANCOVAs and survival analyses were used to test the effectiveness of MST.

Results

The multi-informant data showed that MST is effective in diminishing delinquent behavior as reported by adolescents and parents, with d’s larger than at post-test assessment immediately after ending of the intervention. The official judicial data, however, suggest that there are no differences between MST and TAU in recidivism. Few and inconsistent moderator effects were found.

Conclusions

According to parent and adolescent reports, the beneficial effects of MST were sustained at the follow-up. This was not supported by official data. These results stress the importance of using multi-informant data on delinquency, as each source of information has its own advantages and disadvantages.     

Accounting for Projection Bias in Models of Delinquent Peer Influence: The Utility and Limits of Latent Variable Approaches

Objectives

Projection effects have been shown to bias respondent perceptions of peer delinquency, but network data required to measure peer delinquency directly are unavailable in most existing datasets. Some researchers have therefore attempted to adjust perceived peer behavior measures for bias via latent variable modeling techniques. The present study tested whether such adjustments render perceived peer coefficients equal to direct peer coefficients, using original data collected from 538 young adults (269 dyads).

Methods

After first replicating projection effects in our own data and examining the degree to which measures of personal, perceived peer, and direct peer violence represent empirically distinct constructs, we compared coefficients derived from two alternative models of personal violence. The first model included an error-adjusted latent measure of perceived peer violence as a predictor, whereas the second substituted a latent measure of directly-assessed, peer-reported violence.

Results

Results suggest that personal, perceived peer, and direct peer measures each reflect fundamentally separate constructs, but call into question whether latent variable techniques used by prior researchers to correct for respondent bias are capable of rendering perceived peer coefficients equal to direct peer coefficients.

Conclusions

Research cannot bypass the collection of direct peer delinquency measures via latent variable modeling adjustments to perceived peer measures, nor should models of deviance view perceived peer and direct peer measures as alternative measures of the same underlying construct. Rather, theories of peer influence should elaborate and test models that simultaneously include both peer measures and, further, should attempt to identify those factors that account for currently unexplained variance in perceptions of peer behavior.     

Technology transfer model for Austrian higher education institutions

The aim of this paper is to present the findings of a PhD research (Heinzl 2007, Unpublished PhD Thesis) conducted on the Universities of Applied Sciences in Austria. Four of the models that emerge from this research are: Generic Technology Transfer Model (Sect. 5.1); Idiosyncrasies Model for the Austrian Universities of Applied Sciences (Sect. 5.2); Idiosyncrasies-Technology Transfer Effects Model (Sect. 5.3); Idiosyncrasies-Technology Transfer Cumulated Effects Model (Sect. 5.3). The primary and secondary research methods employed for this study are: literature survey, focus groups, participant observation, and interviews. The findings of the research contribute to a conceptual design of a technology transfer system which aims to enhance the higher education institutions’ technology transfer performance.     

The Efficacy of Ideographic Models for Geographical Offender Profiling

Objectives

Current ‘geographical offender profiling’ methods that predict an offender’s base location from information about where he commits his crimes have been limited by being based on aggregate distributions across a number of offenders, restricting their responsiveness to variations between individuals as well as the possibility of axially distorted distributions. The efficacy of five ideographic models (derived only from individual crime series) was therefore tested.

Methods

A dataset of 63 burglary series from the UK was analysed using five different ideographic models to make predictions of the likely location of an offenders home/base: (1) a Gaussian-based density analysis (kernel density estimation); (2) a regression-based analysis; (3) an application of the ‘Circle Hypothesis’; (4) a mixed Gaussian method; and (5) a Minimum Spanning Tree (MST) analysis. These tests were carried out by incorporating the models into a new version of the widely utilised Dragnet geographical profiling system DragNetP. The efficacy of the models was determined using both distance and area measures.

Results

Results were compared between the different models and with previously reported findings employing nomothetic algorithms, Bayesian approaches and human judges. Overall the ideographic models performed better than alternate strategies and human judges. Each model was optimal for some crime series, no one model producing the best results for all series.

Conclusions

Although restricted to one limited sample the current study does show that these offenders vary considerably in the spatial distribution of offence location choice. This points to important differences between offenders in the morphology of their crime location choice. Mathematical models therefore need to take this into account. Such models, which do not draw on any aggregate distributions, will improve geographically based investigative decision support systems.     

Bayesian Spatio-Temporal Modeling for Analysing Local Patterns of Crime Over Time at the Small-Area Level

Objectives

Explore Bayesian spatio-temporal methods to analyse local patterns of crime change over time at the small-area level through an application to property crime data in the Regional Municipality of York, Ontario, Canada.

Methods

This research represents the first application of Bayesian spatio-temporal modeling to crime trend analysis at a large map scale. The Bayesian model, fitted by Markov chain Monte Carlo simulation using WinBUGS, stabilized risk estimates in small (census dissemination) areas and controlled for spatial autocorrelation (through spatial random effects modeling), deprivation, and scarce data. It estimated (1) (linear) mean trend; (2) area-specific differential trends; and (3) (posterior) probabilities of area-specific differential trends differing from zero (i.e. away from the mean trend) for revealing locations of hot and cold spots.

Results

Property crime exhibited a declining mean trend across the study region from 2006 to 2007. Variation of area-specific trends was statistically significant, which was apparent from the map of (95 % credible interval) differential trends. Hot spots in the north and south west, and cold spots in the middle and east of the region were identified.

Conclusions

Bayesian spatio-temporal analysis contributes to a detailed understanding of small-area crime trends and risks. It estimates crime trend for each area as well as an overall mean trend. The new approach of identifying hot/cold spots through analysing and mapping probabilities of area-specific crime trends differing from the mean trend highlights specific locations where crime situation is deteriorating or improving over time. Future research should analyse trends over three or more periods (allowing for non-linear time trends) and associated (changing) local risk factors.     

The Effect of Incarceration on Re-Offending: Evidence from a Natural Experiment in Pennsylvania

Objectives

This paper uses a sample of convicted offenders from Pennsylvania to estimate the effect of incarceration on post-release criminality.

Methods

To do so, we capitalize on a feature of the criminal justice system in Pennsylvania—the county-level randomization of cases to judges. We begin by identifying five counties in which there is substantial variation across judges in the uses of incarceration, but no evidence indicating that the randomization process had failed. The estimated effect of incarceration on rearrest is based on comparison of the rearrest rates of the caseloads of judges with different proclivities for the use of incarceration.

Results

Using judge as an instrumental variable, we estimate a series of confidence intervals for the effect of incarceration on one year, two year, five year, and ten year rearrest rates.

Conclusions

On the whole, there is little evidence in our data that incarceration impacts rearrest.     

A Sensitivity Analysis of Egocentric Measures of Peer Delinquency to Latent Homophily: A Research Note

Objectives

Egocentric measures of peer delinquency, obtained through a census of a social network, have become the preferred operationalization for examining the relationships between social influence and delinquency. Studies regressing ego’s delinquency on the delinquency of nominated friend/s (i.e. alter/s) conclude that a statistically significant coefficient provides evidence of social influence. However, the inferences drawn from these studies may be biased by the introduction of artificial statistical dependence as a consequence of using social network data in a regression framework. Recent work (Shalizi and Thomas Sociol Methods Res 40:211–239, 2011) shows that latent homophily, or unmeasured confounding of observables, may lead to nonzero estimates of social influence, even if there is no causal significance. To examine this possibility, sensitivity analyses have been created (e.g. VanderWeele and Arah Epidemiology 22:42–52, 2011; VanderWeele Sociol Methods Res 40:240–255, 2011) to determine the robustness of an estimated coefficient to latent homophily.

Methods

In this research note, I examine the robustness of estimates for social influence from two articles (Haynie Am J Sociol 106:1013–1057, 2001; Meldrum et al. J Res Crime Delinq 46:353–376, 2009) using egocentric measures of peer delinquency.

Results

Findings indicate that for large, precise point estimates, highly improbable conditions are needed to explain away the effects of social influence. However, less precise point estimates (i.e. large standard errors) are more sensitive to latent homophily.

Conclusions

The analyses indicate that studies using egocentric measures should conduct sensitivity tests, particularly when the estimated effect is weak and/or has a relatively large standard error. Scripts written in the free programming language R (R Core Team R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, 2012) are provided for researchers to conduct such analyses.     

Forecasts of Violence to Inform Sentencing Decisions

Objectives

Recent legislation in Pennsylvania mandates that forecasts of "future dangerousness" be provided to judges when sentences are given. Similar requirements already exist in other jurisdictions. Research has shown that machine learning can lead to usefully accurate forecasts of criminal behavior in such setting. But there are settings in which there is insufficient IT infrastructure to support machine learning. The intent of this paper is provide a prototype procedure for making forecasts of future dangerousness that could be used to inform sentencing decisions when machine learning is not practical. We consider how classification trees can be improved so that they may provide an acceptable second choice.

Methods

We apply an version of classifications trees available in R, with some technical enhancements to improve tree stability. Our approach is illustrated with real data that could be used to inform sentencing decisions.

Results

Modest sized trees grown from large samples can forecast well and in a stable fashion, especially if the small fraction of indecisive classifications are found and accounted for in a systematic manner. But machine learning is still to be preferred when practical.

Conclusions

Our enhanced version of classifications trees may well provide a viable alternative to machine learning when machine learning is beyond local IT capabilities.     

For whom does prison-based drug treatment work? Results from a randomized experiment

Objectives

Prison-based therapeutic community (TC) drug treatment followed by community aftercare is widely recognized as the most effective treatment paradigm for drug-dependent offenders. However, few randomized experiments have addressed this question and fewer studies have examined how interactions between treatment modality and individual characteristics may explain variations in outcomes.

Methods

Using a randomized experimental design, this study examined the effects of treatment modality [TC vs. Outpatient (OP) group counseling], individual psychosocial characteristics (e.g., risk, negative affect), and interactions on reincarceration over a 3-year follow-up period. Survival analysis using Cox regression with covariates was used to analyze data obtained from 604 subjects at a specialized drug treatment prison.

Results

The expected advantage of TC failed to emerge. Critical and heretofore unexamined interactions between treatment modality (TC vs. OP), inmate levels of risk, and negative effect help explain these unexpected findings.

Conclusion

The superiority of prison TC to less intensive OP counseling was not supported. The effects of TC appear to be conditioned by critical responsivity factors that have received little empirical attention.     

Assessing programs designed to improve outcomes for children exposed to violence: Results from nine randomized controlled trials

Objectives

The study tests whether participation in interventions offered by a subset of sites from the National Safe Start Promising Approaches for Children Exposed to Violence initiative improved outcomes for children relative to controls.

Methods

The study pools data from the nine Safe Start sites that randomized families to intervention and control groups, using a within-site block randomization strategy based on child age at baseline. Caregiver-reported outcomes, assessed at baseline, 6 and 12 months, included caregiver personal problems, caregiver resource problems, parenting stress, child and caregiver victimization, child trauma symptoms, child behavior problems, and social-emotional competence.

Results

Results revealed no measurable intervention impact in intent-to-treat analyses at either 6- or 12-month post-baseline. In 6-month as-treated analyses, a medium to high intervention dose was associated with improvement on two measures of child social-emotional competence: cooperation and assertion. Overall, there is no reliable evidence of significant site-to-site effect variability, even in the two cases of significant intervention effect.

Conclusions

Since families in both the intervention and control groups received some degree of case management and both groups improved over time, it may be advantageous to explore the potential impacts of crisis and case management separately from mental health interventions. It may be that, on average, children in families whose basic needs are being attended to improve substantially on their own.     

Building group capacity for problem solving and police–community partnerships through survey feedback and training: a randomized control trial within Chicago’s community policing program

Objectives

To examine whether group capacity for problem solving and partnership building could be enhanced at police–community meetings by providing the results from community surveys and training for group facilitators.

Methods

A randomized control trial was conducted in 51 police beats in Chicago’s community policing program, CAPS. Unlike control beats, results from web-based community surveys were provided at beat meetings in the feedback and training beats, with facilitators in training beats also receiving training and exercises to guide problem solving about survey results. Analysis included OLS and logistic regression of data from questionnaires administered to police and resident participants, as well as observations at beat meetings, which measured resident capacity, attitudes about the police–community partnership, and problem-solving activities.

Results

Support for hypothesized effects was found with greater resident confidence in their ability to achieve outcomes and solve local problems, as well as officers viewing their relationships with residents at beat meetings more favorably. Effects, however, were inconsistent and limited to the feedback group. While additional training and support provided in training beats indicated fuller engagement in problem solving, possible negative effects on attitudes were observed.

Conclusions

Failure to find more effects is discussed in terms of implementation and resistance. Officer resistance to and a shift in organizational priorities away from community policing worked against achieving full program implementation. The beat meeting context provided a traditional framework for police–resident interactions that precluded more comprehensive use of community data and possibly heightened dissatisfaction with the level of problem solving that occurred.     

Some thoughts on the application of new technology to resource management needs

A brief insight into the phenomena of technology transfer are reviewed. With this as background material two propositions are stated:

1. Technology emerges from the human mind into widespread economic reality, with resulting social impacts over a long period of time. This is the process of technological innovation.
2. There always are some distinctive events, points or stages in this process that mark progress. If they can be identified, progress can be measured and related to time, cost, performance, resource usage, possible impact and other attributes. It is argued that there are at least eight identifiable stages of the process.

These stages are then discussed and conclusions are drawn.     

Terrorism Risk, Resilience and Volatility: A Comparison of Terrorism Patterns in Three Southeast Asian Countries

Objective

This article explores patterns of terrorist activity over the period from 2000 through 2010 across three target countries: Indonesia, the Philippines and Thailand.

Methods

We use self-exciting point process models to create interpretable and replicable metrics for three key terrorism concepts: risk, resilience and volatility, as defined in the context of terrorist activity.

Results

Analysis of the data shows significant and important differences in the risk, volatility and resilience metrics over time across the three countries. For the three countries analysed, we show that risk varied on a scale from 0.005 to 1.61 “expected terrorist attacks per day”, volatility ranged from 0.820 to 0.994 “additional attacks caused by each attack”, and resilience, as measured by the number of days until risk subsides to a pre-attack level, ranged from 19 to 39 days. We find that of the three countries, Indonesia had the lowest average risk and volatility, and the highest level of resilience, indicative of the relatively sporadic nature of terrorist activity in Indonesia. The high terrorism risk and low resilience in the Philippines was a function of the more intense, less clustered pattern of terrorism than what was evident in Indonesia.

Conclusions

Mathematical models hold great promise for creating replicable, reliable and interpretable “metrics” to key terrorism concepts such as risk, resilience and volatility.     

Early prevention of life-course personal and property violence: a 19-year follow-up of the Montreal Longitudinal-Experimental Study (MLES)

Objectives

This study assessed the impact of a multi-component prevention program on personal and property violence across three developmental periods (early adolescence, mid-adolescence and late adolescence/early adulthood).

Methods

The preventive intervention targeted disruptive kindergarten boys from low socioeconomic status families when they were 7 through 9 years of age. A randomized control trial was conducted to assess the impact of the preventive intervention relative to a control group.

Results

Two different approaches to data analysis were adopted: an intention-to-treat (ITT) approach and an instrumental variable (IV) approach. Results from the ITT analysis showed that the level of property violence for the intervention group was persistently lower across the three developmental periods compared to the control group. However, the intervention group did not differ from the control group on personal violence throughout adolescence and early adulthood. Results from the IV analysis generally confirmed these findings.

Conclusions

The discussion focuses on the differential effects of the prevention program on personal versus property violence.     

Poaching, habitat loss and the decline of neotropical parrots: a comparative spatial analysis

Objective

To evaluate the relative contributions of poaching and habitat loss to the endangerment of neotropical parrots

Method

A matched case-control design was employed. Using NatureServe digitized range maps for birds in the Americas, 145 neotropical parrot species were individually matched with 145 control species, from similar ranges and, by proxy, from similar habitats. The control species were taxonomically similar, mid-sized, forest-dwelling birds that, like parrots, use holes and cavities for breeding and roosting. The conservation status of the parrots and control birds was established through the IUCN Red List of endangered species.

Results

Nearly five times as many parrot species (59 out of 145) as non-parrots (13 out of 145) are threatened with extinction to some degree. In 54 out of 65 pairs with unequal conservation status, the parrot species is at greater risk of extinction.

Conclusions

Subject to limitations of the matching employed and the use of range data as a proxy for habitat, it is concluded that poaching is a strong threat to the conservation of neotropical parrots—perhaps stronger than habitat loss. Criminologists therefore have an important part to play in conservation of parrots, and perhaps also in conservation of other endangered species, through identification and implementation of measures to control poaching.     

Does Spending Time in Public Settings Contribute to the Adolescent Risk of Violent Victimization?

Objectives

Using data from a nationally representative survey of adolescents in Finland this research examined the influence of spending time in public settings on the risk of physical assault and robbery victimization.

Methods

Binary and multinomial regression models were estimated to disaggregate associations between hours spent in public settings and characteristics of the victimization incident. The amount of causality/spuriousness in the association was examined using a method of situational decomposition.

Results

Our findings indicate that: (1) an active night life (any time after 6 pm) has a strong effect on victimization for boys, whereas much of the association between night life and victimization is spurious for girls; (2) after-school activity is not a risk factor; (3) adolescents who frequent public places at night increase their risk of victimization by people they know as well as strangers; and (4) much of the risk of night time activity in public settings is alcohol-related.

Conclusions

Our research suggests that a good deal of the risk associated with spending time in public settings is a function of the victim’s own risky behavior rather than inadvertent physical contact with motivated offenders in the absence of capable guardians. In addition, this lifestyle is significantly more victimogenic for males.