| Abstract: | Because research synthesis enables one to determine either the overall effectiveness of different types of treatments, it is becoming increasingly popular as a tool in program evaluation. Numerous methodological problems arise, however, when research synthesis is applied to studies conducted in field settings. The present paper categorlzes and discusses these problems as being threats to either the (1) internal validity (whether one can draw conclusions about cause and effect), (2) statistical conclusion validity (whether one's inferential statistics are capable of detecting cause-and effect relationships, (3) construct validity (whether one's treatments and outcome measures are valid operationalizations of the independent and dependent variables of interest), or (4) external validity (whether one can generalize results to particular populations, settings, or time periods) of research synthesis (see Cook & Campbell, 1979). Specific recommendations are made for minimizing these threats to validity, In order to improve the quality of research synthesis in program evaluation. Finally, the validit framework is applied to an actual synthesis to identify strengths, weaknesses, and potential means of enhancing research quality. |
