Performance measures as forms of evidence for science and technology policy decisions

Amidst current widespread calls for evidence based decision making on public investments in science and technological innovation, frequently interpreted to imply the employment of some bundle of output, outcome, productivity, or rate-of-return measures, the promises and limitations of performance measures, singly or collectively, varies greatly across contexts. The promises reflect belief in, scholarly research supportive of, and opportunistic provision of performance measures that respond or cater to executive and legislative branch expectations or hopes that such measures will facilitate evidence-based decision-making. The limitations reflect research on the dynamics of scientific discovery, technological innovation and the links between the two that even when well done and used by adepts, performance measures at best provide limited guidance for future expenditure decisions and at worst are rife with potential for incorrect, faddish, chimerical, and counterproductive decisions. As a decision-making enhancement, performance measurement techniques have problematic value when applied to the Big 3 questions of U.S. science policy: (1) what is the optimal size of the Federal government’s investments in science and technology programs; (2) the allocation of these investments among missions/agencies/and programs (and thus fields of science); and (3) the selection of performers, funding mechanisms, and the criteria used to select projects and performers.