Evaluation of research in context: an approach and two cases

In this paper we present the findings of two case studies in fields where societal and scholarly output of research are highly intertwined: architecture and law. We analyze the nature of the two fields in terms of research areas and specific aspects of knowledge dynamics. This results in an approach and indicators for contextual research evaluation.

In this approach, the frame of reference is the mission of the research group, as this defines what the group is expected to accomplish. First, the different scholarly and societal audiences (or stakeholders) are identified. Second, in interaction with researchers and their audiences, the specific types of output for these audiences, and the interactions between researchers and audiences are identified. Third, indicators for research output, quality, and impact are developed. Fourth, in contrast to what is done usually, we propose a more general quality concept that takes into account the quality of communication and collaboration between researchers and their audiences: productive interactions. Finally, stakeholders are involved in the assessment, of societal impacts, comparable to scientific peers who are able to evaluate scientific impact.

The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement No. 230330.