Peers Inside the Black Box: Deciding Excellence

By Karen E Mow.

Published by The Social Sciences Collection

Format Price
Article: Print $US10.00
Article: Electronic $US5.00

Excellence is decided inside the grant selection process, in what some applicants call the “black box”. This discussion examines the process from the perspective of peer reviewers as selection panel members and external assessors. It includes discussion of the ways that these peers review and rank applications for research funding. In particular, it seeks to reveal how peers define excellence and whether the ways that grant peer review is operationalised affects the ways that excellence is defined.
Data from 23 semi-structured interviews with panel members and council staff are augmented by responses from 18 panel members and 70 external assessors collected in a survey of applicants. These responses form the basis of a discussion and reflection on how comparative selection decisions are made and the ways that external reviewers see the process of determining excellence. These data open to scrutiny the invisible hand of scientific opinion, the judgements scientists make within research council processes (Rip, 1993).
Excellence emerged as a multi-faceted or polymorphic (Lamont, 2009) construction but clarity and writing dominated opinions about what was the best. Every panel member interviewed held that the difference between the best (5-10%) and the rest is the way that the Text component is written. Applications resonate, are compelling, convincing and shine off the page. They are like really good literature. Panel members believe that communicating the work of research is an integral part of the activity and that all researchers have an obligation to deliver clear stories about their work to the world at large, that in fact, as one influential Panel member said; get over it, it’s the job. In summary, the best applications are well articulated, professionally prepared and exciting to read. Panel members believe that people who can tell the story also do the best science.

Keywords: Peer Review, Research Grant Funding, Grant Selection, Excellence

International Journal of Interdisciplinary Social Sciences, Volume 5, Issue 10, pp.175-184. Article: Print (Spiral Bound). Article: Electronic (PDF File; 641.262KB).

Dr. Karen E Mow

Executive Manager, Institute for Applied Ecology, University of Canberra, Canberra, ACT, Australia

Dr Karen Mow manages the Institute for Applied Ecology at the University of Canberra, Australia. She specializes in building research capacity and developing research teams. She includes teaching and research in a busy leadership and administrative role. Her Doctoral thesis, Inside the Black Box, explored relationships between administrative processes and research funding decisions across several research councils, with analysis from the perspective of the staff, selection panel members, external assessors and applicants.

Reviews:

There are currently no reviews of this product.

Write a Review