Peer review of grant applications is employed routinely by health research funding bodies to determine which research proposals should be funded. Peer review faces a number of criticisms, however, especially that it is time consuming, financially expensive, and may not select the best proposals. Various modifications to peer review have been examined in research studies but these have not been systematically reviewed to guide Health Technology Assessment (HTA) funding agencies.

We developed a systematic map based on a logic model to summarize the characteristics of empirical studies that have investigated peer review of health research grant applications. Consultation with stakeholders from a major health research funder (the United Kingdom National Institute for Health Research, NIHR) helped to identify topic areas within the map of particular interest. Innovations that could improve the efficiency and/or effectiveness of peer review were agreed as being a priority for more detailed analysis. Studies of these innovations were identified using pre-specified eligibility criteria and were subjected to a full systematic review.

The systematic map includes eighty-one studies, most published since 2005, indicating an increasing area of investigation. Studies were mostly observational and retrospective in design, and a large proportion have been conducted in the United States, with many conducted by the National Institutes of Health. An example of an innovation is video training to improve reviewer reliability. Although research councils in the United Kingdom have conducted several relevant studies, these have mainly examined existing practices rather than testing peer review innovations. Full results of the systematic review will be provided in the presentation, and we will assess which innovations could improve the efficiency and/or effectiveness of peer review for selecting health research proposals.

Despite considerable interest in, and criticism of, peer review for helping to select health research proposals, there have been few detailed systematic examinations of the primary research evidence in this area. Our evidence synthesis provides the most up-to-date overview of evidence in this important developing area, with recommendations for health research funders in their decision making.