In 2011, a study published in Science found that Black applicants for the prestigious and highly competitive National Institutes of Health (NIH) R01 research grants were 10 percentage points less likely to receive funding than their white counterparts. NIH R01 grants provide four or five years of independent support and can be crucial to a junior faculty member’s chances for promotion, tenure and ability to pursue a major avenue of research.
In the wake of that finding, the NIH announced a $10,000 prize for the most creative way to detect bias in the R01 peer review process and identify where in the process the disparity originates. The winners were two UW researchers: Elena Erosheva, (pictured right) a professor in the departments of statistics and social work, and Carole Lee, (pictured left) a professor of philosophy who specializes in peer review.
This month, Erosheva and Lee published the findings from their resulting study, which focuses on an early phase of the R01 review process in which assigned peer reviewers read applications and score them on five criteria—significance, innovation, approach of the grant application, investigator(s) and research environment—and then weigh those scores as they see fit to derive a “preliminary overall impact score” that reflects the project’s potential for major scientific impact. The better the overall impact score, the more likely the application is to advance to discussion at the next round and ultimately receive funding.
Erosheva and Lee found that Black researchers receive worse scores on all five criteria—even after controlling for key application and applicant characteristics, such as career stage and area of science—and that these differences entirely account for black-white disparities in overall impact scores. How reviewers weigh the criterion scores appears to not be a factor in explaining black-white disparities in preliminary overall impact scores.
This finding is particularly notable because some people have assumed that use of the five criteria, introduced by the NIH in 2009 to add transparency to the process, would help reduce racial disparities and racial bias in funding decisions.
Like researchers before them, Erosheva and Lee identified substantial racial disparity under the R01 review system. They found that from 2014 to 2016, the overall award rate for black researchers was 55% of that for white researchers—10.2% compared to 18.5%.
“The next step for understanding black-white disparities in NIH peer review is to understand why Black principal investigators are perceived as not excelling along these specific dimensions of evaluation,” says Erosheva.
Erosheva and Lee note that their study examines associations in existing data and does not draw causal conclusions, but they suggest some potential factors worthy of further investigation, including implicit racial preferences, choice of research topic, differences in mentorship and coauthor networks, emphasis on bibliometrics (such as citation counts), and the cumulative effect of disparities over a researcher’s career.
The study’s findings may be particularly valuable to those who study and work to promote diversity in STEM fields.
“This is an exceptional and timely research study that addresses a major scientific problem,” says David Takeuchi, associate dean for faculty excellence at the UW School of Social Work. “The result is a rigorous set of analyses that provide compelling insights about the problem and what more research is needed to correct this inequality.”
Related Articles
Chronicle of Higher Education
UW Center for Statistics and the Social Sciences