Grant Panels as Prom Committees

I am only in my early 50s, yet I have lost count of the number of grant and contract review committees on which I have served. I wish I could say it is because I am some sort of academic superstar. In fact, it is just the combination of an odd specialization that crosses technical boundaries and a substantially cooperative nature. I always agree when asked by panel officers from the National Institutes of Health for two reasons: I have been on the government side of the aisle, and I desperately need people to review my stuff. The upside of my unfettered willingness to do what is essentially continuous community service is that I hear about interesting research and novel techniques that would have never crossed my desk of their own accord. The downside is that I spend a lot of time sitting in windowless hotel conference rooms in and around Washington, trying to stay engaged in conversations at 3 o’clock on Friday afternoons. I have developed waiting habits not dissimilar to those I use for long train rides and delayed flights. I watch and write narrative in my head.

I am happy to say that the story of review panels is a happy one. Despite a well-developed penchant for cynicism, I have found grant review panels actually accomplish their tasks with almost the idealistic aplomb one might hope for. This is by no means a Nicholas Sparks novel with clean moral progression and sentimental tidy outcomes, but rather more a Kazuo Ishiguro story with enough complex ambiguity to render it believable. How can 10 to 25 people of disparate background and expertise, with only limited preparation and exact knowledge, decide on the future of scientific research project funding in a 20-minute discussion? It is done the same way that a memorable high school prom is planned and executed: the power of small group process.

The first challenge of grant review is the scoring algorithms. The NIH has changed this quite recently from a 1-5 scale with decimals to a 1-9 scale with no decimals. Not surprisingly, changing the scale has not done a great deal for how people map their evaluations to the metric. Some struggle mightily to develop their own algorithms to allocate points and retain measurement fidelity across all applications. Others lead with their gut and are quite idiosyncratic from application to application. The adjudication of these mapping processes happens in those airless hotel conference rooms when the applications are discussed and scored. People engage in brinkmanship, acquiescence, passionate articulate speechifying, and occasionally embarrassing backtracking. The consequence, however, is a scoring mechanism for any given application is constructed through a rough form of consensus building. Obviously most of the debate occurs in slicing up those on the margins. Everyone in these rooms recognizes the Elvis on Velvet and the Picasso of applications. Everything else is much harder.

The second challenge is maintaining the democratic process in light of the many and varied egos and rivalries. Small group process is often derailed by a single strong personality. Grant review panels are no different, but the NIH has embedded some mechanisms to shape the process and the panels themselves enforce others. First, there is always a big alpha dog who acts as chair. The panel officers at NIH know their dogs well and usually choose those who understand the value of careful, graceful, and humorous guidance rather than bullying. I have fallen in love with many a chair over a two-day period as he or she gently guides the group to its destination. Second, for the unsolicited grant proposals NIH receives continuously, there are standing panels that have members who stay on in rotating cohorts for two to four years. The combination of familiarity and new blood expedites the process dramatically. It means that the first half-morning is not spent in the painful process of taking everyone’s measure. By the third or fourth meeting, the habits, humor, and disposition of your fellow cohort members are well-known to you and everyone else, so you spend less time thinking about process and much more time thinking about science. And because there are overlapping cohorts, the old guys train and discipline the young ones.  It is usually gently done particularly for those who have not served on a panel before and are over-eager or simply nervous. 

The famously contentious nature of academics is only rarely on display in these meetings because it is tough to remain in the room once you have been openly ugly and particularly hard if you have to return to a similar room four months later. Finally, the groups themselves retain an almost puritanical sense of justice and fairness that is quite surprising given that these panels usually consist of people with some mileage on them. I have seen panels collectively and clearly redirect any member who tries to subvert due process. In a panel not so long ago, a young gentleman who had been in a lifelong headlock with another researcher tried to scuttle an application simply because of this ongoing animosity. The outrage in the room was both palpable and sharp. He was, indeed, redirected.

These scientific grant panels are not some gloriously functional happy family by any means, but the careful structuring and dynamics of well-intentioned human beings in groups actually has the intended outcome.  The decisions are on average correct and mostly just. There are, of course, built-in biases toward larger universities and established researchers, which some misguided champion of the underdog like me will point out with passion on a regular basis. Mistakes are occasionally made because of oversight, hubris, or a lack of expertise. But surprisingly and delightfully, it usually works. Prom night does finally come magically lit and complete with the best band ever, and most people go away happy.