How to Evaluate Academic Research

Recently, the value of academic research, especially in the humanities and social sciences, has been questioned. The current majority party in the House of Representatives has proposed cutting science funding for social science research and eliminating all funding for the National Endowment for the Humanities and the National Endowment for the Arts. New York Times columnist Nicholas Kristof accused faculty of engaging in specialized research disconnected from the interests of the reading public and policymakers, resulting in a broad conversation about whether or not faculty engage in the public sphere.

There is no doubt that academics have a responsibility to engage public debates. In many ways, the university is the critical conscience of a democratic society. It houses experts in various spheres of life who must use their knowledge to enhance the public’s understanding of vital issues. Academic freedom ensures that scholars use their knowledge to inform questions of public importance.

Yet academic research’s value cannot and should not be measured, as some seem to suggest, simply by how many readers a journal article or academic monograph reaches. The purpose of academic scholarship is to engage in disciplinary inquiry — to further scholarly conversations. Such work will never be accessible to the general public since it, by definition, works at the boundaries of knowledge and takes a certain amount of prior knowledge and expertise for granted.

This is not something new. It was recognized by the ancient Roman statesman Cicero in his work The Ideal Orator. Cicero distinguished between oratory, in which one speaks to and with the public, and philosophical inquiry, in which one seeks truth.

Because orators wish to speak to the public, “the procedures of oratory lie within everyone’s reach, and are concerned with everyday experience and with human nature and speech.” That is simply not possible when engaged in scholarly inquiry, however, since scholarship “draws as a rule upon abstruse and hidden sources.” To Cicero, this meant that in inquiry, “the highest achievement is precisely that which is most remote from what the uninitiated can understand and perceive, whereas in oratory it is the worst possible fault to deviate from the ordinary mode of speaking and the generally accepted way of looking at things.”

Because academic research is specialized and takes place on the boundaries of what is known, it requires a community of experts. Disciplines form the communities of inquiry that enable academic research to take place. These communities require a critical mass of scholars to evaluate new work and to develop new knowledge.

As a result, even academics cannot read the work of all academics. As a historian, for example, I rarely read and cannot truly understand publications in academic medical journals, nor journals in chemistry or physics or other fields in which I lack the necessary specialized knowledge and engagement with the academic literature. I never doubt the value of these journals, however, for encouraging new ideas and practices in medicine, or chemistry, or physics. Whether in the natural and physical sciences or the humanities and social sciences, most academic work is by definition inaccessible to the uninitiated.

This is not a bad thing. In the discipline of history, for example, I would worry if the primary criterion for the importance of a piece of academic scholarship were the number of nonacademic readers. As Paula A. Michaels recently put it, if academic history relied primarily on popular readers, most history (and the history of most people) would never get written.

Again, this is true not just for the humanities or social sciences, as is often implied in public conversations about academic research. Public funding for basic research in the sciences is vital to promote the public good, but scientific research is increasingly being privatized, subject to commercial interests and philanthropists.

Basic research in all fields is vital for innovation. The value of a particular piece of scholarship — whether measured in the number of readers, the impact on knowledge, or new products — cannot be known a priori. Every academic researcher is an entrepreneur, every book or article a startup.

Most small businesses fail, but that does not lead Americans to question the value of entrepreneurship. They should have the same attitude for scholarship: most articles may not have a dramatic impact on the field or the public, but we cannot know which ones will. That’s why we need to encourage startups and encourage creativity throughout the academy, just as we do in the economy.

The measure of success also has to be related to the “markets” that academic research serves. In the case of medical research, it is fair to ask whether academic research produces new knowledge that improves medical outcomes.

In the case of such fields as history, then, we must ask whether academic research informs history’s practitioners. The primary places where history is practiced, of course, is in schools. We must ask whether the curriculum offered in history courses at all levels — from elementary school to graduate school — is shaped by scholarship. Similarly, we should ask whether museums and historical societies offer interpretations influenced by academic research? If so, the case is made: historical outcomes have been improved, just as good medical research improves medicine’s outcomes.

What is true for doctors and historians and other academics is also true for ministers. Few churchgoers read specialized theological journals, but the theological questions explored in these journals affect how ministers engage in their daily work and the ideas that they use to inform that work. No one would doubt that ministers, like doctors, need to have a sophisticated understanding of their theology, even if much of it may be inaccessible to the broader religious public.

None of this is to suggest that academics do not have a responsibility to reach out to the public. Fortunately, we have gifted writers within and beyond the academy to do this. First, as Ezra Klein pointed out, journalists mediate between the academic world and readers. Of equal importance, as Klein noted, over the past few decades, academic publishers have locked up research behind expensive paywalls that serve the interests of publishing companies rather than the public. Published research must and should be made available to all who seek it.

There are also academics who translate academic knowledge for the reading public. They do so through blogs, op-eds, magazine articles, and media appearances, but also through books. Almost every discipline can name many scholars whose books have reached a wide audience. These scholars represent what Cicero referred to as “the ideal orator,” capable of uniting, in Cicero’s words, “wisdom and eloquence.”

Moreover, as teachers, almost all faculty members must strive to be ideal orators, combining wisdom with effective teaching in order to reach out to students in ways that help students make the connection between academic inquiry and broader public and personal questions.

In sum, academic research’s value cannot be measured by simple metrics about the number of readers. We must accept that the very nature of scholarship, whether in the humanities or the sciences, whether in medicine or religion, requires an expert, and therefore a limited, community of inquiry. Most academic work will always remain inaccessible to outsiders.

All faculty members have the responsibility of being ideal orators in the classroom, but the very nature of scholarly inquiry means that most cannot and should not seek to do so in their published research. We must celebrate those scholars who can and wish to combine wisdom with eloquence, however, for they ensure that academic conversations will enter the public sphere. At the same time we must always remember that the only reason ideal orators — whether in the classroom or the public sphere — have something worth sharing beyond the academy is because of the specialized research taking place within the academy.

A more sophisticated understanding of the value of academic research, and especially basic research in all fields, would help us recognize not only the contribution scholarship makes to the public good but also how it does so.