Anastasiia_New/Istock/Getty Images Plus
Perhaps no aspect of publishing evokes as much anxiety as a revise and resubmit decision on a manuscript. Yet despite that, pitifully few resources are available to help academics manage the challenges of responding to reviewer criticisms.
The limited resources that do exist tend to focus on procedural approaches: create a spreadsheet of reviewer comments, start with small critiques and work toward larger ones, draft a response letter, and resubmit. Few articles address the complex challenges likely to crop up -- such as feelings of self-doubt or how to respond when a review comment is biased or incorrect. What should you do when faced with an overzealous reviewer nudging your work in a direction in which it was never intended to go?
In this article, I draw on 15 years' experience -- as a research development professional, National Science Foundation review panelist and journal reviewer -- to provide 10 pieces of advice and insight for crafting responsive revisions while also keeping your voice and remaining true to your basic intent.
No. 1: Be open to improvements. I start with this recommendation because sometimes even the most well-placed comment can trigger the impulse to argue with your reviewer. But it’s important to take time to process those emotions and not discount the opportunities for improvement that your reviewers suggest. That means taking a moment to skim through feedback and then putting your manuscript away for several days or weeks. Not everything has to be done immediately, and writers rarely make their best decisions when emotions run high.
After taking a break, I encourage you to take a critical approach to organizing reviewer feedback. Rather than treating that feedback as a comprehensive checklist of what needs to be done to your manuscript, strategically organize it by section: introduction, literature review, methods, results, discussion and conclusions. Group consistent feedback into themes. Organize from small and easy to larger and more difficult. Finally, make a list of the outliers.
No. 2: Reviewers need hand holding. Over my career, I’ve provided in-house review services for outgoing grant proposals, been a panelist for both university and federal grants, served as an editorial board member, and reviewed articles in my own field of expertise. Though my processes differ for each role, my point of departure remains the same: “How do we make this work publishable?”
I begin as an advocate for any project I review and deploy all the tools the writer provides to help ensure the project is published. Nevertheless, I am often called to review articles that relate only tangentially to my own area of expertise. Thus, I need a writer to walk me through a study’s theoretical grounding clearly and quickly so that I can spot the novelty of their work and identify key stakeholders.
In other words, if you receive an unfavorable review, look to areas where your reviewer might have gotten lost -- or where you didn’t clearly answer important questions about methods, significance or research contribution for the educated nonexpert without insider knowledge of your field or method.
No. 3: Reviewers often expect two rounds of submission. When faced with a critique, it is easy to succumb to feelings of self-doubt, interpreting all criticism as evidence of underperformance. Yet as a reviewer, part of my job is to engage critically with a project -- or to push the boundaries of the study I am reviewing. My goal is to ask the researcher one more time, before this is finalized, “Is this your best work?”
I almost never accept a paper outright in the first round. I generally move between accept with minor revisions and accept with major revisions. The review can be bumped up (to accept) or down (to reject), but I generally work in the middle. In fact, the exact same manuscript may get an accept with minor revisions on a first submission and an outright accept on the second, just because I know that I and one or two other expert reviewers have already vetted the manuscript.
I generally think of the first submission as an author’s dress rehearsal: How does the work play with a smaller audience? What does that audience feel could be explored more in order to respond to the needs of the field? Once the manuscript has been through that round of review, with edits made in response, the author is ready for opening night.
No. 4: Reviewers often feel on display. Once a journal review is complete and editors have made their final decision, reviewers will often get a follow-up email. The email thanks reviewers for their service and includes both the final decision on manuscript acceptance as well as comments from the other reviewers. When those emails arrive in my in-box, I always check my review against the reviews of my colleagues: What did the editor decide, and did the other reviewer(s) come to a similar conclusion? In essence, did I do a good job on my review?
I feel the same way when reviewing grants. I read every proposal, but did I read closely enough? Did I miss any key elements? Did I engage sufficiently with the background, justification and approach?
As a reviewer, it often feels less risky to critique research than to enthusiastically endorse it. Therefore, if you confront a particularly critical reviewer, begin by consolidating all the reviews. Identify suggestions made by multiple reviewers, and then critically evaluate the quality of the rest of the comments. Identify comments made by an overzealous reviewer that do not strengthen your work and talk to your editor about them. Editors can help you to parse out which critiques you can address through clearer explanation and which you can just ignore.
No. 5: Break big suggestions down into manageable steps. Reading through manuscript or grant feedback can be a daunting task -- not only because of the sheer volume of comments that three peer reviewers can generate, but also because of the way in which they deliver those comments. Very minor comments regarding missing citations or typos are listed side by side with overwhelming criticisms likely to provoke stress-induced chafing -- such as the theoretical framing was simplistic and uncompelling or the sample size was small and not representative of the population as a whole.
My advice for those of you coping with broad, overwhelming comments on your manuscript is to translate them into a series of actionable items. Recently, when working with a researcher who had received comments on their literature review, we came up with a series of actionable items designed to address this specific comment: 1) locate two recently published literature reviews on the subject, 2) create a list of topics discussed in the reviews but neglected in the manuscript, 3) identify which of those topics should be included in the paper review, 4) re-outline the literature review section and 5) revise the literature review intro and include two new paragraphs to cover the new topics. While we replaced one comment with five, those five steps were immensely more manageable than a single insurmountable-feeling comment.
No. 6: Recruit help translating feedback. Struggling researchers tend to fall into one of two groups. In the first group are those who follow directions to the letter. Faced with the grant proposal feedback like, “Your sample of five interview respondents appears small,” they will revise with a sample size of six. The second group of struggling researchers are those who interpret all feedback as evidence of some larger, underlying problem with the research. When confronting the same comment, they will throw out the entire qualitative section and start a completely new project using survey data.
Research development professionals, colleagues and mentors can give you invaluable advice in addressing reviewer comments. They are generally a step removed from a grant proposal or manuscript and can take a holistic look at feedback. That means looking for broader themes or continuity across reviewers’ comments.
For example, in grant proposals, the methods sections can be easy to critique. But addressing only the methodological quibbles may keep you from revisiting the degree to which your proposal established the social or scientific intrigue of your work. Other times, the reviewers just want six interview subjects.
No. 7: Organize your response letters section by section. National Institutes of Health revisions allow for a single-page introduction succinctly discussing how you’ve incorporated feedback from the first-round proposal review. Researchers often struggle to capture the scope of their changes in such restricted space, and that is where a strategic grouping of reviewer feedback can be quite useful. Rather than an exhaustive point-by-point list of changes, try to capture the spirit of review comments and broad changes made moving section by section or theme by theme.
This letter format can also be useful in addressing manuscript revisions. If reviewers indicate that your paper’s theoretical framing fails to capture all key relationships between variables, that the discussion of previous research is undercited and that numerous sentences lack relevant citations, it may not be worth three separate notes in a response letter. The message is clear: revisit and justify your theoretical model. If allowed, a robust paragraph indicating how you’ve overhauled the theoretical model may convey far more persuasively than individual comments about the degree to which you’ve made changes.
No. 8: Evaluate (and then re-evaluate) your journal selection. In both grant proposals and in paper reviews, I often recommend that researchers put considerable thought into how and where to target their work. Estimates suggest that well over half of grant proposals are rejected on the basis of something other than science. Likewise, many journals have a host of nonmerit criteria for article selection, including most obviously subject or topic as well as disciplinary or theoretical perspective, methodological approach and target audience. Some even adhere to a single-blind review process, meaning that the reputation of the author or grant writer can be used to sway a reviewer.
For many of us, having articles published in “top” or well-respected journals in our field carries a particular cachet. Unfortunately, those journals can be mired in tradition and brokered by gatekeepers resistant to disciplinary evolution. Getting a foot in the door with a revise and resubmit decision can cause authors to make broad structural changes, temper messaging or shift focus -- only to ultimately receive a rejection with weeks, months or even years lost in revision. When reviewing journal comments, read for comments likely to take you away from your original goal and ask, “Is this comment likely to result in a better paper -- or just a different one?”
No. 9: Watch for superstition. With a foot in the door, it becomes easy for writers to wring their hands, thinking, “What if I change something that reviewers liked?”
This superstitious thinking can undermine responsive revisions. Instead of considering how best to engage with reviewer feedback, writers spend a disproportionate amount of time with a scalpel, surgically removing as little of the original manuscript as possible.
Rather than this surgical approach to revisions, view your manuscript as a single image, where cohesion -- or the fit between pieces -- is more important than any one feature on its own. Reviewers, panelists and editors often respond most positively to engagement and exploration.
No. 10: It’s OK to stand by your work. Reviewers are fallible. They also have the heavy burden of reviewing considerable material from a broad range of fields, often under the auspices of providing service to their discipline. The result is that their feedback can be rushed or even incorrect. For example, in an empirical study I submitted a number of years ago, one of my reviewers commented that because my study was qualitative I should refrain from using the word “data” when referring to my quotes and transcriptions. This comment was inconsistent with my training as a researcher.
Operationalizing variables, creating indexes to describe events, developing methods to organize or synthesize data: these are all evidence of problem solving. There may be other ways to resolve the problem, but that does not necessarily mean that they are better ways to address the issue you want to address. When reading over review comments, if you’re surprised by a comment that is either outmoded or incorrect, do some additional research but ultimately advocate for your work. That can include both a note in the response letter that accompanies your manuscript revision or, preferably, corresponding with a journal editor via email to ask their advice.
In the example above when I messaged the journal editor, they told me that this type of information on reviewers was quite valuable, as it helped them keep track of the level of advice their journal reviewers provided and where their strengths might (or might not) lie.
Receiving a review and resubmit decision on a manuscript can be exciting. It means that reviewers and editors saw the potential in your work and are willing to continue discussing it. Unfortunately, being inundated with literally pages of feedback on a manuscript can also feel like facing a firing squad. But by consolidating, managing emotional responses and feelings of self-doubt, pruning irrelevant or incorrect comments, and involving your editors, you can more effectively engage with feedback to produce a more responsive, clearer and better version of your work.