Let's Review

As many of you recall, one of the first professional tasks you undertook was to write a scholarly review of a book, article, exhibition, symposium, performance, research breakthrough, or new discovery. What we academics now do as a matter of course is use a skill we ought to teach our undergraduate and graduate students: critical engagement with secondary sources. In many respects, teaching students to think critically about work that has already been done is one of the more pragmatic things we can impart. Academics move in critical circles shaped by debate, conferences, and publications, but most professions, at some point, ask their charges to review and assess. A legal brief is only as good as the applicable precedent and case law it rests upon; a good market analysis must evaluate pre-identified trends and projections; computer programmers stay current by poring over the latest papers in their field; and executive summaries of published materials are the lifeblood of successful businesses.

Preparing students to think reflectively and critically is an arduous task, but not an impossible one. It begins with breaking students of the instant gratification they think they get from Wikipedia and the Internet. We must, literally, slow down their minds. In my estimation there is no point whatsoever in railing against Wikipedia and the Web. I’ve had colleagues who’ve told me — with a great degree of righteousness — that they forbid students to use these. Hah! Students will use these; they just won’t cite them. All such prohibitions do is promote a form of plagiarism.

The good news is that we can teach students to evaluate any sources they use, even if it is Wikipedia. A good way to do this is teach students how to review, make them write reviews, and be tough on grading those reviews. I do this through a mnemonic method I call the 4Z method, so-called because the penultimate letter in each word is a Z: summarize, analyze, criticize, and synthesize. I first explain what I mean by each of those terms, and then distribute reviews of various sorts: serious movie and music reviews, book critiques, a review of new research, a product evaluation, etc. In small groups students discuss where they find the elements of each Z. This helps them see that a review need not be written in a mechanistic fashion; it also helps them see that economy of language and selectivity are paramount. Of the four, the hardest to explain is criticism, a word whose default position suggests something negative. It takes some effort to get students to see that criticism is an evaluation that can also be positive.

The next step is to make a class review something important in their field, either a book or a journal article. I do lay down the law on this one — no Web sites. That comes next! Students must use the 4Z method to review their source and I impose a strict word limit (generally either 500 or 750 words, depending on the level of the class). Roughly half of the class will get their reviews back with instructions to do them again. (Despite my best efforts about 25 percent will be lightweight and too much like the thumbs-up/down movie reviews on TV; most of the remainder haven’t read closely enough, have written sloppy papers, or haven’t broken out of the high school book report routine of summarizing without analyzing.) I am the most lenient on evaluating how students synthesize; after all, one can’t expect them to have mastered the field at this stage of their careers. (In research seminars I teach how to write review essays, thereby engaging them in rudimentary synthesis.)

Once most of my class has gotten the hang of reviewing, as a follow-up assignment I send students go back to the Web and have them evaluate a site. It would be naïve of me to think they will bring a critical eye to all future surfing, but if I drive home the point that professionalism involves critical thinking, I’ll take it. (I’ve not done this yet, but later this semester I’ll experiment with having students turn their analytical guns on Wikipedia entries.)

Here’s what I explain. I use this material as talking points and then post the summary on course Web sites so students can reference it.

--Summarize

• What is this about? Write no more than one paragraph.
• What is the problem or issue the author is addressing?
• Give only highlights. For the most part, details will come in the analysis sections.

--Analyze

• What is the author’s main argument?
• What, if anything, is new about the argument? What is the author adding to our field of knowledge?*
• What evidence does the author use to make the argument?
• Illustrate the analysis by referencing an example or two from the author’s study that shows how s/he attacks the issue at hand. Give a brief “flavor” of how the author makes his/her case.

--Criticize

• Is the author’s argument convincing? Is it well-reasoned and rooted in evidence that meets the standards of the discipline?**
• Can the author’s conclusions be tested? Be suspicious of arguments that are based entirely in rhetoric.
• Does the author cite sources? Be wary of studies that do not.
• Is this work controversial? If so, which conclusions seem convincing? Are there any ‘red flags’ that suggest that skepticism is warranted?

--Synthesize***

• How does this work compare with others in the field? Where does it fit in the pantheon of works on this subject?
• How do other experts regard the work?

* Tips on Assessing the Importance of a Study:

A study is probably important if:

1. It says something that goes against what scholars commonly think about a subject.

2. It elaborates upon what is known about a subject. It adds depth to what is already known.

3. It mines new sources or applications that were unavailable to earlier scholars.

4. It takes a new or unusual approach to the subject. This might mean such things as applying theory to the subject, looking at it from a different point of view, testing old assumptions in a new way, updating an old study, or retesting older hypotheses.

5. It doesn’t have to “reinvent the wheel.” A study can be important even if it doesn’t do anything particularly new. It can be considered as important if, among other things:

• a. It is well-written and makes the subject accessible.
• b. It is well-organized and likely to be used by students and scholars within a field.
• c. It brings an old study up-to-date for modern audiences.
• d. It adds details to an existing study that were not very well known.
• e. Uses a larger test group than previous studies.

** What is “Evidence?”

The standards of evidence differ by academic discipline and you need to familiarize yourself with how scholars in your field define it. Consider the following:

1. Scholars tend to be distrustful of arguments whose "proof" lies entirely in words. Language that is merely forceful or clever is seldom enough to carry a debate. Academics seldom accept anything that might fall into a "because I said so" category.

2. Some disciplines do rely more heavily on words and theory — moral philosophy and/or abstract mathematics, for example — but they nonetheless hold to rigid standards such as structured logic, plausible theorems, and reasonable analogies.

3. Among the most obvious forms of evidence are measurable data, examples, and documents. A "you-could-look-it-up" standard is a high one. So too is any argument whose proofs can be tested independently or whose experiments can be replicated.

4. Arguments based in faith or ideologies do not meet standards of academic evidence. This is not meant to pass judgment on these things, simply to acknowledge that they are not scholarly works. A writer can reference beliefs to describe, but not to prove. When they attempt to do the latter, they are stating preferences, not making a scholarly argument. (For example, is it really possible to prove that the Democratic Party is better than the Republican Party; or that Buddhism is superior to Judaism?)

*** Prerequisites of Good Synthesizing:

This is the hardest thing to do. It presupposes that:

1. You have read widely in your field.

2. That you know which works are considered authoritative and what they say.