Misjudgment Calls

Responsible academics have long attempted to discredit the positivistic data generated by IQ tests, variously demonstrating that such instruments favor certain socioeconomic groups under the guise of objectivity, reduce the many types of intelligence into a single rating, and imply a stable position for qualities that are far more variable, even volatile. The resulting bell curves, some scholars have demonstrated, may function as handcuffs for groups that don’t tend to do well. Yet analogs to the oversimplified and unyielding judgments of ability generated by those IQ tests are alive and well in the academy itself today. Too often, in situations ranging from a tenure decision to our expressed or internalized responses to a student paper, we impose firm and final rankings on academic aptitude rather than making a nuanced or provisional evaluation.

Can we generalize about situations ranging from marking a sophomore’s paper in the privacy of one’s office to participating in a meeting on a tenure decision? Clearly issues, stakes, and political implications may differ. The recurrence of certain problems and practices in situations across that spectrum, however, permits — even encourages — certain broad generalizations. At the same time, since some of these issues are field-specific, I am addressing the humanities, and particularly my own discipline, literary and cultural studies. And since the issue of how racial and gendered prejudices can contaminate judgments on intellect has been discussed extensively elsewhere, this essay devotes comparatively less attention to  those issues.

Obviously, many types of judgment are necessary and valuable in such fields and in our universities as a whole; I have repeatedly — though by no means invariably — been impressed with the dedication, expertise, and care colleagues have brought to these responsibilities. And I am not now nor have I ever been a member of the parties opposing tenure, not least because I do not think that move would resolve the disgraceful reliance on adjuncts. But we need to acknowledge and negotiate the problems attending the way we evaluate academic ability.

One such problem is premature judgment. For example, deciding on the basis of a single paper that someone is not likely to be a good student throughout the semester or throughout his or her career is problematic for many reasons. In general the teacher should try to suspend that judgment, or, if it must be made, both bracket it with caveats and gradually buttress or modify it with additional evidence. As the literary historian Avrom Fleishman effectively argues in The Condition of English: Literary Studies in a Changing Culture, evaluations that may be appropriate for a particular example of or even a body of work all too often slide into more definitive overall judgments on the person creating it. Often a firm evaluation of the quality of the work at hand may well be entirely sound; a prognostication of future work feasible though risky, and a judgment on immutable qualities of mind deleterious.

The issue Fleishman identifies is especially risky when judgments are made on whether something or someone is “smart.” As Jeffrey Williams persuasively demonstrated in the minnesota review, the replacement of “solid” with “smart” as a term of praise marks an increasing delight in the startling or counterintuitive argument. The ability to generate such points in a single piece of work may indeed demonstrate the intelligence of its author from some perspectives. But again, doing so begs the question of whether those abilities will be sustained and whether they are adequate predictions in themselves of strong scholarship or criticism.

Moreover, should one privilege one version of intelligence over others? The emphasis on multiple types of intelligence in the work of the cognitive scientist Howard Gardner is an important caveat to making judgments  of intellectual ability.

I vividly remember that after one of my early IQ tests I heard that I had puzzled teachers because I had done very well elsewhere but missed an apparently simple question. I still remember struggling with it: given a picture of a doll and gloves in three different sizes, we were asked in so many words which gloves would fit “this little doll.” I knew that one set of gloves looked right for the doll, but hearing the word “little” made me erroneously decide that the gloves that were best described as “little” were the correct answer. This mistake prefigured both the unusual verbal skills and indifferent visual and spatial abilities that have characterized my cognitive performances to this day — but since it was simply counted as an error, it also demonstrates the problems of measuring intelligence as a monolithic category.

Problems in the concept of “smart” as well as in other criteria for professional judgments are crystallized by the lecture-style presentation that is so important in hiring at many institutions. What are we measuring, and how effectively? Teaching abilities, some would assert. But such presentations at best reveal only a few of the  many skills involved in effective teaching and in fact often serve as an excuse for not assessing other skills, especially at the sort of institution that gives only lip service to the importance of undergraduate education. Are we judging research through these presentations? Yes, and up to a point fair enough. But we risk devoting undue weight to impressions generated by job talks: a careful and protracted assessment of written material is typically both more time consuming (sometimes unfeasibly so) and more valuable.

Yet even faculty members who have reviewed that material sometimes allow their prior judgments on it to be subsumed or virtually forgotten, giving undue weight to the lecture that should instead be evaluated in close conjunction with earlier reading. What all that suggests is that  often we are above all judging perceived smartness — or the performance of it — through job talks, and even judging if the candidate displays (flaunts?) precisely the putative markers of smartness we have ourselves, or to which we may aspire. The Q&A, itself unduly weighted in many decisions, also reflects performance and polish — and at its worst invites judgments based on whether one approves of the answer to one’s own question.

Even if one does decide that smartness in its customary senses of rapidly producing a startling insight is the sine qua non for and best measure of academic ability, or if one assigns that role to other dimensions of intelligence, we certainly risk not measuring them accurately, whether in job talks or many other situations. As noted above, the academy has recognized although not invariably curtailed the impact of racial, ethnic, and gendered stereotypes on judgments of academic ability, but many other prejudices may come into play as well. One of the top graduate students I ever taught told me that she had worked sedulously to discard her Southern accent, correctly perceiving that listeners in other regions might be less likely to take her seriously.

For all the consciousness of class and social status in literary and cultural criticism, in our own personnel decisions we too often interpret as signs of mental prowess mannerisms and behaviors that may well result instead from upper-middle-class breeding. Both verbal facility and refined social assurance, frequently though of course not invariably encouraged more in families from the more elite socioeconomic groups, may convey an impression of smartness. (Notice that “smart” is the very term used for elegant clothing.)

More broadly, some members of the profession will be less likely to identify intelligence in someone with an unpolished social manner — though on the other hand others are more likely to expect smartness there. (Another race in which I have a horse, though one emphatically not ready to be put out to pasture: aren’t colleagues more likely to describe people their own age, rather than significantly older, through these and related positive epithets?) As these instances suggest, both judgments on “smartness” as well as other monolithic overall evaluations may screen other, less savory evaluations, whether or not the person making them is aware of that.

Moreover, as the attacks on IQ tests also revealed, intelligence is far from the “ever-fixèd mark” that Shakespeare associates with love in one of his sonnets (116.5). Pressures of all types may temporarily block its components, notably memory; shortly after my father’s unexpected death, I repeatedly had trouble remembering the number for my ATM card, which I readily recalled before and after that event. People in the humanities may well grow and develop in many ways, not only at the stages of their undergraduate and graduate work but often considerably later in their careers. Often switching to a more congenial specialty or critical methodology produces such growth; its predecessor, less compatible with the interests and abilities of the person in question, may well have been encouraged or even dictated by a mentor or the perceived direction of the field. For such reasons, many people who composed an indifferent first or even second book do much better work later on; those who evaluate them throughout their careers on the basis of their early work, followed by a cursory familiarity with later writing or none at all, risk making unfair judgments.

Even if we do calibrate our scales to arrive at more accurate measures for academic aptitude and abilities, those categories may downplay one characteristic necessary for success: the drive that encourages intense and sustained work. Indeed, certain conceptions of intelligence dismiss that type of work as plodding , instead celebrating explicitly or implicitly a concept related to the Renaissance belief in sprezzatura: according to this model, the truly gifted will, as it were, rapidly and effortlessly  turn out impressive academic work with their left hand, the right hand perhaps holding a crystal glass of, say, Meursault or another premier French burgundy (reminding us again of the implicit role of class in some judgments). But in fact, as anyone who has followed the career of graduate students over the years knows, the difference between a strong career and a disappointed and disappointing one typically involves not only talent and a sadly and increasingly large component of sheer luck. The recently publicized work by Angela Duckworth, a psychologist at the University of Pennsylvania, has demonstrated the effectiveness of what she terms “grit,” a conclusion that may variously to reinforce and to temper judgments made on other grounds.

The prices paid for the mistakes chronicled above are all too evident. Even if the teacher attempts to be tactful, both undergraduate and graduate students sense judgments; whether or not their perceptions are completely correct, thinking one has been classified as second-rate can too readily become a self-fulfilling prophecy. Above all, when the pie is as small as it is in the academy today, we must work to distribute it as fairly and judiciously as possible

How, then, can we avoid such errors, given that academic judgments are so often necessary and even desirable? We need to remain vigilant about the likelihood of mistakes, remembering, for example, that much as opponents of straw votes point out that they tend to solidify what should be tentative positions; the same danger shadows preliminary judgments on a student or colleague. We need to examine why we ourselves may be tempted into deceived and deceiving judgments. In particular, might we find it hard to challenge standards and procedures of judgment that have aided our own professional advancement?