Wednesday
August 27, 2008
Advertisement
Advertisement
News, Views and Careers for All of Higher Education
Feb. 21
Assessment Changes Everything
When I served on college admissions committees in the 1990s, a phrase that kept coming up was “the best students,” in comments like “We’ve got to get the best students” or “Rival College X down the road is beating us out for the best students.” I came to think of the mentality behind these comments as the Best-Student Fetish, a symptom of the increasingly obsessive competition among colleges for the cream of the high school senior crop. The more I thought about the Best-Student Fetish, the more perverse its logic seemed: It is as if the ultimate dream of college admissions is to recruit a student body that is already so well educated that it hardly needs any instruction! Sitting in admissions committee meetings, it was all I could do not to ask, “Hey, why don’t we recruit bad students and see if we can actually teach them something?”
Related stories
- Early Decision — for Faculty Jobs, Aug. 1
- Buying Its Way Onto the Program?, May 2
- ‘All the Sad Young Literary Men’, April 30
- Voltaire Wasn’t Cut Out to Be an Iowa State TA, April 29
- French Theory, April 16
The experience helped me realize that, despite our undoubtedly sincere efforts to make higher education democratic, the top colleges and universities and their wannabe imitators are still set up for the students who are already the best educated rather than for the struggling majority that needs us most. Perhaps we got so used to the split between intellectual haves and have-nots among undergraduates that we concluded that it’s inevitable and there’s nothing we can do about it. This would explain why, in the hundreds of faculty meetings I must have attended in my 40-plus years of teaching, I have never heard anyone ask how our department or college was doing at educating all its students.
That’s why I’ve become a believer in the potential of learning outcomes assessment, which challenges the elitism of the Best-Student Fetish by asking us to articulate what we expect our students to learn — all of them, not just the high-achieving few — and then holds us accountable for helping them learn it. Whereas the Best-Student Fetish asks who the great students are before we see them, outcomes assessment changes the question to what students can do as a result of seeing us.
Furthermore, once we start asking whether our students are learning what we want them to learn, we realize pretty quickly that making this happen is necessarily a team effort, requiring us to think about our teaching not in isolation but in relation to that of our colleagues. The problem is not that we don’t value good teaching, as our critics still often charge, but that we often share our culture’s romanticized picture of teaching as a virtuoso performance by soloists, as seen in films like Dead Poets Society, Dangerous Minds, and Freedom Writers. According to this individualist conception of teaching — call it the Great-Teacher Fetish, the counterpart of the Best-Student Fetish — good education simply equals good teaching. This equation is pervasive in current discussions of school reform, where it is taken as a given that the main factor in improving schooling is recruiting more good teachers.
In fact, this way of thinking is a recipe for bad education. According to Richard F. Elmore’s research on primary and secondary education, in failing schools the governing philosophy is often, Find the most talented teachers and liberate them “from the bonds of bureaucracy,” which are often seen as infringements on academic freedom. (In the movies, the great teacher always works her classroom magic against the background of an inept, venal, or corrupt school bureaucracy.) Elmore reports that the pattern of teachers “working in isolated classrooms” is common in unsuccessful schools, where everything depends on the teachers’ individual talents “with little guidance or support from the organizations that surround them.” Conversely, as Elmore argues, successful schools tend to stress cooperation among teachers over individual teaching brilliance, though cooperation itself enhances individual teaching.
For all its obvious value, excellent teaching in itself doesn’t guarantee good education. The courses taken in a semester by a high school or college student may all be wonderfully well taught by whatever criterion we want to use, but if the content of the courses is unrelated or contradictory, the educational effect can be incoherence and confusion. As students in today’s intellectually diverse university go from course to course, they are inevitably exposed to starkly mixed messages. Though this exposure is often energizing for the high achievers who possess some already developed skill at synthesizing clashing ideas and turning them into coherent conversations, the struggling majority typically resort to giving successive instructors whatever they seem to want even if it is contradictory. Giving instructors what they want (assuming students can figure out what that is) replaces internalizing the norms of the intellectual community — that is, education.
The freedom that is granted us in higher education (at least at high-end and middle-rank institutions) to teach our courses as we please should have always carried an obligation to correlate and align our courses to prevent students from being bombarded with confusing disjunctions and mixed messages. Outcomes assessment holds us to that obligation by making us operate not as classroom divas and prima donnas but as team players who collaborate with our colleagues to produce a genuine program. We all use the P-word glibly, as in “our writing program” or “our literature program,” but we have not earned the right to the word if it denotes only a collection of isolated courses, however individually excellent each may be.
By bringing us out from behind the walls of our classrooms, outcomes assessment deprivatizes teaching, making it not only less of a solo performance but more of a public activity. To be sure, with such increased public visibility may come greater vulnerability: Though it is students whose learning is evaluated in outcomes assessment, it is ultimately the faculty whose performance is put in the spotlight. If we have nothing to hide, however, then less secrecy and greater transparency in our classroom practices should work in our favor. At a time when attracting greater financial support for higher education increasingly depends on our ability to demonstrate the value of our work to wider publics, anything that makes teaching more visible and less of a black box figures to be in our interest. Giving teaching a more public face should help humanists doing cutting-edge work refute the widespread stereotype of them as tenured radicals who rule over their classes with iron fists. But it should also help humanists more generally to clarify to a wider public the critical reading and thinking competencies we stand for and to show that those competencies are indispensable enough to the workplace and democratic citizenship to merit greater investment.
But of course the critics of outcomes assessment are far less sanguine than I am in the face of the conservative politics they see driving it. In a talk delivered at our Modern Language Association “Outcomes Assessment” session, Michael Bennett, presenting what he called “the radical take on learning outcomes assessment,” said this position “can be summarized in one word: resist!” Bennett argued that the push for outcomes assessment must be seen in the context of the increasing privatization of higher education, the co-optation of accreditation by the for-profit educational sector, and the attempt to force colleges to accept a version of the No Child Left Behind law in the schools. As Bennett put it:
“I see the focus on outcomes assessment as a dodge from the real problems with the American educational system: that it is embedded in an inequitable and violent socioeconomic system. The kind of policies that would truly help the students with whom I work are not more hearings, campus visits, and testing but adequate funding for secondary education; child care; a living wage; debt relief or, better yet, free universal postsecondary education; an adequately compensated academic workforce exercising free inquiry and building an educational community; and universal health care.”
Bennett is certainly right that many of the problems of American education — including the so-called achievement gap between students from rich and poor backgrounds — are rooted in economic inequality and that more adequate funding and social services would do much to alleviate these problems. But to see outcomes assessment as merely a conservative dodge designed to distract everyone from structural inequality ignores the ways our own pedagogical and curricular practices contribute to the achievement gap. Though it calls itself “radical,” this view is remarkably complacent in its suggestion that nothing in our house needs to change.
Though Bennett and other critics believe that assessment is an invention of recent conservatives that is being imposed on education from the outside, the truth is that assessment originated from within the educational community itself in the early 1990s, well before conservative efforts to co-opt it. I recall attending my first assessment conference in 1991 and noting the considerable buzz about assessment at meetings of organizations like the Association of American Colleges and Universities. The original motivations of assessment lie in legitimate progressive efforts to reform higher education from within, by judging colleges according to what their students learn rather than by their elite pedigrees.
But outcomes assessment can be used in undemocratic ways, and educators do need to take Bennett’s concerns seriously. We should scrutinize the standards used in assessment, how these standards are determined and applied (and with what degree of input from faculties), and how assessment results are used. Rather than reject assessment and circle the wagons, however, we should actively involve ourselves in the process, not only to shape and direct it as much as possible but to avoid ceding it by default to those who would misuse it. Had we been assessing outcomes all along in the normal course of our work, I doubt that the legislators and privatizers could have rushed in to fill the vacuum we created.
As David Bartholomae observes, “We make a huge mistake if we don’t try to articulate more publicly what it is we value in intellectual work. We do this routinely for our students — so it should not be difficult to find the language we need to speak to parents and legislators.” If we do not try to find that public language but argue instead that we are not accountable to those parents and legislators, we will only confirm what our cynical detractors say about us, that our real aim is to keep the secrets of our intellectual club to ourselves. By asking us to spell out those secrets and measuring our success in opening them to all, outcomes assessment helps make democratic education a reality.
Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.
Advertisement
Comments
Graff’s right
A little anecdotal evidence to back up Graff’s argument. In our history department at a large Western land-grant university, we were dragged kicking and screaming into doing outcomes assessment. We started as simply as possible, assessing just two learning outcomes using two essay-exam responses as our instruments. What we found surprised us. No, it didn’t surprise us that our students performed rather badly at some of our outcomes. It did surprise us that the entire assessment process (especially the measuring) led us to the richest, most intellectually engaging, and most useful faculty discussions we’ve ever had about teaching and student learning. I actually look forward to our assessment measurement day (it takes six of us faculty about 5 hours) each semester and the talk about what we might do to improve. Each of us has changed the way she/he teaches, and we will probably change our major in response to what we’ve found in assessment. And overall the frequency and quality of our talk about teaching and learning is enormously enhanced; there’s a buzz on about teaching and learning. We thought we were great teachers before (and we were), but assessment has helped us teach together. Three years ago I never thought I would have said this, but our “culture of teaching and assessment” is much improved. I am certain this wouldn’t have happened without assessment.
Richard, at 9:35 am EST on February 21, 2008
Why resist assessment? Why embrace it?
I agree that there are some good reasons to resist assessment of learning outcomes, and some reasons for humanists to embrace it.
Two additional reasons for resistance: first, because assessment is always about increasing and guiding control, so it’s reasonable to ask control over what by whom, and make sure that the increased control is used in the interest of the students.
Second, many humanists resist assessment because they assume (or are told) that assessment begins with the description of learning goals, and the more precise their goals the better. In contrast, their idea of good teaching may be more like that of a colleague of mine, Willi Unsoeld, at The Evergreen State College in the mid 1970s. Willi (http://wilderdom.com/Unsoeld.htm) didn’t mind if his students became taxi drivers, if they were better, more humane, engaged human beings. How does one state that goal as a learning outcome? How measure it with a test? Confronted with that definition of assessment, many humanists resist.
In fact, this kind of goal doesn’t invalidate assessment. But it does suggest rethinking it. For example, assessment in the humanities ought to focus more on the learning process. And it ought to include strategies such as essays, projects, and portfolios that are more flexible ways of assessing student development and achievement, strategies that don’t require imposition of the same, behavioral goals on all students.
For more on “What Outcomes Assessment Misses,” see this paper from AAHE: http://www.tltgroup.org/programs/outcomes.html
Steve, Vice President at The TLT Group, at 9:55 am EST on February 21, 2008
A view from the trenches
As a faculty member who has been involved in assessment at the university level, I have several comments. First, Prof. Graff’s article makes no mention of the faculty labor involved in assessment—and it is considerable. While some members of the public may think otherwise, faculty already have a full plate of research, teaching and service. Assessment takes away—a significant amount—from our efforts in these areas. Second, faculty are experts in history, fine arts, anthropology, etc. They are not experts in educational assessment; hence the results are likely to be amateurish and unreliable, especially when universities require assessment but provide no additional resources for it (see above, under the first point). Third, even within disciplines approaches and emphases are so diverse that I’m not sure the team approach Prof. Graff emphasizes is workable. By the same token, this diversity makes a single assessment standard difficult. I believe the best we can do is, as Prof. Graff once said, “teach the conflicts,” and urge our students to think through them. Finally, I am not sure that assessment is great for public relations. While it might help to explain what we do, I think it also lends support to the belief that faculty are unproductive and need some kind of regulation. It also lessons the authority of the faculty.
HA, at 10:05 am EST on February 21, 2008
But HA...
... shouldn’t assessment be part and parcel of being an educator? It shouldn’t be a separate enterprise altogether, but something that’s embedded within the doctoral training programs in the various disciplines. Still, the biggest issue here is the way we train our faculty. The focus is still intensely on research rather than teaching, and so while we may have brilliant researchers on our campuses, we have horrible, ill-equipped teachers. Few PhD programs require any kind of knowledge or experience in curriculum development, teaching, or assessment.
Steve, at 11:15 am EST on February 21, 2008
Gerald Graff’s Essay
Gerald Graff’s argument on behalf of outcome assessment makes my blood boil. He offers a grotesque definition of education as “internalizing the norms of the intellectual community”—something I’d say more aptly sums up Maoist indoctrination. He argues that we ought to protect college students from “being bombarded with confusing disjunctions and mixed messages,” apparently oblivious to the fact that college is indeed a time of life when students are supposed to learn to handle the contradictory ideas that are at the heart of Western culture.
As to his idea that we need “less secrecy and greater transparency in our classroom practices,” I don’t know what prison he works in. Where I teach, there is nothing secret about what goes on in our classrooms. Just because teaching doesn’t take place in a public square doesn’t mean that what goes on in a classroom is a secret. Teaching and learning are both transparent: In my department, we have course syllabi (containing clear explanations for grades), peer observations, administrative evaluations, student evaluations, and a reasonably close tracking of our students’ activities and employment after college. Professor Graff’s argument is that we should now embrace outcomes assessment—i.e., heap on the jargon of outcomes assessment so that we can be clearer (for legislators and parents) that we’re meeting our “learning objectives” and “closing the loop”—or whatever.
Finally, his idea that teaching ought to be made “less of a solo performance and more of a public activity” is just plain sloppy thinking. Since when is “solo” an antonym for “public”? Excellent teachers are excellent precisely because they’re going solo, and the sorry part is that much of their excellence cannot be learned by weaker teachers, even when they try their hardest to emulate them.
Professor Graff’s essay finishes with the idea that we need to find a “public language” with which “to speak to parents and legislators” (i.e., he wants outcomes assessment blather). He couples this with the unsubstantiated idea that the “public” sees college professors as somehow protecting “secrets.” Who is this public, other than some disgruntled legislators, who get a lot of publicity for education-bashing, and where is the empirical evidence that we’re seen in this light?
Perhaps Professor Graff suffers from guilt over what’s happened to his own field—English—during the past several decades. After all, he’s witnessed the destruction of the study of the putative subject at hand in favor of theory. That’s a sad thing, and to the extent that he participated in it, he ought to feel guilty. For the rest of us who didn’t destroy our subjects, however, Outcomes Assessment is a wretched thing.
If you’re up for reading a longer, more carefully thought out attack on outcomes assessment, please see my article on the subject, “A Pedagogical Straightjacket,” in the 7 June 2008 Chronicle Review
http://chronicle.com/weekly/v53/i40/40b00601.htm
Laurie Fendrich, Professor of Fine Arts at Hofstra University, at 11:40 am EST on February 21, 2008
Faculty and assessment
Certainly faculty members should assess their own teaching. Many already do. Within a program there should also be discussion of student knowledge and abilities coming out of particular courses and completing the program. Many departments do this as well.
It’s the idea of reporting the results to the administration that faculty find threatening.
Faculty are concerned that outcomes assessment will end up like student course evaluations in many schools. Feedback comes too late to help the faculty member and the administration uses what is frequently a popularity contest to punish faculty.
Faculty person, at 4:45 pm EST on February 21, 2008
Please tell me....
What is wrong with teaching college faculty “how” to teach (and acknowledging that good teaching incorporates the effective use of assessment?) I acknowledge that you are an expert in your field, but that does not mean that you know how to teach!
What is wrong with encouraging faculty to work together to guide students towards program goals?
What is wrong with opening the classroom door and actually making faculty responsible for their actions?
What is wrong with treating education more like a business? How do you begin your journey if you are unsure of your destination? Which direction do you head first? Of course your destination may change based on what you learn along the way, but if you don’t have a destination you are just treading water.
What is your goal? How are you going to move towards your goal? How will you know if you reach your goal? What will it look when you are there? And isn’t your goal, as a professional educator, to open intellectual doors for your students? How do you know you are doing it effectively if you don’t conscientiously assess?
Why is it that measurable outcomes seem to be offensive only to faculty who appear to place no value in the overall program objectives?
Are the majority of faculty opposed to outcomes assessment so arrogant that they think their courses are untouchable, that no one knows as much as they do “so how dare you try to impose any assessment regime?” Or are you just threatened by the mere thought of someone taking a closer look at what goes on behind your classroom door?
You may not like the fact that institutions must report to governing agencies about their effectiveness, but that goodness that you choose this field rather than virtually any other field that requires practitioners to actually produce something measureable. Can you name another profession where someone is not measured on their primary activity, has a job for life once tenured (in most cases regardless of performance) and has the tremendous level of responsibility that we entrust to those in higher education? Well, can you?
Institutional Researcher, at 5:45 pm EST on February 21, 2008
Institutional Researcher’s Comments
Dear Institutional Researcher:
Before I would ever go down the rabbit whole of answering the multiple hackneyed questions you ask, I have one question for you: WHO, exactly, assesses YOUR outcome?
Laurie Fendrich, Professor of Fine Arts at Hofstra University, at 7:50 pm EST on February 21, 2008
Institutional Researcher
Sorry. I wrote “whole” instead of “hole.” That’s what a glass of wine will do to your spelling. LF
Laurie Fendrich, Professor of Fine Arts at Hofstra University, at 8:00 pm EST on February 21, 2008
Assessing Assessment
“Please tell me…,” he requested, and so I do:
What is wrong with teaching college faculty “how” to teach (and acknowledging that good teaching incorporates the effective use of assessment?)Nothing, if the faculty gets to do it themselves according to the standards of their fields and not on the Procrustean bed of silly, educratic language and charts drawn up by academic bean-counters Alverno College.
What is wrong with encouraging faculty to work together to guide students towards program goals?Depends on the program, depends on the goals.
What is wrong with opening the classroom door and actually making faculty responsible for their actions?Well, the tone of your question, for one thing. “[M]aking faculty responsible for their actions” has a vibe of assumed criminality, or at the least, accusation of it. And students might be the ones most bothered by monitors peering through that open door.
What is wrong with treating education more like a business? Higher education, I presume you mean. The arguments against for-profit grammar schools are fairly irrefutable. Anyway, what’s right with kids getting baccalaureates from an organization whose primary pupose is profit margins and its stock price?
How do you begin your journey if you are unsure of your destination? With the first step, in the direction of David Carradine.
Why is it that measurable outcomes seem to be offensive only to faculty who appear to place no value in the overall program objectives?Because those “overall program objectives” are that stuff cited in my first answer, above.
Are the majority of faculty opposed to outcomes assessment so arrogant that they think their courses are untouchable, that no one knows as much as they do “so how dare you try to impose any assessment regime?” Or are you just threatened by the mere thought of someone taking a closer look at what goes on behind your classroom door?Again (and, thank goodness, finally) the prosecutorial tone. Nonetheless, the answers are “no” (faculty in opposition are not “arrogant”), and “no” as long that someone isn’t one of the the “academic bean-counters” I mentioned in my first answer.
Can you name another profession where someone is not measured on their primary activity, has a job for life once tenured (in most cases regardless of performance) and has the tremendous level of responsibility that we entrust to those in higher education? Well, can you?Sure: men of the cloth, a whole lot of civil servants, parts of the military, parts of the public education establishment. Actually, those fields are plagued by far more entrenched incompetence than is the case among tenured college professors. Rather remarkable, when you think about it.
Now I have a question: What’s an “Institutional Researcher”?
Peter Plagens, at 9:00 pm EST on February 21, 2008
If we can take a momentary break from the never-to-be resolved faculty/administrator wars, it seems to me that what Gerald Graff is talking about in this piece (and in his other works) is students. Are students best served by fragmented faculty fiefdoms and departmental course offerings that have no rhyme or reason? I don’t know. But I do know that in nearly ten years as a faculty member no one has ever asked this question in my department. Never. Not once. For a group of intellectually curious people, this strikes me as strange.
cacambo, at 12:40 am EST on February 22, 2008
Control
Good professors, I believe, have always thought hard about how they do their work and how they might go about doing it better. My view of the assessment fad is that it conceals (barely) yet another effort to control faculty, particularly those in the humanities who are seen as a threat to such politicians as, for example, the ex-governor of Texas, for whom advanced mental activity is indeed a fearsome prospect.
Control freaks are micro-managers, and they have good reason to fear Socratic investigation of the truth. Academic administrators try to smother the truth with paperwork, and to the extent we allow them to do so we are ourselves complicit in the destruction of our profession.
I suggest prudence,compassion, and intellectual rigor as principles on which to continue our activity in this profession, and I suggest that we judge the credibility of administrators by the extent to which they themselves are willing to be assessed—publicly—by their faculty.
Hnaef, at 9:15 am EST on February 22, 2008
Money, mouths and evidence
Thank you to Laurie Fendrich, Peter Plagens and Hnaef for their responses, with which I agree. To Steve I want to ask: do you think university administrations will readjust faculty rewards away from research and toward teaching and assessment of teaching? If not, is it fair to ask faculty to move in a direction that their university is not. The problem with fans of assessment, is that they think the university is a business—except when it come to the compensation of faculty labor. Then were “educators” who should do this stuff as a vocation, even if it means less rewards as our research falters.
More disturbing, though, is the premise of faculty incompetence that runs through posts like Steve’s, Institutional Researcher, or cacambo. Without any evidence, Steve thinks that good researchers make poor teachers (in my experience, at any rate, it’s the opposite), Institutional Researcher thinks because faculty don’t answer his or her particular questions they are directionless, and cacambo likewise assumes that programs of study are without rhyme or reason.
Of course, posts such as Laurie Fendrich’s or Peter Plagens’ continually note the way that faculty have always assessed themselves and thought through their mission. And I can say from my own experience that, long before the assessment fad, my department—and college—has taken its mission seriously and put a lot of thought into it—including our teaching and curriculum. The objection is to having to do everything twice, once in away meaningful to the faculty, and then again in a form made up from afar by “Institutional Researcher.” So before the latter and his or her ilk make condescending remarks about the failure of the faculty, I’d like to see some evidence for that failure.
HA, at 12:05 pm EST on February 22, 2008
PS
I see that Cacambo does offer evidence that his department has never discussed its course offerings. That sounds to me like a problem for his department. It is not my general experience that most departments function in this way. I confess I don’t have evidence based on anything more than my experience, but I’m not calling for a massive shift in goals and resources.
HA, at 12:25 pm EST on February 22, 2008
“Virtuoso” Teaching
This is an impressive essay, not so much for Dr. Graff’s comments on outcomes assessment as for his articulation of good teaching. It does seem that some faculty members engage in a “virtuoso performance,” declaiming for the best students, with the others viewed as a bit of a nuisance.
That may be why outcomes assessment is resisted — faculty members rated as among the best may not be all that effective in coaxing students into better thinking and a desire for knowledge.
Jane S. Shaw, Executive Vice President at John William Pope Center for Higher Education Policy, at 1:25 pm EST on February 23, 2008
No Unfunded Mandates—Unless We Mandate Them
I note that the last comment—positive about assessment—comes from someone who works for the The John William Pope Center, a higher education policy institute that tilts right (see, e.g., its academic advisory committee). Conservatives hate unfunded mandates, but seem to like this new emphasis on assessment (Margaret Spellings, for example), which, at least at my university, is a big unfunded mandate. There is significant added work for the faculty with almost no additional resources for it.
HA, at 9:05 pm EST on February 24, 2008
My colleagues resist assessmentnas strenuously as they resist discussions of what it is we do as a program; why, “we do our own things.”
It is an irresponsibility born of alienation (colleagues who not without reason feel they’re not being paid enough to care) and, I think, fear (of having to admit that one has not kept up with one’s own teaching fields, let alone the changes in the profession as a whole).
If any of us were asked, “What is it that your program accomplish?", or “What do you expect of its majors?", no one could answer ... which is not to say that they wouldn’t anyway.
minority of one, at 2:25 pm EST on February 27, 2008
Will the Real Outcomes Assessment Please Stand Up?
It’s hard to see the words “outcomes assessment” without immediately thinking, “No Child Left Behind all over again!” This equation is either explicit or taken for granted in many of the critical responses to my essay posted on mla.org, insidehighereducation.com (where my article also appeared), and the MLA Radical Caucus Web site. My friend and former colleague Barbara Foley unequivocally equates college assessment with NCLB, stating that “OA is basically No Child Left Behind as applied to higher education. Its implications are profoundly anti-intellectual, conformist, and conservative” (mla.org; 21 Feb. 2008).
In some cases Foley may be right. Some college assessment does seem to resemble NCLB practices in focusing on narrow factual information and skills that can be quantitatively measured by multiple-choice tests. Outcomes assessment can be dominated by narrow criteria that end up not clarifying the academic world for students but instead confusing them further or giving them a misleadingly narrow picture of that world. And some top-down administrative dictation does deprive faculty members and students of input into the formulation of assessment criteria, creating a coercive situation similar to NCLB-style “teaching to the test,” especially for the growing ranks of contingent faculty members who now represent a majority of the faculty on many college campuses.
But at its best, outcomes assessment is a very different and much more useful process, one that stresses broad critical thinking and argumentative competencies and bears little resemblance to NCLB. In a response to my essay on the MLA Radical Caucus Web site, Steven Thomas describes his department’s meeting and reaching agreement that it wants its “students to be able to fairly summarize someone else’s argument and critically evaluate it without making ad hominem attacks. . . ” ( 20 Feb. 2008). Though Steven is ultimately unsure where he stands on outcomes assessment, he observes that in his experience the process “doesn’t force a teacher to teach one kind of content vs. another, but it does identify certain skills that are important.” For example, it forces “a teacher to actually help the students correctly read an argument and learn how to compare and contrast it with other arguments.” Nothing could be further here from the top-down dictation of criteria, the testing only of information and skills that can be quantified, and the disregard of students’ higher-order intellectual prowess that characterize No Child Left Behind.
In a similar vein, consider this account from Richard, a history professor “at a large western land-grant university,” posted on insidehighered.com (21 Feb. 2008):
In our history department. . . we were dragged kicking and screaming into doing outcomes assessment. We started as simply as possible, assessing just two learning outcomes using two essay-exam responses as our instruments. What we found surprised us. No, it didn’t surprise us that our students performed rather badly at some of our outcomes. It did surprise us that the entire outcomes assessment process (especially the measuring) led us to the richest, most intellectually engaging, and most useful faculty discussions we’ve ever had about teaching and student learning. I actually look forward to our assessment measurement day (it takes six of us faculty about 5 hours) each semester and the talk about what we might do to improve. Each of us has changed the way she/he teaches, and we will probably change our major in response to what we’ve found in assessment.
Again, the contrast with NCLB could hardly be more dramatic. As in Steven’s account, assessment here is not about downsizing and policing but about instructors working together to align their criteria and thus give students a better shot at figuring us out — not one at a time by giving each of us what we “want” even when it’s contradictory, but by actually entering our intellectual community. In a faculty culture that’s often demoralized, you just don’t hear professors every day say things like, “the assessment process ... led us to the richest, most intellectually engaging, and most useful faculty discussions we’ve ever had about teaching and student learning.”
The stories Steven and Richard tell echo the encouraging conclusions about assessment drawn by the rhetoric and composition scholar Peter Elbow in a wonderful essay published in 1986 on “competence-based education” — a precursor of current outcomes assessment — in his book Embracing Contraries. (I learned from Elbow that I was mistaken in dating assessment as first emerging in the early 1990s.) After visiting a number of outcomes-oriented college programs, Elbow noted that teachers in such programs tend to be enthusiastic about the outcomes approach because “it helps them to teach better and thereby to feel more sense of accomplishment. It both forces and helps teachers to figure out more clearly what they have been trying to do, to become more aware of their latent assumptions and premises, and often to go through this process collaboratively with colleagues. The result tends to be teaching that is more intentional, effective, and energized” (139).
It seems obvious from these clashing accounts that college outcomes assessment means significantly different things at different campuses, and this applies as well to the economic and political consequences of assessment. Even the best pedagogical principles can be appropriated in disastrous and oppressive ways, and the dangers of such misappropriation (as I conceded in my essay) become all the greater in today’s climate of cost cutting and government supervision of education. But I don’t think it follows, as its critics too glibly assume, that assessment is inevitably compromised and corrupted by today’s social and economic context.
In any case, if the debate over assessment and its political uses and misuses is to advance, we need to determine what outcomes assessment is in the first place—otherwise we may not be talking about the same thing. After all, the conflicting pictures of assessment drawn by the critics of NCLB-style assessment on the one hand and by Steven and Richard on the other may both be true, depending on the institutions we are looking at. If this is so, then the debate over assessment is likely to be advanced not by generalized pronouncements (whether mine or my critics’) about student learning, economic contexts, and the intersection of the two but by close examinations of how assessment is actually playing out, pedagogically and economically, at particular campuses. The favorable view of assessment that I gave in my essay was based on the many positive accounts I have heard from faculty members and administrators who have been involved in assessment on their campuses, positive accounts that far outnumber the negative ones by my admittedly unscientific reckoning. Obviously, we need to move beyond such anecdotal evidence, but I’m not aware of any comprehensive survey that would help settle the question of the extent to which assessment programs match the negative or positive images.
If I am right that college assessment at its best — as Elbow, Steven, and Richard have pictured it — is profoundly different in spirit from NCLB, it might just provide a model for a more humane and educationally intelligent assessment practice in the schools, not least by shifting the emphasis from rote memorization of information to engagement with arguments and ideas. If instituted at the lower grade levels, the focus Steven describes on helping students read and analyze arguments and engage with those who think differently from themselves would figure to improve primary and secondary school education and bridge the chasm between the intellectual worlds of school and college.
A final point. In questioning my defense of the faculty collaboration that assessment promotes, some respondents object that college faculties harbor so many disagreements that it seems unlikely they could work together, much less agree on common assessment criteria. This reasoning leads some respondents to wonder whatever happened to the writer who used to exhort colleges to “teach the conflicts.” There is no contradiction, however, if you accept my premise that college faculties contain not only deep disagreements but also common rhetorical and argumentative practices for negotiating those disagreements, practices like, to quote Steven one last time, “correctly read[ing] an argument and learn[ing] how to compare and contrast it with other arguments.” In fact, my critics in this exchange could not have even begun to disagree with me without relying on this shared practice of summarizing arguments and comparing them with other arguments. But again — and here’s the point — we instructors get so used to these empowering argumentative practices that we take them for granted, but many of our students will never learn those practices unless we are more up front about them, not just in one or two courses but across the entire curriculum. And this kind of transparency is precisely what outcomes assessment is all about.
Gerald Graff, 2008 President of MLA and Professor at U. of Illinois at Chicago, at 1:35 pm EST on March 1, 2008
Graff is Right!
I am continually astounded by the many ways in which the assessment debate can cause ego-driven explosions of ideologies and antipathies that have little to do with the subject at hand. To me it all seems quite simple, in principle at least. Most of us are part of a noble enterprise, a university or a college. What is the purpose of that enterprise? What is its product? I would argue that it is simply “learning.” That is what students do in our classrooms and that is what our faculties do in their scholarship and research. If one accepts that idea, then is it not reasonable to expect that an education institution should attempt to ascertain whether its product actually exists, and, if it does, to discover something about its quality and quantity?
I think the answer is obvious. Of course, properly assessing learning is as challenging a task as acquiring the learning itself, and we’re not yet very good at it. But that difficulty should not be used as an excuse to evade our responsibility to learn whether we actually do produce a product.
Don Langenberg, Chancellor Emeritus at University System of Maryland, at 10:00 am EST on March 3, 2008
Don Langenberg is also right
Thanks to Don Langenberg for his comment. I have been following the comments on this article and have found them to be very interesting. Clearly the issue of learning outcomes assessment is an emotional one for many faculty. At Capella University, we are engaging faculty in discussions and decisions about learning outcomes. This has resulted in great discussion, general ownership and support for assessing learning outcomes, and for discussion about learning and teaching in general. See http://theother85%.com for additional consideration of this topic and others.
Mike Offerman, Capella University, at 11:15 am EST on March 3, 2008
What’s NCLB got to do with it?
I know that one academic rarely says “outcomes assessment” without another academic’s responding with dire talk about the failure of the Bush Administration’s NCLB program. NCLB is higher ed’s version of “NAFTA” as a code word for “disaster.”
And I’m not a NCLB booster. I’ve worked enough with K-12 teachers to know that NCLB has indeed been a bad program. But it’s not been bad because it asks that teachers think about what they would like students to learn and then asks them to find responsible and reviewable ways to assess what students actually have learned. Rather, it’s been bad because it has 1. often (but not in fact always) set trivializing goals, 2. often (but not in every case) created intrusive and hollow assessment measures, 3. almost always failed to provide the resources to be responsive to even the intelligent findings its assessment programs have generated, and 4. in the funding measures it has established has created a remarkably backwards system in which those schools with least resources can be punished for poor performance by loss of resources.
All those things are true, and it’s a partial list. But again, those reasons are not reasons for higher ed faculty to evade questions about how best to set goals and to assess whether one has met them—not even slightly.
I’ll grant that NCLB-like measures might someday be proposed for us, and thereby become a real enemy rather than the boogieman they are now. But the best way to ensure they aren’t and don’t is to engage this matter honestly and openly ourselves. Graff’s article does that. As one who has already been benefiting from several years of doing this with my own classes, and finding myself a happier and more successful teacher for having done so, I agree completely with Graff’s notion that an intelligent and carefully articulated set of outcomes can make a big difference to the way we teach and work, and I thank him for venturing into this conflict with so careful and so moderately argued a piece.
John Webster, at 3:30 pm EST on March 3, 2008
History and Institutional Research
The association of assessment in the minds of so many faculty respondents with NCLB shows their ignorance of the history of their profession. For the most part, the profession of a professor is teaching, and the history of teaching in the last 25 years, at least, includes assessment. I am surprised at how many think it’s a new thing. Many faculty appear to be actually hostile to learning about their profession, regarding any theories about learning, learning styles or the development of cognitive abilities as irrelevant to their work or at worst the inventions of an intrusive administration. This ignorance harms students who do not learn just as the faculty member did when in college.
The remark of the respondent who asked “what is institutional research” also reveals a regrettable disdain for anything else that is going on in the institution besides his classroom, not to speak of the hostile question by Laurie Fendrich as to who assesses our outcomes. Our outcomes are primarily management information, and we are continually working to improve our delivery of what people (department chairs and up, as a rule, but sometimes individual faculty or staff) need to know to make decisions. We should conduct periodic needs assessments to see if we are meeting the information needs of the community. Information sometimes comes in the form of test results, which is how some of us get into assessment. I myself am directly involved in supporting faculty assessment of student outcomes, and they are doing pretty well, thank you, and have for many years. This aspect of my job is assessed by a faculty committee as well as by my boss.
Grocheio, Asst VP Planning and Institutional Effectiveness at Shorter College, at 4:40 pm EST on March 3, 2008
The Assessment of Gerald Graff
The reservation I have about outcomes assessment is that it will inevitably “assess” those things that are most easily assessed. . . which often means things that are trivial. For a writing course, it is fairly easy to assess grammar errors or spelling or punctuation or T-units or sentence variety. It is difficult (and expensive) to assess the quality of thinking, the effectiveness of language, the sophistication of argument. I fear an increased emphasis on assessment will mean an increased emphasis on the trivial over the profound.
Peter Adams, Professor at Community College Baltimore County, at 6:35 pm EST on March 3, 2008
Re Grocheio
First, I asked “what is an institutional researcher?” not “what is institutional research?” The difference: the former is how a respondent described himself/herself occupationally, the latter is a general activity. Sometimes, it’s curious to see how the latter is elevated (if that’s the word) to the former, e.g. “meeting attendance” to “meeting attender,” or whatever into “Assistant Vice-President of...Institutional Effectiveness.”
Second, I don’t hold in disdain anything that goes on in the instution outside my classroom. I hold in disdain only those educratic bean-counters from outside the field who a) want to assess what I do by means of silly little charts and approved lists of verbs ("the student will...[fill in the blank]"), and b) contribute to the endless profusion of administrative positions (which is the one thing administrators do best: create other positions, usually with “vice-” and “assistant” in the title) that weigh down higher education and deplete resources that would be better spent elsewhere.
Third, an undeniable political element has crept into the whole “outcomes assessment” business. Conservative politicians and trustees, who’ve long thought that insufficiently supervised faculty constitute a seething mass of Ward Churchills, want to use OA as a club with which to beat liberal professors. Which is to say, as soon as Margaret Spellings goes away, maybe OA can regain some credibility.
Finally, anybody is welcome to assess how I teach by asking anybody anything they want—students, ex-students, colleagues, superiors, professionals out in the field where my students have gone. I just do not wish to have to check little boxes and line up long vertical columns of approved verbs relative to “strategies,” “goals,” “objectives,” and other educratic fetishes.
Kids, can you spell a-p-p-a-r-a-t-c-h-i-k?
Peter Plagens, at 9:10 am EST on March 4, 2008
Show me the outcomes, the money and the data
For Professor Graff and the other advocates of assessment in this discussion, I have three questions.
1) The advocates of assessment do not seem to have understood what the critics of it in this discussion have been repeatedly saying: the choice is not between assessment vs. no assessment. Faculty, departments and universities already have mechanisms for assessment. In my department we have regular meetings at which we discuss goals (aka outcomes) and the curriculum, we have teaching and salary evaluation, and, of course, we have externally mandated assessment on the old 7-10 year schedule—not the new perpetual one. So from now on, can we all promise to refer (as I will from here on out) not to assessment, but to a “new layer of assessment,” which is really what we’re talking about. So question #1, how do the old forms of assessment articulate with the new layer of assessment? What is the value added to this new layer? Why was the old assessment insufficient? (Okay, I cheated; there are multiple subquestions here, but you get the point). Since advocates of the new layer of assessment like to talk about clear statements of outcomes, I would think these questions would be paramount.
2) No advocate of the new layer of assessment in this thread has yet talked about resources. But as someone involved in the new layer of assessment, I can tell you that the labor is significant. So do the advocates of the new layer of assessment also advocate reducing expectations for faculty research, or the amount of time spent preparing classes, or at least paying faculty stipends for new work (in business, don’t people who work extra get bonuses?). Let me add that our assessment office has just enough staff to assign tasks and provide some data. They do not help us with the actual work, nor do they provide any material resources to conduct our department’s new layer of assessment.
3) As Prof. Graff says, there’s no systematic studies of the benefits of this new layer of assessment. I would then ask Prof. Graff and other advocates of this new layer of assessment why they don’t instead advocate doing such study *first,* rather than trying to promote a national movement. Such study seems especially important given that faculty are being asked to do this labor without compensation, and to the detriment of their genuine research and teaching pursuits. We don’t introduce drugs on a wide scale without studying their efficacy and side effects. If the advocates of this new layer of assessment really take it seriously, shouldn’t they want to offer more than anecdotes in defense of their position?
BTW, on the point about common goals. Any goals that common—e.g. don’t argue using ad hominem attacks—hardly require faculty time to discuss. As Prof. Graff acknowledges, we’ve internalized these common goals, and any teacher worth his or her salt already teaches them.
Apologies for the long post.
HA, at 10:20 am EST on March 4, 2008
“Asessment” v. evaluation
It may be valuable to distinguish between “assessment” and “evaluation"—the latter has to do with measuring against a standard to determine success or failure to some degree. Assessment asks “what is,” not necessarily “is this good enough?” The two can be linked, of course, but the latter is more like the NCLB-type assessment. The former is the kind that Graff—and long before, Elbow and others—has been talking about. The latter, I care little about but will fill in the forms as the bureaucrats send them to me. The former, I care passionately about since I direct a writing “program” (loosely called) with 30 sections of English 101 every quarter (and other writing classes), but I can’t guarantee that a student will experience even nearly the same thing in section A as in section Q. Without assessment—the kind that evokes invigorating, enriched discussion of what is happening in our classes—our instructors, overworked and underpaid, simply can’t know what they’re students are supposed to be doing in any concrete way (though in the abstract, they might know that they’re students are supposed to develop “complex theses,” or something); thus, they can’t adjust their teaching. Assessment, for our program, is faculty driven and is all about energizing faculty to see what they’re doing as teachers so that they can do it better to serve students. I fill out the top-down “assessment” forms for the deans but they’re not really “assessment” but rather weak forms of evaluation. If we don’t as faculty together look reflectively at the work students do, we can’t be effective teachers nor create a coherent program. That’s all assessment means.
Jeffrey, Professor, English at CC in Washington State, at 6:30 pm EST on March 4, 2008
Assessment Changes Everything
I hope those who are “boiling mad” at Prof. Graff would consider the following: most antagonists seem to equate assessment with external control, not realizing that the assessment ought to be part of any learninginstitution’s regular activity, else how could we know that what we are doing is useful or effective? The key would be, then, a marriage of convenience between outside political forces demanding “accountability” on the one hand, and internal curiousity about whether or not we are doing in our classes what we think we are doing, on the other hand. Perhaps like many such marriages of convenience, love and even intellectual passion might be the result of the “attitudinal effects of mere exposure” — as someone once put it.
JVK
John V. Knapp, Professor of English at N.I.U., at 2:30 pm EST on March 6, 2008
I’m still waiting...
Prof. Knapp—like many of the other advocates of assessment in this thread—implies that assessment is something new. He writes, “most antagonists seem to equate assessment with external control, not realizing that the assessment ought to be part of any learninginstitution’s regular activity, else how could we know that what we are doing is useful or effective?”
His “antagonist” here seems most directly Laurie Fendrich (who uses the phrase “blood boil” that Knapp refers to). Now Fendrich writes, “In my department, we have course syllabi (containing clear explanations for grades), peer observations, administrative evaluations, student evaluations, and a reasonably close tracking of our students’ activities and employment after college.” One could add, and I have already, the kind of curricular discussion and planning (informed by what’s happening in the classroom) that departments do all the time.
Advocates of assessment need to detail why these kinds of assessment are insufficient, who will pay for their new layer of assessment, and how we know that this new layer of assessment will be value added (I’m sorry, but a sexy “arranged marriage” is just a metaphor). Until they do, I don’t see how the advocates of assessment can expect to be taken seriously, or not be the source of some serious boiling blood, when they keep suggesting the same false idea that departments and colleges are not already assessing—indeed the idea is not only false but also insulting.
I feel like an annoying guest at a party because I’ve posted so much on this (I know that many of you have already excused yourself to get a drink), but I promise to stop when someone who’s for assessment actually moves the debate forward by addressing these questions of 1) articulation (only Jeffrey has started this, bless him, but in a way that is not really a case for assessment as much as testimony to how much departments have always assessed themselves in ways organic to them), 2) resources and 3) evidence of effectiveness (see my previous post on them).
HA, at 8:35 pm EST on March 6, 2008
Advertisement
Jobs Related to Assessment Changes Everything
or search for jobs directly.
Assistant Professor — History of Art, Design and Visual Culture: Contemporary Art & Theory
University of Alberta
The Department of Art and Design in the Faculty of Arts at the University of Alberta invites applications for a tenure track ... see job
2008/09 Teaching Specialist or Lecturer: Geography
University of Minnesota, Twin Cities
The University of Minnesota is a premier employer and a talent magnet attracting leading faculty and staff from around the ... see job
2008/09 Teaching Specialist/Lecturer — Writing Studies
University of Minnesota, Twin Cities
The University of Minnesota is a premier employer and a talent magnet attracting leading faculty and staff from around the ... see job
Director and Professor — Center for the Humanities
University of California, San Diego
Director and Professor Center for the Humanities University of California, San Diego see job
University Based Programs Manager
University of Pennsylvania
Creates and supervise operational plans: scheduling and staffing. Manage and utilize resources, staff, and budgets. Set ... see job
Humanities (Adjunct) Instructor
Hillsborough Community College
Hillsborough Community College is a public, comprehensive multi-campus, state-supported community college located in the ... see job
Assistant Professor in Bioarchaeology
NC State University
Join the Pack! A community with nearly 8,000 faculty and staff, and 30,000 students. NC State is one of the largest employers ... see job
Reading (Adjunct) Instructor
Hillsborough Community College
Hillsborough Community College is a public, comprehensive multi-campus, state-supported community college located in the ... see job
Adjunct Faculty Credit — Humanities
Harper College
Job Description: Teach Humanities surveys
Duties of Position: Adjunct faculty ... see job
Associate/Full Professor
NC State University
Join the Pack! A community with nearly 8,000 faculty and staff, and 30,000 students. NC State is one of the largest employers ... see job
Home
| Sign up | Log in
| About us
| Work for Us
| Press Releases
| Rights and permissions
| Contact us
| Advertise
| Partners
| RSS 
| Podcasts
| Links
| Privacy
| My Account
© Copyright 2008 Inside Higher Ed
Of course
” .. I see the focus on outcomes assessment as a dodge from the real problems with the American educational system: that it is embedded in an inequitable and violent socioeconomic system ..”
Of course. It could not a monopolistic, soul-crushing educational bureaucracy, with a dysoptic sub-organizational structure of unions and tenure that make the USSR’s Kremlin seem like Apple Computer.
With legal decisions like Grigg v. Duke Power that force young people into that mindless bureaucracy to qualify for white-collar work. Even the head of the Bush Derangement Syndrome Society, George Soros, bemoans the lack of focus on educational performance.
Well, Mr. Soros has a point. Perform — or go. Even Mao, Lenin, Fidel, and Ho fancied themselves as literary performers.
And as for allegations of “socio-economic” influence — studies of correlation and causality about stable, structured family units and student performance are totally false, not requiring rigorous review and analysis. Of course.
L.L., at 7:10 am EST on February 21, 2008