Nov. 13

Outcomes Assessment: No Gain, All Pain

Something is certainly afoot. The public disclosure systems put forward by the National Association of Independent Colleges and Universities and the Voluntary System of Accountability from the country’s two big public university groups are major national initiatives encompassing some of America’s most impressive institutions. Miami Dade College’s effort to embed 10 desired learning outcomes into the curriculum, and a report of outcomes measurement by discipline, are two other accountability approaches that certainly bear watching.

Looking for a job?

See all 127 new postings

Browse all job listings:

Faculty: 2,574
Administrative: 2,177
Executive: 156
FEATURED EMPLOYERS

Even homey old IPEDS has put on a fresh coat of paint, as the Education Department’s College Opportunities Online database (COOL, née PEER) has become Navigator, with an even more attractive set of tools, in the hopes that this time a few more somebodies will use the hundreds of millions of data elements sitting patiently by.

All of this activity is more a function of the skill of Secretary of Education Margaret Spellings in moving her agenda than it is a recognition that there is merit to the numerical assessment of student outcomes.

One would have expected a brief hiatus, a quiet spell to see where this activity will lead, and whether or not it will produce outcomes useful to teaching, to learning, to higher education.

One would also have expected a great deal more caution in pressing the assessment agenda onto colleges given the experience of the last 20 years, with no outcomes to show for all the time, money and effort invested in assessment. It’s not to be. In fact, the rhetoric from Washington hasn’t let up – and is now abetted by voices urging an international assessment effort of the kind being examined by the Organization for Economic Cooperation and Development.

Assessment has virtually engulfed American higher education. Thus, many of the national conferences that should be convening America’s foremost educators to address issues such as access, the achievement gap and diversity are instead devoting numerous sessions to assessment. Most of these gatherings are devoted to “how,” rather than to “why” or “whether.” Nor is there any effort to discuss outcomes, policies, or improvements that have emerged from all of this costly and all-consuming assessment activity. All this activity, without a shred of evidence that the data we have collected — or will collect — will ever address ”national needs” or “improve institutional performance.”

The colleges, too, have been diverted. Superimposed on each institutional mission is the need to produce outcomes to someone else’s satisfaction. Successful teaching, learning, and research aren’t enough anymore. Colleges must provide evidence of an ongoing outcomes evaluation, and produce evidence leading to continuous improvement, or some such. One thing is clear: the student as a human being is no longer the sole product of the institution.

One example: a school creates a special curriculum with lots of individualized tutorial and counseling help. These are inputs, and therefore not considered in the measurement of student learning outcomes. If the school cannot demonstrate numerically that students benefit from the extra help, the whole effort is ignored by the assessment scheme, and successful (but not numerically measurable) outcomes in the students, discounted.

In essence we are in danger of distorting the very nature of our colleges and universities.

Everyone teaches to the test, and therefore the test influences the curriculum, shrinking it by encouraging an emphasis on the items to be examined numerically. This applies to colleges, and of course, to faculty members, particularly the growing number of non-tenured adjuncts who are under pressure to show evidence of successful teaching outcomes. Whether explicit or not, the need to assess focuses our emphasis on what can be counted, rather than on what counts.

Assessment, as sometimes implemented, can monopolize the time and attention of a faculty. Does anyone have any idea how many meetings, conferences, and calls are needed for this purpose? Timetables and protocols, models and alternatives, memos, e-mails and faxes, reports, agendas and minutes all within a specified format, shepherded by experts and consultants, and overseen by levels of administrators – draining away time and productive energy in an activity that heretofore has proven to be useless. This is all intended to be a permanent feature of college life, with continuous iteration of instruments and strategies following closely upon all the continuous improvements that are pouring in. [How many faculty members does it take to create a survey? Answer. We don’t know. We’re waiting for the answer to our survey.]

Putting aside the human and career costs to faculty, there is a real loss to students. Students need professors who talk to them between class, in the hallways and in offices. They need relaxed, unhurried conversations which counsel, encourage, gently challenge and explain. Some need an extra few minutes at the blackboard after class to discuss something that wasn’t perfectly clear the first time. This human element is particularly important to students who are less confident, less secure about their place in a college, and usually first generation. Paradoxically, this new assessment pressure on teachers is coming at a time when a new demographic is beginning to appear in our college classrooms.

The pain to students is likely to be much more direct, and much more widespread. The six-year graduation rate, for example, flies in the face of the need for 18 to 21 year olds to be able to grow, to change their mind about a major, to discover new interests, new opportunities, and new paths to a career. Some need to take a year off, and to mature. Some of us succeeded precisely because we had that time, and nobody was standing by with a 6-year stopwatch in hand.

The need to show strong graduation rates will ultimately determine who will be admitted to a program or school, and who will be counseled away from more challenging (STEM, among others) sequences.

Retention rate considerations will also have a corrosive effect. Schools and faculty members like to view their role as one of serving, as part of a mission. Will this image, this attitude survive in an atmosphere where retention rates determine a school’s success and inevitably its rank? With a retention mandate in place, how many faculty members will advise a student to transfer to another more challenging school, write the letter of recommendation, and call colleagues in this other school encouraging them to accept this student?

Student engagement is often important for students and exceedingly so for schools which use this characteristic as an outcome measure. Will all students be encouraged to ‘engage,’ or will schools recognize that many of their young people need to hold jobs to pay for their schooling? I have had students who came to class virtually asleep on their feet, and others who needed every available moment to keep up with the work. Are these people going to be advised to become engaged?

Undergraduate research and civic service are other measurable outcomes which may create a conflict between the interest of the school and the needs of students. Clearly, such conflicts will be inadvertent and even unconscious, but they will increasingly appear. Finally, change will not be precipitous, although faculty members who have been tapped to contribute to the assessment effort will disagree vehemently.

For the most part we have embarked on a great social science experiment with today’s students paying a price for outcomes which may, at best, emerge in a decade or so. In a global sense, we may yet find some benefit from all this effort. For the student currently within the pipeline, the assessment movement will be all loss.

Bernard Fryshman is an accreditor and a professor of physics.

Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.

Comments

Assessment Pain

At my college, the dean of instruction declared, “If it hasn’t been documented, it hasn’t happened.”

Tell that to the hundreds students with whom I’ve spent extra, undocumented hours helping them.

A great many of the students at our rural community college need more: more time to try to catch up on concepts they were never taught in high school and more patience as they work through difficult concepts. Some of them simply need someone who will listen to them for a little while as they articulate their otherwise unspeakable fears about the challenges they face.

Shall I assess my time spent with them? Refuse to meet them unless they first do a pre-test and then agree to follow up years later by filling out a survey of the results of our office time together?

Of course not. Assessment mania is a slap in the face to all of us who genuinely care about our students as people.

Jackie, Professor, at 7:40 am EST on November 13, 2007

Teaching to the test

We had this happen this fall quarter at the community college where I teach. Supposedly a “few” instructors were not doing a good job so ALL instructors (put adjunct in for all) are required to use one test put together by one fulltime instructor. I have been assessing the tests (I should say the instructor who wrote the tests did not want or see a need for this and was given limited time to write three exams) and so far each test has used material not in the textbook that the test writer brings into his classroom. Specific chapters have more questions rather than a balance between all the chapters. So what do I get told? Well, then just change it...I spent more time yesterday “fixing” than if I would have been allowed to write and use my own exam. And, today, I have to remind students to “follow the study guide” as it has the outline of what I am supposed to be teaching. Not sure this is “teaching” any more...

Jody, at 8:05 am EST on November 13, 2007

outcomes assessmentI

Outcomes assessment, as it is now practiced, is a wholesale sellout of the meaning of higher education. Until faculty and college administrators demonstrate some courage, and resist the flimsy ideas on education that are being perpetrated by mediocre educators (e.g., Margaret Spellings, whose qualifications for speaking on the subject are that she holds a B.A. and “is a mother,")and are utterly untested and unproven, we are in the position of passiver observers standing by while higher education is destroyed. I am on record with a scathing attack on outcomes assessment practices, in my essay, “A Pedagogical Straightjacket,” The Chronicle Review, in The Chronicle of Higher Education, 7 June 2007. I urge everyone to step forward and resist this movement.

Laurie Fendrich, Professor of Fine Arts at Hofstra University, at 8:20 am EST on November 13, 2007

I believe that America’s institutions of higher education will not be held directly accountable, even if we Americans could agree on what the word entails. Higher education is being held indirectly accountable. We increasingly buy products from other nations with more cost effective education systems. Recent claims that the U.S. has no engineering shortages reflect in part the high cost of our engineering graduates. Engineering graduates are often lower cost to employ overseas.

Marvin McConoughey, at 9:40 am EST on November 13, 2007

Outcomes Assessment Yields Evidence of Learning

Good grief. These comments are precisely why the public is calling for accountability. It is no longer sufficient for us to write lofty verbiage regarding the excellent programs we offer. Outcomes assessment has little to do with exit exams and everything to do with measurement of learning within curricula intentionally designed to produce student learning with the end in mind. That is, we should be able to demonstrate that our students are achieving what we say they are learning. What is wrong with a little evidence?

Have a happy day!

Cal, at 9:40 am EST on November 13, 2007

Assessment takes many forms

Not all forms of assessment are the same. If you are interested in reading about a form of assessment that is faculty-authored, that is not centered on quantitative measures, and that encourages faculty involvement with students and students’ engagement in the learning, see my recent articles “Assessment from the Ground Up” and “Encouraging Assessment from the Ground Up” in earlier additions of Inside Higher Ed. Also, while Secretary Spellings and I would have many higher education policy disagreements, I must point out as a logic teacher that attacking her level of education is an ad hominem argument and is beside the point. And by the way, sometimes it takes a little courage to advocate for assessment, too. Donna Engelmann

Donna Engelmann, Professor of Philosophy at Alverno College, at 9:40 am EST on November 13, 2007

Assessment is not new

According to Pinar, Reynolds, Slattery & Taubman (1996), “rigorous forms of evaluating educational achievement are believed to have been developed first in China by the Sui emperors 589-613 CE” (p. 796). Assessment, therefore, is not a new phenomenon that only “recently” came to play an important role in teaching and learning.

My sense is that this author’s negative reaction, and many faculty members’ negative reactions, to assessment relates more to this quote:"Superimposed on each institutional mission is the need to produce outcomes to someone else’s satisfaction”[italics added]. The negative reaction to assessment is more about the perception that external groups are exerting control and influence over educational experiences.

The author of this piece then goes on to write: “Successful teaching, learning, and research aren’t enough anymore. Colleges must provide evidence of an ongoing outcomes evaluation, and produce evidence leading to continuous improvement, or some such. One thing is clear: the student as a human being is no longer the sole product of the institution.” I must ask — how do you know you are engaged in “successful teaching, learning, and research?” Without some kind of assessment activity, how do you gather evidence to support your claim? Isn’t it reasonable that those who provide resources and support for our institutions of higher education receive some real evidence that our institutions are successful?

T-bone, at 9:40 am EST on November 13, 2007

Assess the preconditions, not the outcome

I always like using and analogy of a monastery for the academy. What if you were asked to “assess” the “outcomes” of monastic practice in a particular monastery? You might rely on “experts” such as more senior monks (preferably third-party), but ultimately I think we’d all acknowledge that the “outcome” is between the practitioner and his / her god. However, I think we’d all also agree that there are certain practices, environmental conditions, rituals etc. that are more or less conducive to spiritual development. And these CAN be measured. I am aware that the feds want standardized testing, but “stepping forward and resisting” this “movement” is kind of a dead end. I think that by measuring engagement we have a way forward. And if we improve engagement, I think the outcomes will take care of themselves.

Mark Freeman, Director of Institutional Research, at 10:20 am EST on November 13, 2007

Defining what is important

My greatest fear of the assessment movement is that test developers are put in the driver’s seat when it comes to deciding what should taught. Colleges and programs teach toward thousands of valuable “outcomes,” some cognitive, some addressing larger goals of education such as valuing lifelong learning. Which of these will be tested?

In student writing, for example, which of the following will be privileged over the others: correctness, organization, use of rhetorical strategies, persuasive argument, evocative language, voice, understanding of the topic, what has been learned through the writing process, or one of many other characteristics of good writing? Only the test developers know, and they aren’t saying because they have to protect their proprietary interests.

They do have to make some decisions about what is valued, however, whether those decisions are made intentionally after much deliberation, or whether they are based on implicit and unconsidered assumptions. At some point, underlying values will emerge in the test itself or the way scorers are trained to rate student responses.

How much do test developers know about your discipline? Could they pass an upper-level undergraduate course in your program? Are they the right people to determine the curriculum of your courses? In essence, that is what test developers do when testing is used to compare schools or for accountability.

Not all assessment works that way, of course. Assessments conducted by faculty and used by faculty to learn more about their own pedagogical practices and improve instruction can be catalysts of real and meaningful change. But let’s use measurement specialists as consultants, not drivers, and leave the policymakers who hold BA’s in political science and “are mothers” out of it.

Lee Griffin, at 10:20 am EST on November 13, 2007

frightened fryshman

Fryshman is frightened, and perplexed by the recent surge in public accountability proposals.

He is frightened because he sees it as a looming threat on the horizon; and he is perlexed because he does not fully understand the forces at play that have given rise to these collective proposals.

While I cannot address his fear, I can offer some context for understanding these diverse institutional responses to increased demands for accountability. Fryshman is twice missing the point when he opines that “[a]ll of this activity is more a function of the skill of Secretary of Education Margaret Spellings than it is a recognition that there is merit to the numerical assessment of student outcomes.”

First, Spellings is not to blame, and second, the institutional responses are not just proposals for the quantification of learning outcomes, and it is a fundamental error to assume that they are.

The diverse institutional plans that we are seeing are responses to changes in the institutional environment and its demands. Typically, and especially so in higher education, colleges and universities have great difficulty changing and adapting to meet the demands of uncertain environments.

Successful responses to these new demands are unknown, and many of these innovative responses will fail. They may, in fact, all fail. Such is the uncertainty faced by the institutions. This is what frightens Fryshman.The pressures Fryshman complains of go to the heart of the legitimacy of American higher education. It is this taken-for-granted legitimacy, that, as an accreditor, he cannot see. Whether driven by credential inflation (see link), college president’s galloping salaries, an incestuous accrediting system without public accountability, mounting student loan debt that makes starting a family after graduation a distant dream, or the overschooling in engineering and the sciences that Congress heard about this week – it is the core legitimacy that must find new ways to assert itself.

And, with the worsening of these problems (they are not going to go away), the need to address the legitimacy question will also remain, until it is properly addressed.

This accounts for the wide diversity of responses among institutions – responses that may be the first wave of a cascade of accountability measures relating to institutional legitimacy. Fryshman should be more concerned that these attempts, or at least some of them, succeed. The future environment for higher education is, of course, fundamentally uncertain.

Glen S. McGhee, Dir., at Florida Higher Education Accountability Project, at 10:20 am EST on November 13, 2007

Credentials of US Secretary

While I agree with Donna Engelmann of Alverno College on most points, I will take issue with her interpretation of challenges to the US Secretary as ad hominem attacks. They are attacks on her credentials, not her person.

Would we accept the holder of bachelor’s degree as Attorney General, Surgeon General, the head of the Federal Reserve, or the local fire inspector, for that matter? Why then, do we think that “anyone” can manage education?

I would prefer a Secretary who had at least passing familiarity with Piaget’s steps, Bloom’s taxonomy, Maslow’s hierarachy of needs, Dewey’s learning by doing, Astin’s student development model, Vygotsky’s social construction of knowledge, Cross and Angelo’s classroom assessments, Palomba and Banta’s essentials of assessment, or at least some of the above. Spellings’ initiatives do not reflect any such understandings.

I am sensitive to criticisms that educational research and scholarship has not risen to the level of rigor expected in scientific research, but educational researchers must contend with limitations imposed by the human quality of their subjects, so some allowances must made. Naming a BA in political science to head the nation’s efforts in education hardly advances the profession of education, and both reflects and maintains an attitude that in education, “anyone can do it.”

Lee Griffin, at 11:20 am EST on November 13, 2007

Re: Assess the pre-conditions, not the outcomes

Freeman, in his comment above, writes, “However, I think we’d all also agree that there are certain practices, environmental conditions, rituals etc. that are more or less conducive to spiritual development. And these CAN be measured.”

There is a some amount of circular logic used in this kind of thinking. In determining what to measure for “engagement” surveys, like the NSSE, look for research that shows the correlation between specific engagement behaviors and educational outcomes. When they find elements that have strong correlations (or at least moderate correlations) [e.g., talking with faculty member outside of class], they add them to their engagement model. But without research on learning outcomes, it is impossible to know which engagement activities lead to positive outcomes.

Even though “engagement” may be perceived to be “easier” to measure than learning outcomes, measuring “engagement” without the backing of research that assesses student learning outcomes is meaningless.

Going back to Freeman’s analogy, you cannot build a valid model of “religious engagement” without examining which practices actually lead to increased religious outcomes.

T-bone, at 12:35 pm EST on November 13, 2007

Fryshman’s Folly

Glen S. McGhee’s insightful analysis of Professor Fryshman’s paranoia, and that of his equally vexed colleagues, is helpful context, but permit me as someone totally outside your industry, but as a thrice-over consumer of your services, to add more: 1. You have my sympathy. Accountability, at least for the teaching part of your job, was largely what you were hoping to escape when you entered higher education, where outcomes used to be measured in published journal articles. Now, it looks like you’re going to be graded and paid based on both parts of your job. Has to make those “Paper Chase” reruns that stoked your fervor for academe too painful to watch anymore. If you’re looking for empathy, try commiserating with a family physician with an all-Medicare practice. 2. It’s not really your fault. This movement toward accountability is a direct result of large universities, encouraged by their benefactors, shortchanging their undergraduate programs in a desperate effort to compete as research centers. Undergraduates at large universities have become sacrificial sources of cash to cover fixed costs, while the competition rages on for grants and endowments for hot-topic graduate and professional programs. Administrators stuff undergrads in lecture halls, subject them to unqualified TAs and adjuncts, and grade-inflate them out the door in five or six years, while placating them with fitness centers, fancy food courts and WIFI. Eventually, someone was bound to notice. And now they have. Those same undergrads, and their parents, are learning how worthless those undergrad diplomas from State U. really are after employers discover they can’t think their way through a simple problem. And they discover this only after getting themselves ever-deeper into hock to cover double-digit annual increases in State U’s tuition. Outcomes analyses, when parents are finally able to pry the data from your fingers, will show what ought to be obvious: That colleges are much better at educating undergraduates than universities, because that’s what they’re set up to do. By essentially providing a liberal education — teaching kids how to think and analyze ideas and problems without the distractions of an ESPN-featured sports team or the overhead of Nobel-winning faculty, is exactly what provides the best return on an undergraduate educational investment. And, as one who hires undergrads as entry-level employees, it’s more than enough for my needs. I can pick out small-college BAs from Mega U. BAs within the first minute of the interview. I much prefer to hire the former, and train them. If they later want a Mega U graduate degree, they have my full support. 3. New Mantra: Participate or Perish Glen S. McGhee is right — the problems (i.e, parents/employers like me, and government officials like Secretary Spellings) are not going to go away. Why? Because we’re confident that you can do this assessment thing. How do we know that? Because you’re the experts in analyzing everything else in the world. And because we have firsthand knowledge of your admissions departments, which wonderously assess and compare a worldly senior from a large, suburban public high school on one hand, with a shy, rural home-schooled kid on the other, and make perfectly fine decisions on which one to admit. You have my full confidence in your ability to find a way to do to yourselves, what for years you have done to them. But, “For the Ease of Masters,” no more.

Paul Gallagher, at 3:00 pm EST on November 13, 2007

Assessment

The things that scare me about assessment in its current incarnation are (1) it seems to be designed to please politicians and (2) it has been excessively simplified (these may be related, of course).

I have no objection to assessment in principle — obviously we want to know how effective we have been in teaching, what methods work better than others, and so forth. However, it seems to me that the current wave of assessment mania is starting to lose sight of the complexity of a college education. The results of our teaching efforts cannot be reduced to a number, or a page of numbers, or even a portfolio. Each of these can be a valuable tool, of course, but none is the whole story.

I understand that the public wants to know that its tax dollars are being spent wisely; I think that faculty object to assessment not ab initio, but rather when they are required to condense their assessment of a student into a few words or numbers. If we have to cut out the broad scope of detailed skills our students have acquired when we are assessing them, how long will it be before we have to cut them out of the curriculum as well? I maintain that if all we’ve done for our students after they earn a degree can be summarized on a page or two, we haven’t done our job very well.

I am afraid that if the politicians and the public do not accept the fact that the results of a college education cannot be expressed in sound bites, we risk losing more and more ground to countries where complex and analytical thought can prevail.

Suzanne Willis, Professor, Physics at Northern Illinois University, at 3:25 pm EST on November 13, 2007

Concretizing the Abstract: not a good idea

As the late philosopher Alan Watts observed, the abstract cannot be made concrete without losing its essence. Although the attempt to reduce the goals of higher education to a short list of discrete, measurable behaviors has appeal to the behaviorists, bureaucrats and politicians amongst us, doing so will trivialize what is really accomplished in successful higher education. That accomplishment is at its core abstract and even ineffable. Let us not trivialize our own mission! Also related is the elevation of “accountability” to sacred cow status. I predict that the ultimate “outcome” for the accountability fetish, and its conjoined twin, “assessment,” will be the same as it has been for all panaceas and educational fads: utter oblivion.

Dan Dydek, Professor of Psychology at Austin Community College, at 6:45 pm EST on November 13, 2007

Be afraid; be very afraid

If I learn to write paragraphs like:

“The diverse institutional plans that we are seeing are responses to changes in the institutional environment and its demands. Typically, and especially so in higher education, colleges and universities have great difficulty changing and adapting to meet the demands of uncertain environments.”

can I, too, become an overpaid educational bureaucrat ?

Paul Gallagher may be right that the degrees from Mega State U are worthless, but let’s stop blaming the TAs. Here’s one useful piece of information in the dreck put out by NSSE: college students devote, on average, less than 15 hours per week to their studies. Gallagher is right that liberal arts college students do a bit more work than their counterparts at Mega State U, but they’re not really close to the Carnegie norm (2 hours of study for each hour in class).

Can we do anything about this? You bet. I’ve just finished six years as director of a medium-sized academic program (200 undergraduate majors, 60 MA students). When I came on board, our faculty decided to start measuring student effort on our quarterly evaluations. For the last six years, over 60% of our students have met the Carnegie norms (and that’s in a bad quarter — usually it’s between 2/3 and 3/4). Nothing has ever been easier than getting faculty support for this policy, at least among my colleagues in the program. Surprise, surprise — faculty members like being evaluated on academic rigor rather than customer satisfaction.

The down side? Creating a rigorous program in a university where students study less than ten hours per week (that’s the reality at DePaul — data courtesy of NSSE again), is no way to make yourself popular among administrators, or among department chairs who would rather ignore this home truth.

And where’s assessment in all this? Nowhere to be found. Student effort, after all, is an “input". We’re in the business of creating specious “output” measures.

Michael McIntyre, DePaul University, at 6:45 pm EST on November 13, 2007

delayed gratification

I wholly agree with what Suzanne Willis wrote above. For me, the fundamental flaw of educational assessment is that notion that what has been learned must be learned immediately upon completion of a course or unit within a course. Student came in as X and emerged as Y as if s/he were a widget. I don’t think (and I’m sure someone can correct me if I’m wrong) that learning in most topics or disciplines is measurably discreet in this way.

Since I teach writing and literature, there’s no way the lessons of Shakespeare can be quantified and when a student might finally apply those lessons, even assuming there are lessons to apply, that might not happen until weeks, months, or years later. Sure, I could measure whether a student knows the difference between an Italian/Petrarchan or English/Shakespearean sonnet, but what’s that got to do with real learning? It’s when they understand what’s going on in the sonnet, when they combine the rhyme scheme with the logical scheme with the content to see how it applies to their life that they have actually learned something worthwhile, yet something that is immeasurable, something that outcomes assessment can never assess.

bradley bleck, instructor at Spokane Falls CC, at 6:45 pm EST on November 13, 2007

Lofty verbiage

“It is no longer sufficient for us to write lofty verbiage regarding the excellent programs we offer.”

Cal, the fear felt by many is that if faculty and students are beset by pervasive testing requirements there will be no one leaving our universities will be able to write lofty (read: abstract) verbiage regarding much of anything. Of course, this assumes that the student studies nothing outside their class assignments.

Joseph C, at 4:45 am EST on November 14, 2007

Gooses and Ganders

Having recently moved from a long career as a secondary English teacher to the community college, I am fascinated by the reactions of the higher ed community to the push for standardized, measurable outcomes and assessments. The comments here echo the longer, equally heated reactions among P-12 educators, particularly since the implementation of NCLB. I remember predicting early on that if the non-educator bureaucrats were allowed to impose their form of measurement and evaluation at the lower levels, that colleges would be next. Ironically, some in higher ed who thought all this pressure was great for elementary and middle school teachers are screming the loudest now that it’s their turn.

TeachMoore, Teacher at Miss Delta CC, at 4:45 am EST on November 14, 2007

Elitism

Lee Griffin commented above: “Naming a BA in political science to head the nation’s efforts in education hardly advances the profession of education, and both reflects and maintains an attitude that in education, “anyone can do it.””

I have never read a more elitist view. If our universities are run by these know-it-all PhDs, then I would be very afraid for our country’s colleges and universities. It’s no wonder the public mistrusts the academy. A few extra years in school, some academic hoops, and a dissertation do not a good leader make. Hell, even those credentials don’t even guarantee that someone can *teach* — the primary purpose of education.

Steve Mahoney, at 8:50 am EST on November 14, 2007

solidarity needed to resist outcomes assessment

In my current institution and department, we care more about providing students the opportunity to educate themselves than we do about testing whether they have taken advantage of that opportunity. Assessment assumes that when students fail to learn, it can ONLY be the fault of the teachers. This assumption is clearly untrue. At my previous institution, assessment was widespread, but my wise department simply had a committee which met once a semester for ten minutes to fabricate some bureaucratic drivel for the bureaucrats so we could be left alone to teach as we had always had. My advice to departments who agree that assessment is crap is to resist it at every level.

Angelo, Prof. at Liberal Arts College, at 1:10 pm EST on November 14, 2007

The Last Dinosaurs

It happened to physicians. “Insurance companies can’t possibly understand why we do what we do, or why it needs to cost what it does.” It happened to computer programmers and accountants. “People outside the U.S. can’t possibly understand the complexity of our data systems and regulations. There is no way that someone in Bangalore can do what I do here for a tenth of my wage and deliver anything close to the value I provide.” It’s happened to our military. “We’d like to use our own soldiers and Marines to provide security for diplomats and visiting Congressmen. But we can’t afford a single mistake, and only Blackwater’s mercenaries can provide that."A few years ago, it was your P-12 peers. Then the University Faculty Senates got really nervous.

The solutions? Dig wider moats. Build higher walls. Tell students that you love them, buy them cups of coffee at the Union, and inflate their GPAs a few more tenths of a point. As one who works for a organization that gets assessed every three months when it publishes its quarterly financial results, welcome to the Real World. We will soon begin to find out, whether you welcome it or not and whether you agree with the methodology or not, just which of your organizations are worth four years and $200,000, and which are not. Because the “U.S. News and World Report” fraud you have perpetrated for so long just won’t cut it any longer.

Paul Gallagher, at 2:30 pm EST on November 14, 2007

Dangers of Assesment

A real concern that many of us have is that the test will emphasize many fields that are not part of the “core.” The problem comes at the very first stage of assessment: what should we measure? Unfortunately, many academics (like myself) fear that what will be measured are things like writing ability, quantitative skills and subject knowledge that is overly broad. In my field of political science, many good PhD programs do not use the GRE subject test in political science, because it simply fails to measure political science skills and knowledge well. For example, “which of the following states has a unicameral legislature?” (a real GRE question I saw). The problem with assessment, particularly on the federal or state level, is that it often simply means standardized tests. Often, standardized tests measure a student’s ability to take standardized tests. That’s not what education should be about. When students can pay money to learn tricks to do better on the SAT, GRE, LSAT and MCAT, what makes you think that those measures are all that valid?

Assessment takes place; it does so because of professional norms. We assess ourselves. And, when we go up for tenure and for retention or promotion, we are assessed by our peers, who don’t want to be saddled with poor colleagues. Ironically, the more you push testing in education, the less things like these norms get taught and learned.

Finally, in all candor, those of us teaching first-generation-in-college students, students facing economic pressures, student graduated by lesser secondary education systems, or students with special needs are frankly scared that our lives (and, given the academic calendar, losing a job essentially means being unemployed for over a year, with 6 figures of student loan debt..it’s a big deal) will be ruined because our students just simply won’t do as well as others will.

John, Asst. Professor at Large State University, at 2:45 pm EST on November 14, 2007

Assessing outcomes of assessment outcomes

‘Nor is there any effort to discuss outcomes, policies, or improvements that have emerged from all of this costly and all-consuming assessment activity. All this activity, without a shred of evidence that the data we have collected — or will collect — will ever address ”national needs” or “improve institutional performance.”’

Indeed! We must have metrics in place to measure the success of the assessment outcomes before we move forward with assessment!

That statement just strikes me as a touch hypocritical. If the assessment process doesn’t work, then why would assessing the assessment process’ effect on our students’ learning have any more validity?

Mary, at 10:10 am EST on November 15, 2007

Goat-Ropers Do Assessment, Too

Good essay, Bernard. Glad you had that cask of brandy strapped around your neck, too.

Here in Texas, the right-wingers who control everything have not only loaded university faculty with massive burdens of mindless paperwork (the small-government types are all secretly convinced that enough paperwork will solve everything), but they have also decided that our class sizes are way too small.

Dang, if this keeps up I’ll have to cut back on the time I spend talking about the various unspeakable punishments that await plagiarists.

Maybe those prices on the Titanic tickets weren’t as great as they seemed.

Hnaef, at 9:40 am EST on November 16, 2007

Outcomes Assessment

Bravo to Prof. Fryshman. I agree that “outcomes assessment” is a stupid waste of time, perpetrated on faculty by pea-brained bureaucrats who, because they are uneducated themselves, do not understand the nature or value of real education. I, for one, believe that the university professors of this nation need to engage in a massive “civil disobedience” campaign in which we all simply refuse to do “outcomes assessment.” I suggest we adopt the motto: JUST SAY “NO” TO OUTCOMES ASSESSMENT. If we all refuse to do it, what are the stupid administrators going to do to us?

Barbara Hannan, Associate Professor of Philosophy at University of New Mexico, at 2:20 pm EDT on July 22, 2008

Jobs Related to Outcomes Assessment: No Gain, All Pain

or search for jobs directly.

Assistant Dean of Assessment, Office of Undergraduate Education
Rutgers University

The School of Arts and Sciences, Rutgers University, New Brunswick, invites applications for an Assistant Dean of Assessment ... see job

Associate Director of Customized Learning Solutions for Continuing Education and Conferences
Drexel University

The Goodwin College of Professional Studies at Drexel University invites nominations and applications for an experienced ... see job

Assessment Specialist — Job ID 899
SUNY — Empire State College

This is a professional academic services position with primary responsibilities of assessing prior learning and working with ... see job

Associate Director of Academic Affairs & Transcripts
Walden University

Walden University is an accredited graduate institution with a mission for social change, Walden is dedicated to high ... see job

Director, Alcohol & Other Drug Policy Initiatives
University of Pennsylvania

The nation’s first university, Penn is a world-renowned leader in education, research, and innovation. Situated on a ... see job

Staff Psychiatrist
Princeton University

Position Summary: Princeton University seeks a staff psychiatrist for a 10 month part-time (60%,) staff ... see job

Assistant Director, Frist Campus Center NEW POSITION
Princeton University

Position Summary: The Assistant Director of the Frist Campus Center is responsible for the overall ... see job

Facutly Assistant II
Harvard University

We have an exciting opportunity for a Research Technical Analyst at Harvard Kennedy School. see job

Coordinator/Duluth Medical Research Institute
University of Minnesota, Twin Cities

The University of Minnesota is a premier employer and a talent magnet attracting leading faculty and staff from around the ... see job

Director E
University of Pennsylvania