“The final version released Wednesday strives to strike a balance between responding to the public pressure for more accountability in measuring and reporting learning outcomes, yet firmly embracing the idea that individual institutions should decide for themselves what to measure and how to do so.”

Which, of course, vitiates any value of the reported data. Comparison with other institutions should not be a cause of fear for educators. If it is, people should properly look to why they feel they cannot effectively measure up to standards achieved elsewhere.

JBM, at 7:35 am EST on January 31, 2008

Both varied and comparable

Those who believe that the information that academic institutions provide on their actual educational results requires uniform standards in order to be comparable are not seeing the academic advantages of the model being proposed by AAC&U and CHEA. The colleges and universities that are working with us are developing the capacity to say: this is who we are (the institutional mission), this is what we expect our graduates to have demonstrated (expected student learning outcomes per institution and per program), these are the standards we use in evaluating student work (explicit criteria of evaluation), and this is how our students are doing (aggregated data on actual student learning outcomes). Margaret Spellings’ niece (or any one else looking at specific colleges and universities) would have good information that permits her to compare one institution with another; yet the faculty are still deciding the standards.

David Shupe, eLumen Collaborative, at 9:05 am EST on January 31, 2008

hypocrisy

The businessman Charles Miller has a good point. We should stop being hypocritical about requiring standardized testing for college entry while rejecting similar measures of what students take with them when they exit. If a nationally standardized test such as the SAT can measure academic preparedness, why can’t a similar exit exam, like a GRE, do the same? With a national exit test we could easily compare the performance of institutions by seeing the value each adds to the scores of their students. At this point, I suspect many academics reading this would be hitting their mental nonsense alarm buttons. So what’s wrong with this argument? To start with, the SAT doesn’t work. It doesn’t measure or predict academic performance. See John Douglass’ book on the University of California, or my own, for documentation of the uselessness of the SAT. We must discontinue our hypocritical and self-serving use of a proxy for SES that doesn’t predict college grades and then, with cleaner hands, rethink how each institution can justify to the public the expense it requires of taxpayers and families.

Joseph A. Soares, Associate Professor at Wake Forest University, at 10:00 am EST on January 31, 2008

I don’t mind reporting outcomes, but the “comparng campuses” movement has led to such nonsense as the idea form, a ridiculous tool that combines touchy-feely ideas with false rigor. We should avoid this and the like.

TBD, at 11:20 am EST on January 31, 2008

Joan’s national scores?

So, Joan, why aren’t these scores listed at the College of Nursing USF site? When will they be? Why doesn’t every school list them?

sceptic, at 12:05 pm EST on January 31, 2008

Really?!?

Every article I see about this says “people” want this sort of thing but I have never really heard who or why. I have the distinct impression that this is a solution in search of a problem.

Who is displeased with the quality of graduates of American Universities and why?

Without a compelling answer to that question the effort to quantify successful outcomes is a waste of everyone’s resources.

Maybe it’s not about satisfaction, perhaps we’re talking about comparison shopping here. If that’s the case, it is also a colossal waste. Performance of previous students are a poor predictor of how well one educational environment will work for one given individual, at one time, in one major, with one set of instructors and peers. Coming up with some sort of “Outcome Success Index” is just a marketing tool to be manipulated in front of lazy shoppers.

All the information a parent needs to determine the quality of an institution is already available. It just requires a little research, and the process is personal and specific enough that it should require that research.

David, at 4:50 pm EST on January 31, 2008

Response to Absent Referent

originally posted by absent referent:

“what if blah blah blah, world not fair, blah blah, democracy, blah, fluffy bunnies and rainbows, blah”

Like many others, you seem to make the mistake of equating “democracy” with “fairness", especially in economics. Look at where the rest of us are coming from; democracy is a tool that may indeed help creat fairness, but we’re interested in equality of opportunity, not outcome.

Do you think that isn’t “fair"? Well, honestly, that’s just too bad, because I don’t think it’s “fair” either if I’m punished for my productivity so people like you can try to level the playing field. A reasonable amount of that is fine — we can afford it. The problem with your talk about avoiding “private tyrannies” isn’t central planning, its the inferred government that would be powerful and pervasive enough to carry out the level of meddling you describe.

Tim in TX, at 9:15 am EST on February 1, 2008

To Tim in TX

Hey Tim, my fellow Texan! Each of us is operating a different cognitive schema, template, paradigm, what have you. So each of us thinks the other is crazy.

As I state in my piece, the automatic assumption about my concept of democracy is that of Central Planning or a Big Brother to impose leveling, which contradicts “democracy.” I think you’re thinking like a well-programmed computer. A commuter can’t solve problems that haven’t already been solved. But now you have the problem of justifying a pretty tired old solution that is rapidly becoming anachronistic.

Aside from “fairness,” how do we know we are NOT living in a democracy? The very fact that outcomes are so unequal. You see it as a symptom of health; I see it as diseased. The input is unequal, so the outcome is too. A society of equals would practice such checks and balances that outcomes in material wealth would be maintained as more or less equal. Power would thus be shared. Just as the body needs a balance of vitamins and minerals to function well, so also the body social needs a balance of material prosperity, lest certain parts of dead and decaying flesh poison the body social as a whole. Gated communities won’t help. This is why equality of outcome is important both to society and the environment. It need have nothing to do with an Elite managing it all from above. The point is that everyone deserves to engage in self-management. At the same time, it’s conceivable that within that material equity humans could enjoy even more freedom than the most privilege elites, living in luxury and splendor, now experience.

My guess is they are mostly pretty unhappy, discontent, and wishing they didn’t have to go to so much trouble to avoid paying taxes, and wishing they could do more to further weaken the threat of concerted political power from below. They might try buying out and consolidating the biggest media outlets as propaganda arms. They might reserve the right to declare class war on those below, while whining about the illegitimacy of class warfare rhetoric from those below. They might take over the government and use it to their own ends while feeling resentful if the population tries anything of the kind.

My cognitive schema, I confess, triggers an automatic assumption too. It assumes there’s contradiction in the old saying, “Equality of opportunity regardless of outcome.” To you it’s a poetic paradox and perhaps the very epitome of justice, if not “fairness.” In my framework, once there’s been unequal outcomes there can be no equality of opportunity. So the notion of its existence must be questioned as such. It’s a phrase often thrown out as a “thought stopper.” There are minor, isolated opportunities, but Opportunity on a large scale is dead once there’s been an Outcome on a large scale. The winners are now in a position to dominate and steal from the losers. They now even have the power to define “stealing,” and the victims can be accused of whining.

This is pretty much the same condition in both a Communist and a capitalist state, though the former is more extreme. There’s also “equality of opportunity” in a Communist state so long as you adopt the prevailing ideology and strive to get with the program (doctrine) and serve Big Brother so he might promote you up the line and you can feel “successful.” (See the film Fight Club, as an example of a character split in two as he struggles with this dilemma.) But the Communist State would never allow ALL its citizens to adopt its values all at once and strive for promotion; the pyramid’s too thin at the top. Neither can a capitalist state allow that. It’s called the “Dictatorship of the Pre-existing Outcome” as far as the larger population is concerned. Both are profoundly contemptuous of democracy. The Communist State is just more honest with itself about that.

This has implications for “outcomes assessment” in Higher Ed. It may be a lack of imagination, hardened cognitive schemas, and good old ideology that makes our education system into the accomplice of an undemocratic status quo, rather than a democratic solver of problems not yet solved.

Absent Referent, at 11:20 am EST on February 1, 2008

Parallel Universes

Assumptions, assumptions. But see Participatory Economics as one well-worked out model of an alternative. It’s not about people or their talent being punished so wealth can be redistributed to the derelict.It’s about everybody participating, and wanting to participate, and being productive, developing their talent, and experiencing empowerment through such innovations as balanced job complexes (something we all presumably do around the house in our families.) It’s about an economy that is highly productive while not overproducing or ruining the earth with overconsumption. It’s about full employment and more time for civic participation and leisure. But in order to pull that off, the entire citizenry would also be educating itself for these radical new skills. For it’s about supply and demand, but no market, planning but not in a centralized, authoritarian way. And it’s about getting rid of a professional managerial class as in capitalism or a coordinator class as in the former Soviet Union.

There ARE some pretty counterintuitive things in the model, like paying ourselves more when we ALL do menial work, and paying ourselves a little less when we do our favorite, creative, empowering work, both forms of which would be balanced. And we can’t really get to that model from here without a long transition and a gradual internalizing of those skills. But that’s a new and as-yet unsolved problem. The model HAS been shown, however, to DRIVE the values of solidarity, equity, self-management, diversity and efficiency.

The model would indeed develop everyone’s talents to the maximum. Our own education system has built into it all kinds of, mostly unconscious, exclusions, and that serves what IS, as though WHATEVER IS, IS RIGHT. That’s what we’re teaching, by and large: adapt to what IS. I have no problem with that per se. But we should teach ourselves not to stop there.

The model would also preclude an underground economy. There would be no need of it, nor could one hide one’s ill-gotten gains among those whose ill-gotten gains are considered legitimate by a capitalist culture.

It’s a very detailed, complex model, one that has been worked out mathematically as represented in a book, Participatory Economics, Princeton UP, 1991. I haven’t the space here to explain past the avalanche of automatic, erroneous assumptions that my comments invariably rile up. Go to www.parecon.org and read up on it. You won’t agree with it, but at least you’ll have to think past the usual erroneous assumptions in order to argue with it, and that’s a good thing. Personally, I’ve found it very educational to argue with it. Yes, I have problems and questions about the model. What the model does do is seek to awaken our desire for something better and more genuinely democratic. And that Desire is itself educational, and ought to be considered when we do assessment.

Absent Referent, at 5:15 pm EST on February 1, 2008

lbillman touts exit exams, but doesn’t emphasize the most important aspect of them — they must count to the student taking them.

When college-bound students at a college prep school take the SAT, the primary purpose is to get the student admitted to the college of his/her choice, and that supplies the motivation to the student to perform. If we then take the average scores of all of these students and use it to assess their school, that application is a “free rider” on the purpose of the test. There is no such “free ride” available at the end of college. Most students do not go to graduate school, and no prospective employer gives a rat’s *** about a GRE score (which has no predictive power for graduate school performance and even less value in any job anybody could think of).

So all of you folks who think it’s a no-brainer to require all graduating seniors to take the GRE or it’s equivalent, and then you are going to use those scores to assess the “effectiveness” of the institution — what incentive do the students have to do well on the test? Suppose you are a graduating senior who has done very well and learned lots of valuable things in college. Your prospective employer wants to fly you out for an exhausting 3 days of interviews at their headquarters. You return home on the “red eye” flight, and slide into the desk where they plop down this exam which has *no* *effect* *whatsoever* on *your* future life. So, what do you do? Do you tell your future employer to f-off because your school’s rating depends on you doing well on the exam? Or do you go to the interviews, come back, stick your name on the exam paper, turn it in, and go back to your dorm room to get some sleep?

So you are going to base your college admissions decision based upon the performance of college seniors who are so hung-over from the graduation party from the night before that they can barely remember their own names? Well, that’s pretty idiotic.

cathyf, at 6:50 pm EST on February 4, 2008

Exit tests

Yes. Have exit tests. Make them mandatory and universal with no exceptions for foreign students, athletes, etc. Don’t let higher education dictate test content. And, make results a mandatory inclusion on all transcripts. Have a multiplicity of tests for various graduates, plus a generalized knowledge test. We now have a slippy, sloppy, slovenly national evaluations of students that suggests a deep fear of accountability in higher education.

Marvin McConoughey, at 4:45 am EST on February 7, 2008