Thank you

Thank you for such a wonderfully written piece. In this frenzy to measure everything (and in fact having measured nothing), your article brings some balance and rationale thinking. I concur with you wholeheartedly. Thank you!

Jim, at 9:45 am EDT on June 8, 2007

The Importance of Scale

Professor Fryshman’s identification as a fellow physicist inspires me to couch a comment on his conclusion in terms of an analogy between two seemingly simple but fundamentally important questions, “What’s in the atom?” and “What have students learned?”

Ernest Rutherford began to answer the first question a century ago using table-top experimental apparatus. The search continues. Recently, we’ve been reading about the imminent commissioning of the Large Hadron Collider near Geneva, a scientific instrument seventeen miles long.

The second question is much more difficult than the first, dealing as it does with the human mind. Professor Fryshman describes current initial efforts to answer it, and concludes that we must vastly increase the scale of our efforts to develop sound scientific bases for reliable and valid assessment instruments.

I strongly agree. I also agree that the federal government should take primary responsibility for providing the substantial financial investments that will be necessary. However, I differ with Professor Fryshman’s view that we cannot expect commercial vendors “to undertake the vast expenses associated with establishing that their assessment products are reliable, valid, relevant and comprehensive.” It seems to me that we should expect of them what we expect of pharmaceutical companies, which are expected to support the large-scale clinical trials necessary to demonstrate that their products are safe and effective.

Finally, I would urge patience on all concerned. Discovering how really to assess what students have learned is likely to take at least as long as it is to discover what’s really in the atom.

Don Langenberg, Chancellor Emeritus at University System of Maryland, at 10:15 am EDT on June 8, 2007

Granted while creativity, critical thinking and dissemination skills are difficult to gauge. However, testing discipline specific knowledge is rather straight forward. For the sciences, and engineering standardized exams in statistics, thermodynamics, mechanics, organic chemistry mechanisms, etc. are out there or could be easily be developed.

Red Stater, at 11:15 am EDT on June 8, 2007

Cart Precedes Horse — Problem-Solvers Wanted

Leonard says: “We have research in hand; too many college graduates are not proficient in communication and quantitative skills. This research has long been complemented by employer complaints that in aggregate college graduates are ill prepared to collaborate, solve problems and communicate.”

FWIW, I do agree that college graduates are not proficient in communication (broadly speaking). But one of the devils in the details is what we’ve learned from Writing / Composition Studies: that communicative performance (and judgment of that performance) is entirely contextual. Simply because a student has learned how to communicate “well” by the standards of top-shelf engineering journals, because that student can report on analysis of some proposed new material for improving structural density in steel manufacturing, wouldn’t mean that that same student could write a coherent report for psychologists, or business leaders, or a Congressional oversight committee on the very same subject, let alone on subjects *other* than engineering.

Your audience (and what you can assume they know) and your own knowledge of a field as well as your knowledge of communicative conventions for that field and for the document produced will condition how effectively you can communicate with that group. This means that if business leaders want better communicators, it’s going to be up to schools and colleges of *business*—not the rest of the university—to really provide the training space necessary for students to learn those skills as they apply to the world of business. Likewise for any other discipline whose private sector counterparts are bitching for better graduates but don’t want to support discipline-specific change and instead advocate blind, sweeping changes for HE across the board.

Bottom line: the complaint from business leaders that graduates cannot communicate, like the complaint that graduates cannot solve problems or work collaboratively, may have merit, but the drive for numbers-based assessment isn’t going to jumpstart the pedagogical efforts needed to address those deficiencies because they are all contextual, like writing is. (Would you want someone who is a fantastic problem-solver when it comes to, say, addressing your personal accounting, trying also to solve the problems you have with your car? Or your health? Of course not. Problem-solving, too, is contextual—there is no global problem-solving ability, just as there is no global writing ability.) But these very things that are so critical are also difficult to reduce to simple numbers. What schools of education learned long ago but few other disciplines seem to catch on to is that portfolio assessment of performance makes more sense in assessing graduates’ abilities and knowledge. When’s the last time you heard of a College of Business assessing its juniors and seniors for communicative ability by offering recursive feedback on student presentations and papers and grading those students on their assignment portfolios?

But even assuming these crucial skills are so reducible to easy numbers, the contextual nature of these abilities still means that HE would have to devote A LOT more of its resources (meaning, more tax dollars to boot) just to work at ensuring the presence of classroom factors that permit development of students’ abilities in problem-solving, writing/communication, and collaborative work, including small class sizes (necessary for professors to provide concrete, individualized feedback to students so that they actually learn what they’re supposed to), long before we ever get around to assessing student performance for real outcome gains. But it seems to me that every time someone acknowledges that more funding is needed to produce quality education and produce the outcomes sought after by business, that ballooning class sizes and similar barriers to student learning are overwhelming HE and that small-class size is just one of many things that we must learn to afford if we really expect those outcomes, it’s business that pisses and moans about the sink hole that is Higher Ed. “Do more with less,” they say. But there is a point at which you simply do not have what it takes to “do more"—something even business leaders understand. It seems we need more problem-solvers among business leaders themselves, some talented people who can take an honest look at education not from a business standpoint but from one of learning and educational/psychological development (i.e., the natural state of the college student). That might help businesses understand what’s really going on before they decide they know how to mold students into what they want them to be post graduation.

The assessment rush is just the natural progression of the vapid assumption that if we can’t quantify it and make it available in the marketplace (for ever smaller sums of money), it can’t be doing students any good. This attitude is antithetical to actually learning anything; anyone who’s spent time in really educating others can tell you that people don’t always learn immediately, nor do they learn much more than how to parrot back thoughtless responses if they haven’t had time and space to reflect on, analyze, and integrate that learning with pre-existing knowledge and to continue practicing learned skills to stay sharp. In short, real education will never be the commodity that business seems to want it to be, and the drive to force education into the capitalist mold of a service product only ensures that we get even further away from producing those outcomes business claims to desire in its newly hired graduates.

Sick of Biz driving Ed, at 3:10 pm EDT on June 8, 2007

Adminstrators who don’t know what actually goes on in a classroom? Assessors, admistrators and national leaders who get obsessed with reconstituted teaching fads? Mindless busy work and paper work? Committee mania? Welcome to the world of standards-based / data-based education. High schools have been dealing with this for years now. Has it helped? Not much, so let’s make all the colleges do it too and see what happens. It will keep instructors busy in their “down time” and away from doing what’s important—like teaching.

stephanie, HS Teacher, at 6:20 am EDT on June 12, 2007