Views
Comparatively Speaking
Richard Sloan, writing in Blind Faith (St. Martin's Press, 2007), examines critically the hypothesis that "frequency of prayer can be associated with health outcomes." But in so doing, he goes beyond outcomes. Skirting accountability (if I pray, should I expect a health outcome?), he lands squarely on comparability:
"If we are truly interested in collecting the information relevant to health outcomes, then we should want to know whether it is better for our health to attend a Catholic Mass or a Quaker meeting."
The flaws in this presentation are immediately obvious: frequency of prayer, although it can be measured, does not begin to reflect the complex and comprehensive nature of religion.
Moreover, comparability of religions makes no sense. Religions are different by choice, not at all influenced by a simplistic measurement that is both limiting and largely irrelevant.
People, their lives, their interactions and their institutions occasionally reveal an aspect that can be measured. But we do so at our peril: Can we measure the quality of a marriage by the number of shared smiles, or a legislator's effectiveness by the number of votes she casts in a year? Even more treacherous would be an attempt to use these measurements to establish accountability, to compare, to improve.
And so to higher education, just now beginning to emerge from two decades of assessment. Assessment was a largely undefined activity that resulted in the collection of billions of elements of data, with no results and to no effect. The process was characterized by external pressure, by blissfully unaware faculty, and by administrators forced to comply with Dilbert-like mandates.
We are much smarter now, particularly those of us who attended the National Postsecondary Education Cooperative's 2006 National Symposium on Postsecondary Student Success.
We heard about the potential of the Collegiate Learning Assessment, among others, as a means of establishing student success over a wide range of learning outcomes. We also heard about the limitations, as one speaker reported that his institution had to pay students to take the CLA exam.
We learned of experiments being carried out with scientific rigor, so that results, when available, will withstand searching scrutiny and perhaps give us real answers. It was encouraging to learn that we are not alone in being troubled by the fact that measurements are being carried out with little thought to reliability, validity and relevance to actual student learning.
Best of all, there was a general tone of "we are not there yet," a message that should calm the ardor of those who insist on measuring postsecondary student learning outcomes, whether or not these measurements are reasonable.
Leaving measurement, we, like Professor Sloan, can skirt accountability. Accountability based on a limited slice of the postsecondary experience makes no sense and will ultimately collapse of its own weight. Appropriate accountability, on the other hand, is regularly demonstrated by colleges and universities in a multitude of ways, as was first established in 1994 by the National Association of Independent Colleges and Universities' Task Force on Appropriate Accountability.
This brings us to comparability. No less than religions -- or marriages -- or legislators, colleges and universities have unique personalities, cultures, missions, histories, areas of expertise and communities of service. In fact, much of the success of American higher education lies in its diversity, its competition and its differences.
For the most part, comparability is inapt. Regrettably, the measurement of student learning outcomes is sometimes justified by claiming that this will enable postsecondary institutions to be compared, and thereafter to be "improved."
American colleges and universities already operate within a culture of continuing improvement, self generated and responsive to changing needs and changing opportunities. But, improvement dictated by specious comparisons that are in turn based on "measurement" could seriously compromise the diversity of institutions that is so healthy a characteristic of American higher education.
We are in the midst of an unprecedented examination of higher education. Some have used this to advocate measurement of a kind that limits both teaching and learning, to establish accountability on the basis of irrelevant yardsticks, and to speak of comparability as an inevitable consequence.
In this atmosphere, a message from the education secretary reaffirming the strength of higher education and identifying diversity as an aspect to be preserved would be highly reassuring.
