Assessment

Essay: Research universities must pay more attention to student learning

It is my view that most of us engaged in education at our nation’s leading research universities focus our attention upon the wrong issues.  These universities are wondrously complex institutions that defy easy analysis or understanding.  We therefore tend to concentrate upon their most visible components, such as scientific research, star professors, state-of-the-art facilities and technology, economic development, international impact, and football and basketball teams. 

It has become a cliché that American universities are the best in the world. This claim, while valid in important dimensions, can lead to complacency and neglect of serious problems.

Much of our international reputation is based upon two outstanding features of American universities: unrelenting commitment to an atmosphere of free and open inquiry, and excellence in scientific research. These twin advantages attract the best talent from around the world to American universities, not only to our graduate programs but increasingly to our undergraduate colleges as well.

In other aspects of our enterprise, however, we find ourselves hard-pressed. Our funding model, first of all, is under severe duress. States have repeatedly reduced their support of public universities, most severely in the past five years, a disinvestment that now threatens to erode their quality and competitiveness. 

Some public universities have understandably attempted to make up the deficit in state support by raising undergraduate tuition aggressively and increasing the proportion of out-of-state students. But this strategy undermines the public mission of providing access, creates anger in the state, meets resistance in the legislature, and has now attracted the attention of the White House. As states have shifted the burden of paying for college from their general funds to students and their families, the perception has grown that higher education, once seen as a public good, has become a private interest. And these coping mechanisms, if continued, will lead to general deterioration in the quality of undergraduate education, the very part of our universities that depends most upon state support.

At private universities, tuition and fees plus room and board have, in some cases, reached $55,000 per year. Although most students do not pay that full cost, and though generous financial aid policies and endowment spending have actually brought down the real costs for the average student over the past five years, a degree carrying a price tag of well over $200,000 creates automatic sticker shock in the public.  It also raises real questions about whether we have been paying enough attention to holding down expenses. 

The airwaves are rife with predictions of disruptive change coming to the economic model of higher education.  It is no wonder that parents paying and borrowing for a college education steer their children toward practical majors that seem to promise instant employment, and discourage them from studying the liberal arts and sciences in pursuit of a well-balanced education. A private interest in education today means a purely economic one.

From this inversion of values flows our second problem: a redefinition of the purpose of undergraduate education. Fifty years ago, when I started college, there was a widely shared view in America that the purpose of a college education was to prepare students to become educated citizens capable of contributing to society.  College was in the public interest because it gave graduates an understanding of the world and developed their critical faculties.

Today, many Americans believe that the sole purpose of going to college is to get a job -- any job. The governors of Texas and Florida are quite clear on this point, and draw the corollaries that college should be cheap and vocational, even when delivered at major research universities like the Universities of Texas and Florida.  A university education is more than ever seen as strictly utilitarian.  The reasons are clear: a) as more students and families pay a large share of the costs, they naturally want a ready return on their investment; b) the most desirable jobs in this highly competitive job market require a college degree; and c) the gap in lifetime earnings between college and high school degree holders is huge. 

Today, as many Americans hold a purely instrumentalist view of undergraduate education, they want a detailed accounting of its value. Hence our third problem: close public scrutiny and political accountability.  Parents want to know, what did my daughter learn, and how does it contribute to her career? State legislatures want to know: what is the graduation rate at our university? How many undergraduate students do faculty members teach? And much more.

These questions put us in an uncomfortable position, because in some cases we do not know the answers, and in others we know them but do not like them. Many of us have eschewed the use of instruments assessing the value of general education, particularly at our major universities. We have, often for good reason, lacked confidence that such instruments are reliable measures of the value of a research university education, particularly if they are based on a one-size-fits-all approach. 

However, given the level of scrutiny and skepticism in the public and in state houses, research universities need to take this issue seriously. 

The professionalization of the professoriate has been crucially beneficial for research and graduate training at many institutions, but at most large universities, it has been problematic for undergraduate education. Several recent studies, some flawed but still indicative, have revealed that a significant percentage of students do not improve their critical thinking and writing much at all in the first two years of college. This should come as no surprise, given the dearth of small classes requiring active participation and intellectual interaction.

Too many students are adrift in a sea of courses having little to do with one another.  Many courses, even at the upper division level, have no prerequisites, and many require no debate or public speaking or the writing of papers that receive close attention and correction.  A student’s curriculum is a mélange of courses drawn almost haphazardly from dozens of discrete academic departments.  And there is substantial evidence that students are fleeing demanding majors in favor of easier ones that have the added lure of appearing to promise immediate access to jobs.

The combination of drastic state disinvestment in public universities, student careerism, and pedagogical failings of our own has serious consequences for the country. To take one significant example, we now know that more than 50 percent of the students starting college with a stated desire to major in science or engineering drop out of those majors before graduating.  

We can no longer blame this problem entirely on the nation’s high schools. A substantial body of research demonstrates conclusively that the problem is frequently caused by poor undergraduate teaching in physics, chemistry, biology, math, and engineering, particularly in the freshman and sophomore years. Students are consigned to large lecture courses that offer almost no engagement, no monitoring, and little support and personal attention.  The combination of poor high school preparation and uninspiring freshman and sophomore pedagogy has produced a stunning dearth of science and engineering majors in the U.S.  Our country now falls well behind countries like China and India in turning out graduates with strong quantitative skills. 

According to the Organisation for Economic Co-operation and Development, the U.S. in 2009 ranked 27th among developed nations (ahead of only Brazil) in the proportion of college students receiving undergraduate degrees in science or engineering.  As a result, American students are a dwindling proportion of our graduate enrollments in science and engineering. An administration report not only states that foreign students are earning more than half of U.S. doctoral degrees in engineering, physics, computer sciences, and economics but also estimates that the United States, under current assumptions, will in the next decade under produce college graduates in STEM fields by one million.

I fear the practical as well as intellectual consequences of these trends. However, I am not a pessimist; I am a realist. In this, the 150th anniversary year of the Morrill Act, I think we can do something to reverse these trends, if we muster our collective will to do so. The anticipated report of the National Research Council on the state of our research universities will, I hope, focus national attention on the problems and opportunities confronting these vital institutions.

But over time, the renewed public investment in higher education that our country needs is unlikely if we do not acknowledge our own shortcomings and begin to address them. First, we need to say loudly and clearly that improving undergraduate education will receive our closest attention and best efforts. We need to alter faculty incentives by making undergraduate teaching at least equal to research and graduate teaching in prestige, evaluation, and reward. And we need to do research-based teaching that takes account and advantage of the latest findings of cognitive science, which are extensive, on how students learn. In brief, they learn by doing, not by just listening to someone else; they learn by solving problems, not by passively absorbing concepts; they learn best in groups of peers working things out together.

Fortunately, some of our best universities are leading the way. Initiatives at such institutions as Johns Hopkins University, Stony Brook University, the University of Michigan, Stanford, Yale, and others offer great encouragement. The remarkable thing about them is the acknowledgment by faculty that we need to focus much more attention on undergraduate education, and that we need to deliver it more effectively than we have been doing. I find these examples exhilarating and promising.

At the Association of American Universities, we hope to disseminate the findings of such research across our universities, both public and private, and thus to stimulate more students to persist in their study of math and science and engineering. We have embarked on a five-year project led by top scientists and experts in science pedagogy designed to help science departments implement these new teaching methods. One of my hopes for the future of research universities is that student learning will be at the center of faculty concern, research will inform teaching, undergraduate classrooms will be places of engaged, participatory learning, and a university education will be not just a means to an entry-level job, but an invitation to a lifetime of learning.

I am well aware of the difficulty of changing those cultures. It will take a broad and deep effort to realize serious and sustainable gains.  The stakes are high, not just for our universities but for the country.  In the global knowledge economy, an educated public is essential not just to economic competitiveness but to national well-being. 

Hunter Rawlings is president of the Association of American Universities. This article is adapted from a speech delivered on February 28, 2012, at the De Lange Conference at Rice University.

AP enrollment grows along with questions of privilege

Smart Title: 

Report says black, Native American and Hispanic students aren't taking AP classes they could succeed in, but are failing the exams they do take at a higher rate.

Academically Adrift's testing suggestions spawn resistance

Smart Title: 

It starts with "ass" -- but get your head out of the gutter; it's not what you think. Still, some deans suggest avoiding the word around professors.

Cavanaugh essay how accreditation must change in era of open resources

Imagine yourself emerging from the Way Back Machine in London, England. It’s 1526. Henry VIII is on the throne. You furtively duck into a shop, and quickly head to the back room. You’ve come to buy an English translation of the New Testament. The mere possession of this book is punishable by death.

In the 1520s, having open access to books (knowledge) was a dangerous game. It threatened the establishment. It meant that ordinary people could see for themselves what the elite had guarded so closely.

Enter Thomas Cranmer, Archbishop of Canterbury, who commissioned the publication of the “Great Bible,” in English, making it available to every church where it was chained to the pulpit to ensure it was accessible and didn’t “disappear.” While the publisher paid for this access with his life, in three short years readers were provided so that everyone, even the illiterate, could hear the Word of God proclaimed in their native English.

Fast-forward 472 years. You’re a college student. You’ve taken advantage of some amazing opportunities in the online world. You’ve listened to Nobel laureates discuss the Eurozone crisis and explain how current difficulties relate (or not) to classical theories of economics. You’ve worked through the underlying physics and chemistry for nearly every episode of "MythBusters." You regularly watch the TED lectures. And you’ve even taken courses from the Open Learning Initiative and from OpenCourseWare at MIT. Now you want the academic credit for those forms of learning.

Although you won’t actually be burned at the stake as Cranmer was, you have a very good chance of experiencing the modern version of this torture because it is equally threatening to the elite. It goes something like this.

First, you’ll be asked to produce the sacred document, otherwise known as a transcript, indicating that you officially took the course. No transcript you say? Sorry — your learning is then considered “illegitimate,” and you’re then often cast out into the night where there is weeping and gnashing of teeth as you stumble back to the very beginning of college to start over.

While an exaggeration, today -- through such outlets as TED, various open-source course initiatives, and primary sources through digital content providers -- we all have access to the knowledge that previously was the province of academia. In the same way that access to the New Testament gave otherwise uneducated English people access to the very heart of Christianity, that access is “dangerous.” It threatens the central notion of what a college or university exists to do, and so, by extension, threatens the very raison d’etre of faculty and staff.

Threats to a well-entrenched status quo are not well-received. But the funny thing about many of them — whether books or ideas — is that they often quickly become the mainstream.

Vu Déjà 

Higher education is facing the very situation that confronted our colleagues in the P-12 world when home schooling threatened the world order. Initially considered a fringe activity of substandard quality, the sector figured out that if appropriate standards (i.e., learning outcomes) were agreed upon and stated clearly, it didn’t really matter what path students took to get to the knowledge destination.

Higher education needs to take a lesson from that experience and work much harder on specifying our analog of the Common Core State Standards. The tools are there, and have been there for a very, very long time. It just has not been in our self-interest to develop and agree on them. But we’d better, and we’d better do it now. Otherwise, it will be done to us.

What Thomas Cranmer figured out was that it was impossible to execute people fast enough to stem the desire to access the new sources of knowledge. So he wholeheartedly adopted the reform, and made it his own. What we need to learn from that is to accept the reality that anyone can access the same information we academics used to carefully mete out, so the best approach is to adapt and make that reality our own. We need to create a higher educational system that embraces competency-based achievement, realign the milestones by which we gauge increasing levels of knowledge/competence, and redefine degrees on this basis.

We have an instructive example. Standard 14 of the Middle States’ Characteristics of Excellence pertains to the assessment of student learning. The standard requires that students be told what “…knowledge, skills, and competencies [they] are expected to exhibit upon successful completion of a course, academic program, co-curricular program, general education requirement, or other specific set of experiences… .”

As stated in the Standard, the objective is “…to answer the question, ‘Are our students learning what we want them to learn?’ ” Such assessment is “an essential component of the assessment of institutional effectiveness” (Characteristics of Excellence,, p. 63). The description then goes on to discuss how learning outcome assessment should be designed and its results used. Nowhere is there a discussion of credits.

Given that we already have an accreditation system based on the assessment of student learning (i.e., knowledge/competence acquisition), then it is a rather straightforward matter of taking the existing approach to the next step to complete the conversion process from one grounded on credit accumulation (irrespective of learning) to one based on demonstrated learning outcomes.

More specifically, we need to adopt the approach already taken in many professions of clearly articulating what students are supposed to know and be able to demonstrate at various levels of educational attainment, and create accreditation standards and metrics that reflect it. This would put real teeth in the assessment of student learning outcomes by putting consequences on not doing it well, as well as put the focus on where the content comes from and its quality assurance that underlies the knowledge/competence we expect students to acquire.

When that happens, the recognition of prior learning becomes very straightforward, and its source becomes irrelevant as long as the appropriate competencies are shown. In other words, we already have all the basic elements necessary to take the Cranmeresque step of moving from banning the immediate and unquestioned acceptance of demonstrated knowledge/competence to creating the postsecondary equivalent of the Book of Common Prayer.

The Brave New World

Adopting an accreditation system predicated on the authentic assessment of student learning outcomes liberates faculty to serve a much more important role — that of academic mentor and guide for the student’s learning and knowledge/competence acquisition process. In a way, this will return us to the past, whereby through the judicious use of technology faculty will be able to provide far more individualized instruction to many more students than the current system could ever possible allow or support. In another way, it means that the kind of individualized attention we give to doctoral students can be extended to all. This would be a major improvement for students and faculty alike.

To continue to have legitimacy, accreditation must focus on the core issue — student learning. Accreditation must begin certifying that students actually learn, and that what they learn matches the stated objectives of a course, an academic program, or a specific set of objectives (such as in general education). In short, accreditation must move from certifying that an institution claims that it is doing what it is supposed to do to certifying that students are learning and progressing in their acquisition of knowledge/competence.

Because people can simply wander around the Web and pick up content that is neither amalgamated by a content provider nor verified for accuracy, it will become necessary for some entity to engage in quality assurance in terms of learning outcomes. The job of verifiying bona fide knowledge/competencies and establishing where along the continuum of knowledge/competence acquisition a student falls can become the province of organizations that resemble LearningCounts.org, or even broader entities.

In both cases (i.e., using content offered by an certified provider or doing it on your own with no official guidance), a credential or type of certification would be provided each time a new level of knowledge/competence is reached. The student would then deposit those credentials or certifications into a credential bank for future reference. The student -- not the registrar’s office -- owns the credential

Degrees Deconstructed and Decoupled

We get to this alternate accreditation world in two ways: by clearly defining what each degree means and aligning accreditation with content providers (not institutions that confer degrees).

This requires that we come to quick agreement on what different types of degrees mean. In the United States the TuningUSA effort is just beginning the work of more clearly articulating what knowledge/competencies a student is supposed to demonstrate before being awarded a postsecondary degree.

This is in contrast with the current practice of awarding degrees based on a student's spending a specified minimum amount of clock-defined time amassing an arbitrary number of credits and obtaining a minimum set of grades. Nothing in the current definition says anything about what knowledge or competencies the student actually demonstrates. We need to test it — look up the degree requirements for English literature degrees across a variety of institutions and compare them. This loose approach is in contrast to efforts in other parts of the world, such as Europe, where degree qualifications discussions have been ongoing for over a decade.

Once the accreditation focus is placed on student learning outcomes for real, accreditation becomes tied to learning and is decoupled from institutions granting degrees. Accreditation then becomes aligned with entities that provide content and the parcels or “courses” in which they are delivered. The seal of accreditation would then be placed on the separate pieces of content offered by content providers who demonstrate that the content offered comes with embedded authentic assessment of learning. To be sure, most of these providers will still be postsecondary institutions, but the accreditation umbrella is extended more broadly to reflect the current reality that content comes from many sources.

In such a system, regional accreditation no longer gives thumbs-up or thumbs-down only on the traditional degree-granting institution. Rather, it focuses on what is provided by any entity that wants to claim it’s in the business of offering content. If and only if that content meets certain standards would it be “accredited.”

Shifting the focus from the institutional level to the content level would strengthen the link between accreditation and federal financial aid eligibility. If and only if a student was using content from an accredited source would the student be able to apply for and receive federal financial aid. Likewise, if the student has amassed knowledge/competencies from self-instruction or from noncertified sources and wants to convert that into “certified learning,” then federal financial aid could be spent only at accredited entities in that business.

Charting a Future Course

The possible future I have described here is both scary and exciting. We can choose to sit down in the captain’s chair and help chart our own course by fully embracing new opportunities while really being serious about quality as defined as authentic assessment of the acquisition of knowledge/competence. Or we can put up the shields, claim that the way we provide access to knowledge now is to remain immutable for all time and that change will bring our world crashing down and condemn us to eternal damnation, and have a modern equivalent of Thomas Cranmer bring it all crashing down.

It’s up to us. Shields will not work. We have only one real option if we want to build on the true legacy and meaning of education: to boldly go where accreditation has never gone before.

John C. Cavanaugh is Chancellor of the Pennsylvania State System of Higher Education. This essay is adapted from a speech he gave Tuesday at the annual meeting of the Middle States Commission on Higher Education.

CLA experiment focused private colleges' attention on assessment

Smart Title: 

Report finds that use of standardized measure of student learning drove curricular innovations at consortium of private colleges -- but produced little (so far) in measurable improvement.

18 Sanctioned by Southern Accreditor

Smart Title: 
Southern Association of Colleges and Schools places five colleges on probation, while 13 others are warned. Finances and inadequate numbers of full-time faculty draw scrutiny.

Pages

Subscribe to RSS - Assessment
Back to Top