News, Views and Careers for All of Higher Education
April 3, 2006
An association of public research universities is studying the possibility of creating a new, voluntary system to define and measure the outcomes of undergraduate education, in a way that would allow for public comparisons of similar institutions.
The National Association of State Universities and Land-Grant Colleges — a group of more than 200 public research universities, including flagship institutions in every state — is floating this idea as a federal panel on the future of higher education has become increasingly critical of the way the government and colleges assure their quality. Leaders of the Secretary of Education’s Commission on the Future of Higher Education have for months been scaring colleges with talk of using new national tests to assure quality. And just last week, the panel infuriated accreditors when it released a report suggesting that the regional system of accrediting colleges be replaced with a national one.
While college leaders have acknowledged that they could do better at measuring what they do to educate students, they have done so largely by suggesting tweaks or improvements in current systems of oversight. The idea being considered by the land-grant group has the potential to more dramatically change the way colleges are evaluated and to change the dynamics of the way higher education is responding to the federal commission.
The group has released few details about how a new system might work, but is expected to present ideas about it to the federal commission as early as this week, when the panel meets in Indianapolis. And while the commission is clearly putting this issue on the universities’ agenda, an explicit role of the universities’ effort is to make sure that any new system for evaluating public higher education is developed “within and shaped by” those institutions, rather than imposed by federal authorities, according to a letter to NASULGC’s presidents recently sent by Peter McPherson, the new president of the association and the former president of Michigan State University.
McPherson’s letter said that the association’s Council on Academic Affairs (the provosts of the group’s member institutions) had already been discussing the idea and agreed that it was time to formally consider creating such a system. A statement adopted by that council said the system would work at “defining outcomes and contributions, by type of classification of university.”
Further, the statement said: “It is apparent that there are opportunities to approach accountability in a number of ways including defining and communicating the different roles of universities that are relevant to parents and students, to the community boards, to states and at the national level.”
A group of university leaders — likely also including members of the American Association of State Colleges and Universities — will soon be appointed to flesh out the idea.
In an interview, McPherson said that any system created would be focused on “outcomes,” but he said that there would not just be a focus on any single measure, such as graduation rates. He also said it was important that those outcomes consider the skills and knowledge that students bring to college — so institutions aren’t just judged superior for being able to admit better prepared undergraduates.
He also said any new system should be easily understandable by people who are not immersed in higher education. There is an important role for reports from accreditors and others that are “more nuanced” and “less easily communicated” than the kind of system his association may help create, he said. But he said college leaders need to get over the idea that just because they need such reports, they can ignore public demands for information that they can understand.
By failing to come up with good systems, he said, people end up relying on flawed alternatives like the rankings of U.S. News & World report. “We’ve got measurements out there, but they are ones with which we have vehement disagreements,” he said.
Any new system will work only if colleges approach it with the right attitude, McPherson said. It is true, he said, that American colleges and universities are the best in the world. But that doesn’t negate other realities: that “we’ve got some real problems.” The first step to solving problems, he said, “is to state them,” and the new system the association is talking about could create both public and internal pressure to solve problems once they are identified.
Sally Mason, provost of Purdue University and chair of the NASULGC group that has worked on this idea, said that she sees the idea as a natural outgrowth of how she and her counterparts at other institutions are constantly considering how they can improve programs and their institutions. Such efforts might have more impact if they were part of an entity or construct that crossed campus lines, she said.
“We need to be able to convey to the public at large that we are seriously engaged in the pursuit of quality education, efficiency in our use of resources, and accessibility to all students who are prepared and willing to earn a college degree,” Mason said.
Mason said she didn’t see the possible new system duplicating the work of accreditors. Specialized accreditors focus on specific areas, she said, while accountability issues are “a part of the work of [regional] accrediting agencies” but “are not their only focus.”
A range of metrics — already collected in some form by most institutions — could be used in the system that universities may create, she said. Mason cited data on graduation rates, admissions, applications, student demographics and faculty demographics. She said that “local context’ remained important for considering “why our measurements might be different from those of selected peer institutions,” but she said it should still be possible to make comparisons. “We can and do use metrics to set goals for improvement,” she said. “What we hope to achieve in our work is a strong message regarding our commitment to accountability and the mechanisms that each of us will use to achieve greater accountability.”
Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.
Advertisement
Dr. McPherson and NASULGC are to be strongly commended for their leadership in developing an accountability system appropriate for the academic institutions involved and also for the students and the public. This effort has obviously been underway for some time. The public will surely be served even more as we are able to watch its development.
Charles Miller, Chairman at Commission on the Future of Higher Education, at 9:15 am EDT on April 3, 2006
I am sure that the higher education industry’s decission to continue to use adjunct professors at increasing levels will take care of the poor education that students are receiving. All we need is more standardized tests and everything will magically improve, then our industry can take the next logical step and fire all remaining faculty and we can move to online classes taught from professors based in Hong Kong and India.
Dick Shoenthal, HR Uberlord at Landgrant University, at 10:10 am EDT on April 3, 2006
Many of today’s college students can’t perform at a college level. They don’t have (and probably will never have) the abstract reasoning skills necessary for college as it was in the 1950s. Someone said, here or elsewhere, that most today don’t take the college prep track in high school. You know, four years of math through trig and solid geometry, four years of science, four years of English, three or four years of a foreign language.
As someone else said, you don’t need college grad level skills to work as a management trainee at Enterprise Rent a Car, but Enterprise wants college grads anyway. Why? I’m not sure.
Most of us state college folks are really running post high-school finishing school. I don’t really think we are failing to do anything, given the students we have to work with. Our grads can do a modicum of research usually and write a report. They can probably program a spreadsheet rudimentarily, or can learn to anyway.
To a great extent, higher education is now a Ponzi scheme. Colleges compete over enrollments. They enroll students who could never have gone to college in the 1950s. The students go into hock. Some get jobs based on their degrees, and try to pay their lenders back. Others cannot. Employers get a slightly better socialized work force for information economy jobs, and people waiting for work don’t riot because they’re in school.
T BD, at 1:10 pm EDT on April 3, 2006
The reporting infrastructure already exists within the U.S. Department of Education to help hold schools accountable. All postsecondary schools are legally required to use the Integrated Postsecondary Educational Data System (IPEDS) to report many data elements such as completion rates, number of faculty and support staff, etc. This system could be easily tweaked to include reporting on national test results if that is what the nation decides is needed.
The one data element that IPEDS does not collect is the Cost of Attendance (COA). This figure includes tuition, fees, and books, AND all other living expenses that most students incur. It is the most comprehensive measure of the tru cost at most schools, yet it is not mandated as a reporting element to IPEDS, or anywhere else.
COA probably never will be required reporting because it would cause true sticker shock in the American public. Schools like Yale and Penn would come in about $50 to $60,000, and even small specialty schools can top $40,000 COA. If the feds really want to do something “student friendly, they’d force school to disclose COA on their websites and in their recruitment materials.
feudi pandola, at 1:30 pm EDT on April 3, 2006
Have truer words ever been written?
” .. To a great extent, higher education is now a Ponzi scheme. Colleges compete over enrollments. They enroll students who could never have gone to college in the 1950s ..”
That, as contrasted to the specious claim that adjuncts decrease quality (most of my best instructors were adjuncts), has been a rare moment of clarity.
There are too many four-year colleges. There needs to be fewer, to reduce student debt and public tax burdens. This is not about political incumbents’ re-elections, or faculty unions and wages — it is about what student debt and runaway college costs are doing to Generation Y.
B.J., at 1:40 pm EDT on April 3, 2006
Interesting. So now we may get serious about assessment of entering strengths (ACT, SAT, can bench press 440, 4.85 50 yard spring, excellent evaluations as MacBurger’s Asst. Mgr. position) to actual evaluation of what the college/university was able to do with those incoming resources at what cost?
Student “A” — stellar student at Podunk H.S., ACT of 25 (combined), works full time, makes it through State University with a 3.5 GPA, and has a combined GRE score of 1450. Cost $54,000 (including room & board that “A” would have had to pay anyway) over 9 semesters. “A” is now a regional mgr. for McBurger’s pulling down $68,000.
Student “B” — average student at Status Posh H.S., ACT of 27 (combined), no job, no athletic talents, on yearbook staff (no leadership responsibilities), GPA of 4.50. “B” makes it through Polished Ivy University in 12 semesters, no outside job, Asst. Ed. of school literary magazine for two semesters (staff writer for 3 semesters), GPA of 4.00, GRE combined score of 1725, and it only cost $180,000 (HEY, “B” was in a fraternity), and landed a job doing food reviews for La Gourmand (grease by the gallon) owned by “B"’s family.
Student “C” was home-schooled, no school activities except summer softball in the part, ACT combined score of 30, no outside employment but helps a great deal around the house. Leads church youth group (ooooo can we even consider that?) “C” zooms through undergrad in 3 years, 4.5 GPA (yes, on a 4.0 scale), GRE of 1750, high enough GMAT to be accepted into three top-tier medical schools. Cost $11,380 (oh yes, first three semesters at Jr. College and next three at Little State College, and “C” lived at home).
Now which student got the most for his or her (or Uncle Sam’s) money?
Which institution increased the level of these students’ critical thinking?
Which of these students could have possibly prepared for professional or graduate school on his or her own?
Who SHOULD have been admitted to Polished Ivy University instead of Student “B” ?
If “B” attended State U. would he/she have done as well as the outcome from Polished Silver Spoon in the Mouth Ivy U. ?
If “C” attended Polished Ivy University might it take him/her more or less time to complete the undergraduate degree (with all those distractions & an idiot roommate) and would “C"’s work have been objectively evaluated by his/her radial to the left of liberal professors?
Dr. F. Gump, at 2:35 pm EDT on April 3, 2006
The most important things to measure are the actual student learning outcomes, the skills and knowledge gained by students. Retention and graduation rates, cost of attendance, faculty credentials, even subsequent job placement and earnings are not student learning outcomes.
Essentially, assessment starts with stating your goals, defining appropriate activities for students to engage in reaching those goals, measuring the outcomes, making changes as appropriate, and continuing this cycle. While some faculty insist that the most important skills, such as critical thinking, cannot be measured, they will also assert they “know it when they see it” and point to the grades they give. The challenge is to change the focus from evaluating students to assessing effectiveness of the program (i.e. learning activities). We should then be able to explain, in simple terms parents and even politicians can understand, that we know what we are doing, are doing a good job, and have the evidence to prove it.
A. George, Dir Ir and Assessment at U of Idaho, at 4:10 pm EDT on April 3, 2006
I’d like to ask a question of A. George. It appears from his or her comment that students are an inert mass, without agency and without responsibility for their education. How, consequently, does one take into account a recalcitrant student? If students do poorly in a class, or on the “assessment” of that class, how much is that the fault of the professor, and how much the student? And why should the professor, or the institution, be held liable for students who refuse to come to class, do not do the reading, and do not do the assignments?
Peter C. Herman, Prof. at SDSU, at 6:05 pm EDT on April 3, 2006
” .. why should the professor, or the institution, be held liable for students who refuse to come to class ..”
There is a growing number of academic departments and colleges that initiate “automatic drops” for students who do not regularly attend class. That is, students are required to “swish” their mag-tape IDs when entering the classroom. A number of consecutive abscences, and the student is automatically dropped from the course and a penalty fee assessed (because they prevented another student from taking their place).
If someone doesn’t want to cooperate, and just pick up a paycheck, there are a million ways to passively-aggressively obstruct assessment. By comparison, solutions are relatively easy to make, if a productive solution is really the goal.
A.D., at 10:35 pm EDT on April 3, 2006
I’m disappointed that NASULGC is seems to be interested in describing “the skills and knowledge that students bring to college” but evidently not the ones they leave with. While it’s considering publishing “data on graduation rates, admissions, applications, student demographics and faculty demographics,” the obvious ommission of evidence of student learning suggests that they haven’t heard the clear message that’s been coming from the states and the feds for about 20 years: that they want to know what our graduates know and can do.
Peg Miller, Director, National Forum on College-Level Learning, at 10:00 am EDT on April 4, 2006
One post claimed that the Feds do not collect data on cost of attendance. This is simply untrue. The IPEDS Institutional Characteristics report, Pard D (Student Charges — Price of Attendance) clearly requires any college participating in Title IV programs to report this data (tuition & fees; books & supplies; room & board; and other off campus expenses) each year. The data for any college can be found at the NCES/IPEDS website: http://nces.ed.gov/ipeds/
Steve, Institutional Researcher at Roanoke-Chowan CC, at 10:55 am EDT on April 7, 2006
I stand partially corrected on the IPEDS reporting of the Cost of Attendance. It is collected for four year schools, but at my three year nursing program, the IPEDS Part D collects only the average tuition and fees. It does not ask anything at all about other costs and living expenses which can double the cost of attendance.
There is, however, no legal requirement that schools disclose their cost of attendance on their websites or in their recruitment literature. A google search of many, many schools showed that most do not report this figure anywhere on their websites. Publishing the COA should be required as schools strive towards transparency with the public.
feudi pandola, at 2:25 pm EDT on April 13, 2006
Advertisement
or search for jobs directly.
Job Summary The individual will assist in the successful implementation of the programmatic activities and ... see job
Appalachian College Association Berea, Kentucky Vice President for Academic Programs The Appalachian College Association, one ... see job
Columbus State Community College invests in employee development by providing numerous resources, partnerships, training and ... see job
South Texas College Job Announcement # 2010 – 3010 Position pending Board approval of FY10 Budget Please note that only ... see job
Posting Description: The Department of Physical Medicine and Rehabilitation is looking for a full-time or ... see job
SYSTEMS ADMINISTRATOR Hofstra University seeks a Systems Administrator responsible for all operational aspects of the ... see job
Prince George’s Community College is a learning-centered, comprehensive, two year college located in the Washington, DC ... see job
The University is located in Southwest Florida midway between Naples and Fort Myers. The 760-acre campus is situated in one ... see job
Georgia Gwinnett College, the 35th member of the University System of Georgia, is a premier 21st century four-year liberal ... see job
The nation’s first university, Penn is a world-renowned leader in education, research, and innovation. Situated on a ... see job
Catching up with Derek Bok?
Peter McPherson has a strong reputation for managing the chaos that is academia. He and others have obviously read the new book by Derek Bok, “Our Underachieving Colleges” —
http://www.goactablog.org/blog/archives/2006/02/derek_boks_new.html
and realized something real and significant has to be done ASAP. Good luck.
A.D., at 6:15 am EDT on April 3, 2006