Assessment

Improving Graduation Rates Is Job One at City Colleges of Chicago

Smart Title: 

City Colleges of Chicago have a 7 percent graduation rate. If that number doesn't go up, the system's chancellor, presidents and trustees could lose their jobs.

Awards recognize colleges that excel in assessing learning campuswide

Smart Title: 

Ten colleges earn national award for meaningful, cross-campus efforts to track student learning and use results to improve classroom practices.

IT think tank's call for alternative forms of credentialing and measuring competency

Smart Title: 

Technology think tank says standardized testing by outside groups and alternative forms of credentialing could create helpful competitive pressure on higher education and the traditional college degree.

WICHE's interstate passport seeks to help students transfer while preserving colleges' autonomy

Smart Title: 

New interstate network seeks to help students transfer across state lines without losing credits, but also defers to faculty members at each college about how to measure learning.

Defining disciplinary learning outcomes and principles for assessment (essay)

The higher education lore is that faculty members cannot agree on anything. Like other myths, this accepted folk wisdom is far from the truth. 

Indeed, over the course of our careers, we have repeatedly observed faculty members coming together collaboratively to address the challenges faced institutionally or in higher education more broadly. More recently, we have been heartened and inspired in particular by those who spent the last several years grappling with a fundamental question: what should students learn in higher education?

Instead of ignoring external pressures to measure and improve college outcomes, faculty members came together under the auspices of the Social Science Research Council's Measuring College Learning Project, which we have helped lead, to address these pedagogical challenges. Faculty members in six disciplines -- biology, business, communication, economics, history and sociology -- engaged in invigorating discussions, lively debates and difficult conversations. Supported by their disciplinary associations and encouraged by their collaborative spirit, they have articulated frameworks for defining learning outcomes in six disciplines and the principles for assessing learning outcomes in the 21st century, as described in the recently released Improving Quality in American Higher Education: Learning Outcomes and Assessments for the 21st Century.

In our work, we have found that faculty members readily agree that higher education is not about efficient acquisition of surface content knowledge and the simple regurgitation of memorized facts.That does not mean that content is unimportant. Content is indeed crucial, but primarily as a building block for more complex forms of thinking. Faculty members are eager to get students to apply, analyze and evaluate from their disciplinary perspectives, to acquire a disciplinary mindset and think like a biologist or an economist.

Faculty members across disciplines in the MCL project rather quickly coalesced around “essential concepts and competencies” for their disciplines, which represent ideas and skills that faculty believe are fundamental to the discipline, valuable to students and worth emphasizing given limited time and resources.There are similarities across disciplines including an emphasis on analytical writing and problem-solving, but these generic skills take form, are defined and are honed within specific fields of study. They are not abstract ideas, but concepts and competencies that faculty members engage, develop and deploy in their work and value in their disciplines.  

Faculty members are also often seen as resisting assessment. But, in fact, what they resist are simplistic assessments of student learning that focus on recollection of knowledge, rely on blunt instruments and are narrow and reductionist. They resist, as would all other professions, externally imposed mandates that fail to reflect the complexity of their jobs or that misrepresent the purpose of higher education. But they also believe that what they are doing makes a difference -- that they are teaching students how to see the world in a new light -- and they would be eager to have the tools to demonstrate their contributions to the development of student cognitive capacities.

Constructive conversations about learning outcomes and assessments require the proper context and frame.That is rarely offered in a world in which we in higher education are on the defensive, trying to argue against externally proposed accountability measures based on distal labor market outcomes, instead of being proactive and making the case on our own terms.There is no shortage of proposals in the public sphere about what higher education should do. But those conversations often lack the voices of faculty members, who are the professionals with responsibility for defining, enabling and assessing what students learn.

The faculty should be at the forefront of the conversations about the purposes of higher education and thus at the center of defining and measuring undergraduate learning outcomes.That is not only a matter of professional duty but also of doing justice to our students.Students from all backgrounds and institutions should have an opportunity to demonstrate their knowledge and skills.

Years of institutionally incentivized grade inflation and proliferation of course titles have all but made transcripts irrelevant. In our research, we found that most employers do not even ask to see them. And while some recent efforts have aimed to add extracurricular activities and other accomplishments to college transcripts, none of those tell us what students actually know or can do.Taking a class is not the same thing as mastering the concepts and competencies presented. Being a member of a club similarly says little about the skills a student has developed.

In addition to placing faculty and student learning at the center of the conversation, the MCL project is committed to recognizing the complexity of what higher education aims to accomplish and ensuring that any measure of learning is part of a larger holistic assessment plan.The project focuses on the disciplines.That does not preclude making sure that students are also civically minded and globally competent. It only means that we need to be clear about which part of the puzzle one hopes to address with a disciplinary focused initiative.

The MCL project is committed to ensuring that institutions use assessment tools on a voluntary basis. We have elaborated elsewhere the pitfalls of externally imposed accountability. Only by willingly looking in the mirror will higher education institutions make progress toward improving student learning outcomes.

While assessment should be voluntary, it need not be a solitary endeavor. Collaborating with other institutions makes us not only realize that we all face challenges and struggle with current circumstances but also offers insight into possible ways forward. Measures of learning outcomes must be of high quality and comparable, so they can allow multiple institutions to use them and share their insights. Governed by the principle of continuous improvement, assessments -- albeit limited and imperfect -- are necessary tools on the road toward reaching our goals.

As we look toward the future, we are excited and energized by the commitment and thoughtfulness of the faculty members who participated in the MCL project.They have put forth a bold and forward-thinking vision for the future of learning and assessment in their disciplines: a set of frameworks that will be subject to ongoing iteration and improvement in the years ahead. Instead of waiting for the storm to subside, these faculty members and their disciplinary associations have tackled the challenge head on. They have paved the way for a more promising future.

Josipa Roksa is associate professor of sociology and education at the University of Virginia. Richard Arum is chair of sociology at New York University and incoming dean of the School of Education at the University of California at Irvine. They are the authors of Academically Adrift: Limited Learning on College Campuses (University of Chicago, 2011).

Editorial Tags: 
Image Source: 
istock

Growing number of community colleges use multiple measures to place students

Smart Title: 

More community colleges are moving away from relying on placement exams alone to figure out whether incoming students need remediation, but establishing a substitute system can be tricky.

Series of studies seeks to gauge higher ed effectiveness, defined broadly

Smart Title: 

New volume of research examines various aspects of higher education performance, going well beyond labor market outcomes to include academic quality and socioeconomic equity.

Headlines About Gallup Survey on College Worth Are Misleading (essay)

It’s no secret that higher education in America is in a tight spot.

The cost and worth of college is a hot topic -- from dinner parties to political debates. The majority opinion is that college graduates are significantly dissatisfied with what they are receiving for the price of the “product” they receive.

Gallup released its most recent poll data of college and university alumni through its “Gallup-Purdue Index 2015 Report,” which is based on interviews with more than 30,000 graduates. This year, the survey included new questions concerning the “worth” of college. It’s time to step beyond anecdotal evidence and get our hands dirty with some data.

For those of us who fastidiously follow the headlines of Inside Higher Ed and The Chronicle of Higher Education, we initially found that all of our hand-wringing over how the public views higher education might be justified.

Inside Higher Ed led with: “Not Worth It?”

The Chronicle ran: “Just Half of Graduates Strongly Agree Their College Education Was Worth the Cost.” (Note: The article title was changed. The piece was originally entitled, “Just Half of Graduates Say Their College Education Was Worth the Cost.”)

And on Sept. 30, Jeffrey Selingo, former editor of The Chronicle, wrote a piece for The Washington Post entitled, “Is College Worth the Cost? Many Recent Graduates Don’t Think So.”

Yikes. The sky is falling, right?

Well, not really. Each of these headlines seems to insinuate that college grads are disgruntled by the cost of their education. However, if we read beyond the headlines, and take even a quick look at the numbers, we find that the sky isn’t falling.

In fact, maybe things are actually better than we imagined.

Gallup’s chart shows alumni responses to the statement: “My education from [university name] was worth the cost.” Respondents answered on a scale from one (strongly disagree) to five (strongly agree). While the headlines suggest that alumni are dissatisfied, I find myself reading these numbers differently.

Even if we assume that an answer of three (3) is indicative of “neutral,” we still find that 77 percent of recent alumni either agree or strongly agree with the statement that their college or university education was worth the cost.

I read the data this way: most grads believe that their education was worth the cost. That is good news. Even better news is that only 10 percent disagree or strongly disagree. Some additional good news is that, even though the recent graduates who participated in the survey were less likely to think their education was worth the cost, as they get farther and farther away from commencement -- as they are promoted out of entry-level positions -- their satisfaction regarding the cost of education will probably get better (as the Gallup report indicates).

The Gallup report includes significant data -- including factors that lead to student thriving.

But here is my real point: headlines matter.

In our current context bent on scrutinizing higher education, as we look ahead to report cards, and as we struggle to make a case for the import of this sector of society that has been educating citizens in America for nearly four centuries, let’s at least lead with more accurate headlines -- even if crisis sells.

Here’s what the headlines could have been:

“Is College Worth the Cost? Only 10 Percent of Grads Don’t Think So.”

Same numbers.

Entirely different story.

Keith R. Martel is director of the Master of Arts in Higher Education at Geneva College in Beaver Falls, Penn. He is the co-author of the newly released Storied Leadership, a faith-based, narrative approach to leadership.

Editorial Tags: 

The Waning of the Carnegie Unit (essay)

For a century, the Carnegie Unit -- or credit hour -- served American education very well. Created by the Carnegie Foundation for the Advancement of Teaching in 1906, it is now the nearly universal accounting unit for colleges and schools. It brought coherence and common standards to the chaotic 19th-century high school and college curriculum, established a measure for judging student academic progress, and set the requirements for high school graduation and college admission. But today it has grown outdated and less useful.

A time-based standard, one Carnegie Unit (or credit) is awarded for every 120 hours of class time. The foundation translated this into one hour of instruction five days a week for 24 weeks. Students have been expected to take four such courses a year for four years in high school, with a minimum of 14 Carnegie Units required for college admission. The Carnegie Unit perfectly mirrored its times and the design of the nation’s schools.

An industrialized America created schools modeled on the technology of the times: the assembly line. With the Carnegie Unit as a basis, schools nationwide adopted a common process for schooling groups of children, sorted by age for 13 years, 180 days a year in Carnegie unit-length courses. Students progressed according to seat time -- how long they were exposed to teaching.

At colleges and universities across the nation, the Carnegie Unit became more commonly referred to as the credit hour. The common semester-long class became three credit hours. The average four-year degree was earned after completing 120 credit hours. Time and process were fixed, and outcomes of schooling were variable. All students were expected to learn the same things in the same period of time. The Carnegie Unit provided the architecture to make this system work.

But in the United States’ transition from an industrial to an information economy, the Carnegie Unit is becoming obsolete. The information economy focuses on common, fixed outcomes, yet the process and the time necessary to achieve them are variable. The concern in colleges and schools is shifting from teaching to learning -- what students know and can do, not how long they are taught. Education at all levels is becoming more individualized, as students learn different subjects at different rates and learn best using different methods of instruction.

As a result, educational institutions need a new accounting to replace the Carnegie Unit. A 2015 report by the Carnegie Foundation made this clear, stating the Carnegie Unit “sought to standardize students’ exposure to subject material by ensuring they received consistent amounts of instructional time. It was never intended to function as a measure of what students learned.” States have responded by adopting outcome- or learning-based standards for schools. They are now detailing the skills and knowledge students must attain to graduate and implementing testing regimens, such as fourth- and eighth-grade reading and math exams, to assess whether students have met those standards, testing regimens to assess student progress and attainment of outcomes.

This evolution is causing two problems. First, both the industrial and information economy models of education are being imposed on our educational institutions at the same time. At the moment, the effect is more apparent in our schools than colleges, but higher education can expect to face the same challenges. Today, schools and colleges are being required to use the fixed-process, fixed-calendar and Carnegie Unit accounting system of the industrial era. They are also being required to achieve the information economy’s fixed outcomes and follow its testing procedures. The former is true of higher education, and government is increasingly asking colleges and universities for the latter.

Doing both is not possible, by definition. Instead, states need to move consciously and systematically to the information economy’s emerging and increasingly dominant model of education, which will prevail in the future. The Carnegie Unit will pass into history.

The second problem is that the steps states have taken to implement standards, outcomes and associated testing are often incomplete and unfinished. They are at best betas quickly planned and hurriedly implemented, which like all new initiatives demand significant rethinking, redesign and refinement. In the decades to come, today's tests will appear primitive by comparison to the assessment tools that replace them. Think of the earliest cell phones -- they needed development and refinement.

Unfortunately, however, states’ mandates go beyond the capacity and capabilities of their standards, tests, data systems and existing curricula. For example, despite growing state and federal pressure to evaluate faculty and institutions based on student performance, most states do not have the data or data systems to make this possible.

If Information Age accounting systems for education are to work as well as the Carnegie Unit did, the tasks ahead are these:

  • Define the outcomes or standards students need to achieve to graduate from school and college. While the specific outcomes or standards adopted are likely to vary from state to state, the meaning of each standard or outcome should be common to all states. A current example is coding. Today states, cities and institutions differ profoundly in their requirements in this area, however, it is essential that the meaning of competence in this area be common.
  • Create curricula that mirror each standard and that permit students to advance according to mastery.
  • Develop assessments that measure student progress and attainment of standards or outcomes. Over time, build upon current initiatives in analytics and adaptive learning, to embed assessment into curricula to function like a GPS, discovering students’ misunderstandings in real time and providing guidance to get them back on track.

These three key steps will lay the groundwork for the education demanded by the Information Age. They will provide the clarity, specificity, standardization, reliability and adoptability that made the Carnegie Unit successful. It will create an educational accounting system for the information economy that is as strong as the Carnegie Unit was for industrial America.

I do not pretend doing this will be easy or quick. It is nothing less than the reinvention of the American education system. It will require bold institutions to lead, as universities like Carnegie Mellon University, the Massachusetts Institute of Technology, Southern New Hampshire University and Western Governors University are doing, to create and test the new models of education for the Information Age. It will take a coalition of state government, educational institutions and professional associations like accreditors to turn the innovations into policy.

We don't have the luxury of turning away from this challenge. Our education system is not working. In contrast to the industrial era, in which national success rested on physical labor and natural resources, information economies require brains and knowledge. The future demands excellent schools and colleges.

Arthur Levine is the president of the Woodrow Wilson National Fellowship Foundation in Princeton, N.J. He served as the president of Teachers College, Columbia University, from 1994 to 2006.

Editorial Tags: 

Nonacademic skills test from ETS fills in blanks on student's likelihood of success

Smart Title: 

Colleges are using a nonacademic skills test from ETS to try to boost graduation rates and in remedial course placement. One university gives the test to all its athletes.

Pages

Subscribe to RSS - Assessment
Back to Top