A central tenet of the student learning outcomes "movement" is that higher education institutions must articulate a specific set of skills, traits and/or dispositions that all of its students will learn before graduation. Then, through legitimate means of measurement, institutions must assess and publicize the degree to which its students make gains on each of these outcomes.
Although many institutions have yet to implement this concept fully (especially regarding the thorough assessment of institutional outcomes), this idea is more than just a suggestion. Each of the regional accrediting bodies now requires institutions to identify specific learning outcomes and demonstrate evidence of outcomes assessment as a standard of practice.
This approach to educational design seems at the very least reasonable. All students, regardless of major, need a certain set of skills and aptitudes (things like critical thinking, collaborative leadership, intercultural competence) to succeed in life as they take on additional professional responsibilities, embark (by choice or by circumstance) on a new career, or address a daunting civic or personal challenge. In light of the educational mission our institutions espouse, committing ourselves to a set of learning outcomes for all students seems like what we should have been doing all along.
Yet too often the outcomes that institutions select to represent the full scope of their educational mission, and the way that those institutions choose to assess gains on those outcomes, unwittingly limit their ability to fulfill the mission they espouse. For when institutions narrow their educational vision to a discrete set of skills and dispositions that can be presented, performed or produced at the end of an undergraduate assembly line, they often do so at the expense of their own broader vision that would cultivate in students a self-sustaining approach to learning. What we measure dictates the focus of our efforts to improve.
As such, it’s easy to imagine a scenario in which the educational structure that currently produces majors and minors in content areas is simply replaced by one that produces majors and minors in some newly chosen learning outcomes. Instead of redesigning the college learning experience to alter the lifetime trajectory of an individual, we allow the whole to be nothing more than the sum of the parts -- because all we have done is swap one collection of parts for another. Although there may be value in establishing and implementing a threshold of competence for a bachelor’s degree (for which a major serves a legitimate purpose), limiting ourselves to this framework fails to account for the deeply held belief that a college experience should approach learning as a process -- one that is cumulative, iterative, multidimensional and, most importantly, self-sustaining long beyond graduation.
The disconnect between our conception of a college education as a process and our tendency to track learning as a finite set of productions (outcomes) is particularly apparent in the way that we assess our students’ development as lifelong learners. Typically, we measure this construct with a pre-test and a post-test that tracks learning gains between the years of 18 and 22 -- hardly a lifetime (the fact that a few institutions gather data from alumni 5 and 10 years after graduation doesn’t invalidate the larger point).
Under these conditions, trying to claim empirically that (1) an individual has developed and maintained a perpetual interest in learning throughout their life, and that (2) this lifelong approach is directly attributable to one’s undergraduate education probably borders on the delusional. The complexity of life even under the most mundane of circumstances makes such a hypothesis deeply suspect. Yet we all know of students that experienced college as a process through which they found a direction that excited them and a momentum that carried them down a purposeful path that extended far beyond commencement.
I am by no means suggesting that institutions should abandon assessing learning gains on a given set of outcomes. On the contrary, we should expect no less of ourselves than substantial growth in all of our students as a result of our efforts. Designed appropriately, a well-organized sequence of outcomes assessment snapshots can provide information vital to tracking student learning over time and potentially increasing institutional effectiveness. However, because the very act of learning occurs (as the seminal developmental psychologist Lev Vygotsky would describe it) in a state of perpetual social interaction, taking stock of the degree to which we foster a robust learning process is at least as important as taking snapshots of learning outcomes if we hope to gather information that helps us improve.
If you think that assessing learning outcomes effectively is difficult, then assessing the quality of the learning process ought to send chills down even the most skilled assessment coordinator’s spine. Defining and measuring the nature of process requires a very different conception of assessment – and for that matter a substantially more complex understanding of learning outcomes.
Instead of merely measuring what is already in the rearview mirror (i.e., whatever has already been acquired), assessing the college experience as a process requires a look at the road ahead, emphasizing the connection between what has already occurred and what is yet to come. In other words, assessment of the learning that results from a given experience would include the degree to which a student is prepared or “primed” to make the most of a future learning experience (either one that is intentionally designed to follow immediately, or one that is likely to occur somewhere down the road). Ultimately, this approach would substantially improve our ability to determine the degree to which we are preparing students to approach life in a way that is thoughtful, pro-actively adaptable, and even nimble in the face of both unforeseen opportunity and sudden disappointment.
Of course, this idea runs counter to the way that we typically organize our students’ postsecondary educational experience. For if we are going to track the degree to which a given experience “primes” students for subsequent experiences -- especially subsequent experiences that occur during college -- then the educational experience can’t be so loosely constructed that the number of potential variations in the order of a student experiences virtually equals the number of students enrolled at our institution.
This doesn’t mean that we return to the days in which every student took the same courses at the same time in the same order, but it does require an increased level of collective commitment to the intentional design of the student experience, a commitment to student-centered learning that will likely come at the expense of an individual instructor’s or administrator’s preference for which courses they teach or programs they lead and when they might be offered.
The other serious challenge is the act of operationalizing a concept of assessment that attempts to directly measure an individual’s preparation to make the most of a subsequent educational experience. But if we want to demonstrate the degree to which a college experience is more than just a collection of gains on disparate outcomes – whether these outcomes are somehow connected or entirely independent of each other – then we have to expand our approach to include process as well as product.
Only then can we actually demonstrate that the whole is greater than the sum of the parts, that in fact the educational process is the glue that fuses those disparate parts into a greater -- and qualitatively distinct -- whole.
Mark Salisbury is director of institutional research and assessment at Augustana College, in Illinois. He blogs at Delicious Ambiguity.
In many respects, higher education in the United States – with credits awarded on time a student sits in a chair – remains trapped in the 19th century and has been slow to embrace technology.
Online education from traditionally accredited colleges has been available since at least 1999, but almost always at the same high tuition cost as the traditional “physical” courses. New ideas, such as tuition-free massive open online courses (MOOCs), are now emerging, but are generally not accredited.
For a true revolution to occur, regulation will need to change along with the technology. The key advance would be to establish a new private sector accrediting body, the “Modern States Accrediting Agency,” that would ensure the quality and reputation of the innovative courses, make the credits transferable into the traditional system and which would be recognized by the U.S. Department of Education as an approved accreditor in order to qualify students for federal student aid.
The Department of Education began its first online education pilot program in 1999. In 2006, it allowed institutions to offer all of their courses online. However, these courses were offered by institutions accredited in the traditional way, with student enrollment in the courses kept limited, and with tuition set as high (or even higher) than tuition for the physical alternative.
The paradigm began to shift in 2011 when Stanford University offered three of its courses online, free of charge, to any person anywhere who chose to take them.
Since then other innovators have continued to introduce MOOCs, notably led by the nation’s most respected traditional universities, such as MIT and Harvard. MIT, for example, is creating MITx, which is intended to offer a great number of MIT’s courses free of charge or nearly so; taught by MIT’s renowned faculty; and with graded assignments, tests, online discussion groups, online professor “office hours” and other quality advances.
The problem with MOOCs, though, is that there is usually no mechanism for obtaining accreditation and, in U.S. higher education, accreditation is the “coin of the realm,” which gives a degree its value. As a result, most MOOCs can offer students only a letter of completion, a pat on the head and no degree. Few other institutions or graduate schools will recognize completion of a MOOC course for credit, and employers do not know how to judge the student’s level of accomplishment.
Schools like MIT should not be forced to dilute the power of their brand by being forced to give their regular degree to students who simply take some of their tuition-free online courses. However, it is equally inappropriate to give no value to the online learning that occurs in a MOOC, particularly if a student can complete a high-quality, rigorous course and then prove mastery of the material on a separate, proctored, certifying exam.
In the traditional system, a degree is accredited because the degree-granting institution is itself accredited by an agency recognized by the U.S. Secretary of Education. The accrediting agencies (such as the Middle States Commission on Higher Education or the New England Association of Schools and Colleges) are private sector, self-regulatory groups which, in most cases, were created nearly a century ago by the member institutions themselves. The best of these agencies were later recognized by the Department of Education and included on its list of approved accreditors.
Today students can only qualify for federal financial aid if their institution has been accredited by one of these recognized agencies, and these accreditors’ decisions control access to the more than $150 billion in federal aid paid out to students each year. The traditional accrediting agencies, which were founded long ago to serve the needs of the traditional institutions, are not well-suited to lead technological and social innovations that are alternatives to the traditional system. A few experiments with traditionally accredited MOOCs are under way.
However, for the most rapid and effective progress, America needs a new, innovation-focused accreditor, Modern States, which would also be recognized by the Department of Education and which could accredit providers of emerging technologies and ideas in order to drive down costs, drive up quality and to shape federal aid programs in new and effective ways.
Unlike traditional accreditors, Modern States would be able to accredit specific courses, not just the degree-granting institution as a whole. For example, it could recognize that the freshman physics MOOC from MITx is of high quality, and then develop a widely available, proctored test for students who complete that course, similar to an SAT exam or CPA exam.
Students who complete the preapproved, tuition-free MOOC and also pass the confirmatory Modern States assessment would earn accredited course hours from Modern States itself. Enough such courses in the right scope and sequence (say physics from MITx, poetry from Harvard, theology from Notre Dame and so on) could lead to a fully accredited Modern States degree. Modern States would also approve courses and develop tests in vocational areas, in career training fields and at the two-year and community-college level, in order to serve all types of students.
The creation of Modern States could then enable a whole field of academic innovations to bloom, including blends of “bricks and clicks” and new types of federal financial aid models. For example, students might take their core lectures tuition-free and online from a nationally renowned professor in a MOOC, and then attend supplementary weekly study groups with a live professor and other students in their home towns, all at a lower overall cost than a traditional course today.
Similarly, students might take their first year of introductory courses all online for free, but then transfer to a traditional college for the last three years, lowering their total educational costs by 25 percent. Students who complete their educations at a low cost or no cost to the federal government might even be paid a federal bonus upon completing their degree and successfully entering the work force. In this way, the self-motivated Abe Lincolns of tomorrow could finish their higher educations debt-free and with cash in the bank. Meanwhile, U.S. taxpayers would save money.
Modern States could also lead the way in areas unrelated to MOOCs, such as competency-based exams and on-the-job skills training. For example, if there is a national shortage of skilled welders, and if an employer trains a worker in welding who then passes the Modern States assessment, the worker could earn a course credit in welding while the employer itself might be paid some stipend as an “educational institution of one.”
The traditional accreditors were founded by their member institutions, but – with a few exceptions – traditional institutions are not likely to be the best champions for low-cost alternatives to themselves. Modern States should be chiefly formed by a voluntary association of philanthropies and nongovernmental organizations concerned with increasing access to high-quality education while lowering its cost -- groups such as the Bill and Melinda Gates Foundation, the Ford Foundation, the World Bank and so on. Employer and labor groups could join as well, as could providers of the innovative courses, student consumer groups and others.
Membership contributions could fund a staff that would develop a starting catalog of approved courses from the already-existing universe of MOOC and related offerings. The staff would then work with testing organizations to develop the independent assessments needed to prove student mastery of the material. The catalog would cover a range of academic and vocational fields, and grow and evolve over time.
The ultimate path to success for Modern States would be to keep its testing standards high and rigorous so that employers, traditional institutions and the world at large will see that the students are truly deserving of the degree credits. In this way, Modern States could function like a universal version of the CPA exam developers, who have become an accepted standard of testing and quality in the accounting field.
Once Modern States is formed, it would write criteria for granting accreditation that would be aligned with the Secretary of Education’s criteria for recognition of accrediting agencies. After applying these criteria, Modern States could petition the Department of Education for recognition. The petition would be reviewed by staff and by a federal advisory committee that advises the Secretary on whether to recognize accreditors. If the Department grants recognition, then students at the institutions and programs accredited by Modern States would be eligible to participate in federal student aid programs.
Transferability of credits to more traditionally accredited programs would be negotiated by Modern States through reciprocity agreements with other accreditors, as supported by the Department of Education. Modern States would be a private-sector organization, not a government organization. However, political leaders in both parties could help achieve educational goals by expressing clear support for the Modern States approach as outlined here.
In its best form, traditional higher education is one of America’s great treasures, and no online program is ever likely to equal the experience of four years on campus at a great school. However, the high cost and limited availability of such traditional best-in-class programs have placed them increasingly out of reach to many striving students in America and around the world. By unleashing the power of technology and social innovation, Modern States could be the key regulatory mechanism to make education more accessible and affordable, and to bring higher education more fully into the 21st century.
David Bergeron and Steven Klinsky
David Bergeron, the former acting assistant secretary for postsecondary education at the U.S. Department of Education, is vice president of postsecondary education policy at the Center for American Progress.
Steven Klinsky, a New York-based businessman and philanthropist, has been active in education reform since 1993.
With great interest, I read the recent news announcing that the American Council on Education (ACE) had evaluated five Coursera MOOCs and recommended them for credit. But I had hoped for something different.
Having traditional prestigious institutions making their online content open to the world – of course without their prestigious credit attached – was an exciting development. A race to post courses ensued. On the surface, it’s an altruistic move to make learning available to anyone, anywhere for free.
Dig deeper and we are left to ask, how many MOOC courses will really be worth college credit, where will the credits be accepted, and for how long will college credits even be the primary measurement of learning?
Now that ACE has evaluated a few courses, MOOC providers will see how their process goes as students start actually finding proctors and taking tests -- or finding other methods of assessment -- to prove they learned the material. But a few courses will not be enough to really help students earn degrees, and with MOOC courses and providers continuing to proliferate, this does not seem like a viable way to keep up with demand.
Regardless, it is more than likely that the universities that agreed to the ACE CREDIT review are never going to accept an ACE CREDIT transcript themselves. The students with ACE CREDIT transcripts will need to present those transcripts to “lesser known” schools that are not among the elite players – colleges with much lower tuition and a willingness to serve post-traditional students.
More troubling is the fact that the ACE process for credit review is still course-based. Will this really be flexible enough in the future? Will it measure competencies and individual learning outcomes? Even if it seems scalable, will it mean all MOOC evaluations have to run through ACE and only ACE? Will students have to wait until ACE has evaluated a MOOC course before they can get credit?
Moreover, this raises the question: Are course evaluations and testing really the best or only way to deal with this new era of learning? What about experiential learning? If someone has college-level learning from their life experience is it invalid unless they take a course?
As Inside Higher Ed points out in its article, this was a fast move in an industry that moves at a glacial pace. But when ice really begins to melt, it can quickly turn into a waterfall. Students have more options for learning, and can get more information, from a variety of sources. So the question for education becomes, how can we best accommodate that?
I would assert that a portfolio assessment of students’ learning is the best way. Just as an artist shows a portfolio to a prospective employer, students should be able to demonstrate learning from wherever they have learned -- work, MOOCs, informal training, military service, volunteer service, and more -- all in one place. And much of this learning will not involve a course at all.
If MOOCs are to be truly disruptive, they must link to competencies, credentials, degrees and/or ultimately jobs. Using a course-by-course, credit hour-by-credit hour approach to do this will not dramatically change the way people earn degrees. And dramatic change that allows for individual demonstrations of competencies is the only way to provide the education quality and agility necessary to truly recognize learning derived from free resources on the web. By focusing on competencies, we can align and accept learning experiences from everywhere.
Pamela Tate is president/CEO of the Council for Adult and Experiential Learning.
Historians of this period, possessing the clearsightedness that only time provides, will likely point to online learning as the disruptive technology platform that radically changed higher education, which had remained largely unchanged since the cathedral schools of medieval Europe -- football, beer pong and food courts notwithstanding.
Online learning is already well-understood, well-established and well-respected by those who genuinely know it. But what we now see in higher education is a new wave of innovation that uses online learning, or at least aspects of it, as a starting point. The meteoric growth of the for-profit sector, the emergence of MOOCs, new self-paced competency-based programs, adaptive learning environments, peer-to-peer learning platforms, third-party service providers, the end of geographic limitations on program delivery and more all spring from the maturation of online learning and the technology that supports it. Online learning has provided a platform for rethinking delivery models and much of accreditation is not designed to account for these new approaches.
Until now, regional accreditation has been based on a review of an integrated organization and its activities: the college or university. These were largely cohesive and relatively easy to understand organizational structures where almost everything was integrated to produce the learning experience and degree. Accreditation is now faced with assessing learning in an increasingly disaggregated world with organizations that are increasingly complex, or at least differently complex, including shifting roles, new stakeholders and participants, various contractual obligations and relationships, and new delivery models. There is likely to be increasing pressure for accreditation to move from looking only at the overall whole, the institution, to include smaller parts within the whole or alternatives to the whole: perhaps programs, providers and offerings other than degrees and maybe provided by entities other than traditional institutions. In other words, in an increasingly disaggregated world does accreditation need to become more disaggregated as well?
Take the emergence of competency-based education, which is more profound – if less discussed – than massive open online courses (MOOCs). Our own competency-based program, College for America (CfA), is the first of its kind to so wholly move from any anchoring to the three-credit hour Carnegie Unit that pervades higher education (shaping workload, units of learning, resource allocation, space utilization, salary structures, financial aid regulations, transfer policies, degree definitions and more). The irony of the three-credit hour is that it fixes time while it leaves variable the actual learning. In other words, we are really good at telling the world how long students have sat at their desks and we are really quite poor at saying how much they have learned or even what they learned. Competency-based education flips the relationship and says let time be variable, but make learning well-defined, fixed and non-negotiable.
In our CfA program, there are no courses. There are 120 competencies – “can do” statements, if you will – precisely defined by well-developed rubrics. Students demonstrate mastery of those competencies through completion of “tasks” that are then assessed by faculty reviewers using the rubrics. Students can’t “slide by” with a C or a B; they have either mastered the competencies or they are still working on them. When they are successful, the assessments are maintained in a web-based portfolio as evidence of learning. Students can begin with any competency at any level (there are three levels moving from smaller, simpler competencies to higher level, complicated competencies) and go as fast or as slow as they need to be successful. We offer the degree for $2,500 per year, so an associate degree for $5,000 if a student takes two years and for as little as $1,250 if they complete in just six months (an admittedly formidable task for most). CfA is the first program of its kind to be approved by a regional accreditor, NEASC in our case, and is the first to seek approval for Title IV funding through the “direct assessment of learning” provisions. At the time of this writing, CfA has successfully passed the first stage review by the Department of Education and is still moving through the approval process.
The radical possibility offered in the competency-based movement is that traditional higher education may lose its monopoly on delivery models. Accreditors have for some time put more emphasis on learning outcomes and assessment, but the competency-based education movement privileges them above all else. When we excel at both defining and assessing learning, we open up enormous possibilities for new delivery models, creativity and innovation. It’s not a notion that most incumbent providers welcome, but in terms of finding new answers to the cost, access, quality, productivity and relevance problems that are reaching crisis proportions in higher education, competency-based education may be the most dramatic development in higher education in hundreds of years. For example, the path to legitimacy for MOOCs probably lies in competency-based approaches, and while they can readily tackle the outcomes or competency side of the equation, they still face formidable challenges of reliable, trustworthy and rigorous assessment at scale (at least while trying to remain free). Well-developed competency-based approaches can also help undergird the badges movement, demanding that such efforts be transparent about the claims associated with a badge and the assessments used to validate learning or mastery.
Competency-based education may also provide accreditors with a framework for more fundamentally rethinking assessment. It would shift accreditation to looking much harder at learning outcomes and competencies, the claims an entity is making for the education it provides and for the mechanisms it uses for knowing and demonstrating that the learning has occurred. The good news here is that such a dual focus would free accreditors from so much attention on inputs, like organization, stakeholder roles and governance, and instead allow for the emergence of all sorts of new delivery models. The bad news is that we are still working on how to craft well designed learning outcomes and conduct effective assessment. It’s harder than many think. A greater focus on outcomes and assessment also begs other important questions for accreditors:
How will they rethink standards to account for far more complex and disaggregated business models which might have a mix of “suppliers,” some for-profit and some nonprofit, and which look very different from traditional institutions?
Will they only accredit institutions or does accreditation have to be disaggregated too? Might there by multiple forms of accreditation: for institutions, for programs, for courses, for MOOCs, for badges and so on? At what level of granularity?
CBE programs are coming. College for America is one example, but other institutions have announced efforts in this area. Major foundations are lining up behind the effort (most notably the Lumina and Bill and Melinda Gates Foundations), and the Department of Education appears to be relying on accreditors to attest to the quality and rigor of those programs. While the Department of Education is moving cautiously on this question, accreditors might want to think through what a world untethered to the credit hour might look like. Might there be two paths to accreditation: the traditional “institutional path” and the “competency-based education path,” with the former looking largely unchanged and the latter using rigorous outcomes and assessment review to support more innovation than current standards now do? Innovation theory would predict that new innovative CBE accreditation pathway would come to improve the incumbent accreditation processes and standards.
This last point is important: accreditors need to think about their relationship to innovation. If the standards are largely built to assess incumbent models and enforced by incumbents, they must be by their very nature conservative and in service of the status quo. Yet the nation is in many ways frustrated with the status quo and unwilling to support it in the old ways. Frankly, they believe we are failing, and the ways they think we are failing depend on whom you ask. But never has the popular press (and thus the public and policy makers) been so consumed with the problems of traditional higher education and intrigued by the alternatives. In some ways, accreditors are being asked to shift or at least expand their role to accommodate these new models.
If regional accreditors are unable to rise to that challenge they might see new alternative accreditors emerge and be left tethered to incumbent models that are increasingly less relevant or central to how higher education takes place 10 years from now. There is time. As has been said, we frequently overestimate the amount of change in the next two years and the dramatically underestimate the amount of change in the next 10. The time is now for regional accreditors to re-engineer the paths to accreditation. In doing so they can not only be ready for that future, they can help usher it into reality.
Paul J. LeBlanc is president of Southern New Hampshire University. This essay is adapted from writing produced for the Western Association of Schools and Colleges as part of a convening to look at the future of accreditation. WASC has given permission for it to be shared more widely and without restriction.