"Competency-based” education appears to be this year’s answer to America’s higher education challenges, judging from this week's news in Washington. Unlike MOOCs (last year’s solution), there is, refreshingly, greater emphasis on the validation of learning. Yet, all may not be as represented.
On close examination, one might ask if competency-based education (or CBE) programs are really about “competency,” or are they concerned with something else? Perhaps what is being measured is more closely akin to subject matter “mastery.” The latter can be determined in a relatively straightforward manner, using various forms of examinations, projects and other forms of assessment.
However, an understanding of theories, concepts and terms tells us little about an individual’s ability to apply any of these in practice, let alone doing so with the skill and proficiency which would be associated with competence.
Deeming someone competent, in a professional sense, is a task that few competency-based education programs address. While doing an excellent job, in many instances, of determining mastery of a body of knowledge, most fall short in the assessment of true competence.
In the course of their own education, readers can undoubtedly recall the instructors who had complete command of their subjects, but who could not effectively present to their students. The mastery of content did not extend to their being competent as teachers. Other examples might include the much-in-demand marketing professors who did not know how, in practice, to sell their executive education programs. Just as leadership and management differ one from the other, so to do mastery and competence.
My institution has been involved in assessing both mastery and competence for several decades. Created by New York’s Board of Regents in the early 1970s, it is heir to the Regents’ century-old belief in the importance of measuring educational attainment (New York secondary students have been taking Regent’s Exams, as a requirement for high school graduation, since 1878).
Building on its legacy, the college now offers more than 60 subject matter exams. These have been developed with the help of nationally known subject matter experts and a staff of doctorally prepared psychometricians. New exams are field tested, nationally normed and reviewed for credit by the American Council on Education, which also reviews the assessments of ETS (DSST) and the College Board (CLEP). Such exams are routinely used for assessing subject matter mastery.
In the case of the institution’s competency-based associate degree in nursing, a comprehensive, hands-on assessment of clinical competence is required as a condition of graduation. This evaluation, created with the help of the W.K. Kellogg Foundation in 1975, takes place over three days in an actual hospital, with real patients, from across the life span -- pediatric to geriatric. Performance is closely monitored by multiple, carefully selected and trained nurse educators. Students must demonstrate skill and ability to a level of defined competence within three attempts or face dismissal or transfer from the program.
In developing a competency-based program as opposed to a mastery-based one, there are many challenges that must be addressed if the program is to have credibility. These include:
Who specifies the elements to be addressed in a competency determination? In the case of nursing, this is done by the profession. Other fields may not be so fortunate. For instance, who would determine the key areas of competency in the humanities or arts?
Who does the assessing, and what criteria must be met to be seen as a qualified assessor of someone’s competency?
How will competence be assessed, and is the process scalable? In the nursing example above, we have had to establish a national network of hospitals, as well as recruit, train and field a corps of graduate prepared nurse educators. At scale, this infrastructure is limited to approximately 2,000 competency assessments per year, which is far less than the number taking the College’s computer-based mastery examinations.
Who is to be served by the growing number of CBE programs? Are they returning adults who have been in the workplace long enough to acquire relevant skills and knowledge on the job, or is CBE thought to be relevant even for traditional-aged students?
(It is difficult to imagine many 22 year-olds as competent within a field or profession. Yet, there is little question that most could show some level of mastery of a body of knowledge for which prepared.)
Do prospective students want this type of learning/validation? Has there been market research that supports the belief that there is demand? We have offered two mastery-based bachelor’s degrees (each for less than $10,000) since 2011. Demand has been modest because of uncertainty about how a degree earned in such a manner might be viewed by employers and graduate schools (this despite the fact that British educators have offered such a model for centuries).
Will employers and graduate schools embrace those with credentials earned in a CBE program? Institutions that have varied from the norm (dropping the use of grades, assessing skills vs. time in class) have seen their graduates face admissions challenges when attempting to build on their undergraduate credentials by applying to graduate schools. As for employers, a backlash may be expected if academic institutions sell their graduates as “competent” and later performance makes clear that they are not.
The interest in CBE has, in large part, been driven by the fact that employers no longer see new college graduates as job-ready. In fact, a recent Lumina Foundation report found that only 11 percent of employers believe that recent graduates have the skills needed to succeed within their work forces. One CBE educator has noted, "We are stopping one step short of delivering qualified job applicants if we send them off having 'mastered' content, but not demonstrating competencies."
Or, as another put it, somewhat more succinctly, "I don't give a damn what they KNOW. I want to know what they can DO.”
The move away from basing academic credit on seat time is to be applauded. Determining levels of mastery through various forms of assessment -- exams, papers, projects, demonstrations, etc. – is certainly a valid way to measure outcomes. However, seat time has rarely been the sole basis for a grade or credit. The measurement tools listed here have been found in the classroom for decades, if not centuries.
Is this a case of old wine in new bottles? Perhaps not. What we now see are programs being approved for Title IV financial aid on the basis of validated learning, not for a specified number of instructional hours; whether the process results in a determination of competence or mastery is secondary, but not unimportant.
A focus on learning independent of time, while welcome, is not the only consideration here. We also need to be more precise in our terminology. The appropriateness of the word competency is questioned when there is no assessment of the use of the learning achieved through a CBE program. Western Governors University, Southern New Hampshire, and Excelsior offer programs that do assess true competency.
Unfortunately, the vast majority of the newly created CBE programs do not. This conflation of terms needs to be addressed if employers are to see value in what is being sold. A determination of “competency” that does not include an assessment of one’s ability to apply theories and concepts cannot be considered a “competency-based” program.
To continue to use “competency” when we mean “mastery” may seem like a small thing. Yet, if we of the academy cannot be more precise in our use of language, we stand to further the distrust which many already have of us. To say that we mean “A” when in fact we mean “B” is to call into question whether we actually know what we are doing.
John F. Ebersole is the president of Excelsior College, in Albany, N.Y.
It’s surprising how many house pets hold advanced degrees. Last year, a dog received his M.B.A. from the American University of London, a non-accredited distance-learning institution. It feels as if I should add “not to be confused with the American University in London,” but getting people to confuse them seems like a pretty basic feature of the whole AUOL marketing strategy.
The dog, identified as “Peter Smith” on his diploma, goes by Pete. He was granted his degree on the basis of “previous experiential learning,” along with payment of £4500. The funds were provided by a BBC news program, which also helped Pete fill out the paperwork. The American University of London required that Pete submit evidence of his qualifications as well as a photograph. The applicant submitted neither, as the BBC website explains, “since the qualifications did not exist and the applicant was a dog.”
The program found hundreds of people listing AUOL degrees in their profiles on social networking sites, including “a senior nuclear industry executive who was in charge of selling a new generation of reactors in the UK.” (For more examples of suspiciously credentialed dogs and cats, see this list.)
Inside Higher Ed reports on diploma mills and fake degrees from time to time but can’t possibly cover every revelation that some professor or state official has a bogus degree, or that a “university” turns out to be run by a convicted felon from his prison cell. Even a blog dedicated to the topic, Diploma Mill News, links to just a fraction of the stories out there. Keeping up with every case is just too much; nobody has that much Schaudenfreude in them.
By contrast, scholarly work on the topic of counterfeit credentials has appeared at a glacial pace. Allen Ezell and John Bear’s expose Degree Mills: The Billion-Dollar Industry -- first published by Prometheus Books in 2005 and updated in 2012 – points out that academic research on the phenomenon amounts is conspicuously lacking, despite the scale of the problem. (Ezell headed up the Federal Bureau of Investigation's “DipScam” investigation of diploma mills that ran from 1980 through 1991.)
The one notable exception to that blind spot is the history of medical quackery, which enjoyed its golden age in the United States during the late 19th and early 20th centuries. Thousands of dubious practitioners throughout the United States got their degrees from correspondence course or fly-by-night medical schools. The fight to put both the quacks and the quack academies out of business reached its peak during the 1920s and ‘30s, under the tireless leadership of Morris Fishbein, editor of the Journal of the American Medical Association.
H.L. Mencken was not persuaded that getting rid of medical charlatans was such a good idea. “As the old-time family doctor dies out in the country towns,” he wrote in a newspaper column from 1924, “with no competent successor willing to take over his dismal business, he is followed by some hearty blacksmith or ice-wagon driver, turned into a chiropractor in six months, often by correspondence.... It eases and soothes me to see [the quacks] so prosperous, for they counteract the evil work of the so-called science of public hygiene, which now seeks to make imbeciles immortal.” (On the other hand, he did point out quacks worth pursuing to Fishbein.)
The pioneering scholar of American medical shadiness was James Harvey Young, an emeritus professor of history at Emory University when he died in 2006, who first published on the subject in the early 1950s. Princeton University Press is reissuing American Health Quackery: Collected Essays of James Harvey Young in paperback this month. But while patent medicines and dubious treatments are now routinely discussed in books and papers on medical history, very little research has appeared on the institutions -- or businesses, if you prefer -- that sold credentials to the snake-oil merchants of yesteryear.
There are plenty still around, incidentally. In Degree Mills, Ezell and Bear cite a Congressional committee’s estimate from 1986 that there were more than 5,000 fake doctors practicing in the United States. The figure must be several times that by now.
The demand for fraudulent diplomas comes from a much wider range of aspiring professionals now than in the patent-medicine era – as the example of Pete, the canine MBA, may suggest. The most general social-scientific study of the problem seems to be “An Introduction to the Economics of Fake Degrees,” published in the Journal of Economic Issues in 2008.
The authors -- Gilles Grolleau, Tarik Lakhal, and Naoufel Mzoughi – are French economists who do what they can with the available pool of data, which is neither wide nor deep. “While the problem of diploma mills and fake degrees is acknowledged to be serious,” they write, “it is difficult to estimate their full impact because it is an illegal activity and there is an obvious lack of data and rigorous studies. Several official investigations point to the magnitude and implications of this dubious activity. These investigations appear to underestimate the expanding scale and dimensions of this multimillion-dollar industry.”
Grolleau et al. distinguish between counterfeit degrees (fabricated documents not actually issued by the institutions the holder thereby claims to have attended) and “degrees from bogus universities, sold outright and that can require some academic work but significantly less than comparable, legitimate accredited programs.” The latter institutions, also known as diploma mills, are sometimes backed up by equally dubious accreditation “agencies.” A table in the paper indicates that more than 200 such “accreditation mills” (defined as agencies not recognized by either the Council for Higher Education Accreditation or the U.S. Department of Education) were operating as of 2004.
The authors work out the various costs, benefits, and risk factors involved in the fake degree market, but the effort seems very provisional, not to say pointless, in the absence of solid data. They write that “fake degrees allow their holders to ‘free ride’ on the rights and benefits normally tied to legitimate degrees, without the normal investment of human capital,” which may be less of a tautology than “A=A” but not by much.
The fake-degree consumer’s investment “costs” include the price demanded by the vendor but also "other ‘costs,’ such as … the fear of being discovered and stigmatized.” I suppose so, but it’s hardly the sort of expense that can be monetized. By contrast, the cost to legitimate higher-education institutions for “protecting their intellectual property rights by conducting investigations and mounting litigation against fakers” might be more readily quantified, at least in principle.
The authors state, sensibly enough: “The resources allocated to decrease the number of fake degrees should be set equal to the pecuniary value of the marginal social damage caused by the existence of the fakes, at the point of the optimal level of fakes.” But then they point to “the difficulty in measuring the value of the damage and the cost of eliminating it completely.”
So: If we had some data about the problem, we could figure out how much of a problem it is, but we don’t -- and that, too, is a problem.
Still, the paper is a reminder that empirical research on the whole scurvy topic would be of value – especially when you consider that in the United States, according to one study, “at least 3 percent of all doctorate degrees in occupational safety and health and related areas” are bogus. Also keep in mind Ezell Bear’s estimate in Degree Mills: The Billion-Dollar Industry that 40-45,000 legitimate Ph.D.s are awarded annually in the U.S. -- while another 50,000 spurious Ph.D.s are purchased here.
“In other words,” they write, “more than half of all people claiming a new Ph.D. have a fake degree.” And so I have decided not to make matters worse by purchasing one for my calico cat, despite “significant experiential learning” from her studies in ornithology.
The ubiquitous coffee chain Starbucks has received a great deal of positive media attention for its announcement that it will provide full reimbursement for tuition and fees of employees at company-owned stores who enroll in one of Arizona State University’s online bachelor’s degree programs. Education Secretary Arne Duncan even made an appearance at the program’s unveiling, alongside Starbucks CEO Howard Schultz and Arizona State President Michael Crow. But, while I applaud Starbucks for providing financial assistance to students who want to continue their education, the conditions in the model will result in fewer employees successfully completing bachelor’s degrees. Below are the reasons not all employees will benefit.
Only juniors and seniors will get a full reimbursement. The frequently asked questions document on the Starbucks website notes that there will only be a “partial scholarship” for employees who have not at least achieved junior status (likely 60 credits earned). ASU Online’s tuition rates are between $480 and $543 per credit hour, meaning that credits taken at the local community college will probably be a fraction of the cost of the ASU Online credits after partial reimbursement. This means that students are less likely to use the Starbucks program for the first 60 credits, although the promise of future reimbursement may be enough to induce Starbucks employees to go back to college.
Discussion of ASU/Starbucks
On Friday, Arizona State President Michael Crow will discuss the university's new partnership on This Week @ Inside Higher Ed, our weekly audio newscast. Click here to find out more about This Week or here to sign up for an email link to each program.
Students are not reimbursed until they complete 21 credits. This policy was designed in order to encourage completion, as the goal is to motivate students to continue their studies until they are reimbursed. However, given the per-credit cost, a student not receiving any grants from the federal government would have to pay about $10,000 out of pocket (or borrow that amount) before being reimbursed. ASU Online recommends that students take two or three 3-credit classes during each 7.5-week class window, meaning that a continuously enrolled full-time student who started in August would probably complete seven classes by March or May of the following year. Students can also qualify for reimbursement by enrolling part-time, but they may take two years to complete the 21 credits necessary for reimbursement. This also provides a strong incentive for students to stay at Starbucks to claim the benefit, which can limit their mobility as employees but may be worthwhile given the potential value of the benefit.
The delay between paying tuition and fees and being reimbursed introduces substantial risk for students. A student who is willing to pay up to $10,000 and get reimbursed later only if successful likely has a higher tolerance for risk, is more willing to borrow, and is more likely to complete courses than a student who is hesitant to participate in the program. This means that the Starbucks employees who participate in the program as currently constructed are probably from higher-income families with more social and cultural capital — potentially minimizing the social mobility the program offers. Reimbursing students after each successfully completed course would help mitigate this risk and reduce the amount of money students have to pay upfront.
Reimbursements by Starbucks take place after other grant aid is applied, making the company’s contribution smaller. Students are required to file the Free Application for Federal Student Aid (FAFSA) in order to participate in the program and any grant aid received will be applied before Starbucks makes its contribution. Consider the case of a student with a zero expected family contribution, representing the greatest level of financial need, who enrolls for 12 credits in a semester. Her tuition at $500 per credit would be $12,000 for the academic year. She is eligible for the maximum Pell Grant of $5,730 in the 2014-15 academic year, which is applied before any aid from Starbucks. This leaves $6,270 uncovered by the Pell Grant, but Arizona State is offering scholarships of $4,840 per year to all Starbucks employees. The resulting $1,430 would be paid by Starbucks if the student didn't receive any other grants or scholarships. This is an admirable contribution, but most of the burden of financing the student is not on Starbucks.
Online education may not be right for everyone, yet it is the only option funded. It is far easier for Starbucks to work with one college than hundreds for administrative purposes. However, the lack of choice in the program may not be best for all students. ASU Online does offer about 40 majors, but they are all online — and research suggests that online courses may not work as well as face-to-face courses for students from lower-income families. While I don’t know enough about ASU’s programs to pass judgment on their quality, some students may not be interested in enrolling online even if the quality is high and the cost to the student is low.
All of these factors suggest that the percentage of Starbucks employees who successfully complete a bachelor’s degree as a result of the tuition reimbursement program will be fairly low. Starbucks should be commended for offering this benefit to its employees, but policymakers shouldn’t expect this program to substantially move the college completion rate dial in its current form.
Robert Kelchen (@rkelchen) is an assistant professor in the department of education leadership, management and policy at Seton Hall University. He blogs at Kelchen on Education. All opinions are his own.
Universities, as seats of learning and powerhouses of research, are stepping up to assume a new role. In the wake of a global financial meltdown and consequent challenges to the fabric of many societies, universities are emerging as powerful catalysts and indeed drivers of socioeconomic growth – not only through research or technology transfer, but by assuming responsibility for preparing students for jobs in delivering today’s highly skilled workers and tomorrow’s innovators and leaders of industry.
That’s why the employability of our graduates needs to take center stage and why I applaud the Obama administration’s recent call to action in this regard. The emergence of new institutional rankings to compare the "value" delivered, such as graduate employment and earnings across institutions, means that employability has become "our job." And we need to take this responsibility seriously if we want to successfully compete in the global marketplace for higher education. Universities need to understand that we have a social duty and perhaps a moral one too, to help successfully launch our talented graduates into society.
Here in Britain, employability outcomes are already part of our world and feature heavily in the key performance indicators of British universities. Our Higher Education Statistics Agency collects and reports national data on our publicly funded institutions, including employment rate overall from each university and type of employment outcome. And while our American cousins are decades ahead in areas such as philanthropy and have helped our journey, Britain's experience of the employability agenda is one where we can perhaps return the favor. It's this spirit of sharing and exploring wider global education trends that moved me to share some insights into how the employability agenda is influencing behavior among our students and faculty, and in the administration team, too.
It’s clear in Britain that the move to show a return on investment through enhanced employment opportunities – the so-called "graduate premium" – is strongly correlated with the recent significant increase in student fees, or what would be considered tuition in the American context. This was a key part of a public policy shift, across successive UK governments, to recognize more overtly that graduates are beneficiaries of their education and as such should contribute to it directly, in turn reducing the public subsidy for higher education. The fees, covered by a public student loan, are repaid only once the graduate is earning a salary deemed appropriate for a graduate (approximately $35,000) and no payments are needed up-front.
A few things have happened as a consequence. The first, perhaps rather unexpected but of high value, was that we have seen a positive impact on the social inclusion agenda as more students from poorer backgrounds progress to university; analysis from the University and Colleges Acceptance Service (UCAS) indicates that compared with entry rates in 2011, the year before the introduction of higher tuition fees in England, 18-year-olds in disadvantaged areas in England were 12 percent more likely to enter in 2013. The second was, however, anticipated, and is the subject of this commentary, in that students are now much more savvy as education "consumers" and are fiercely attuned to understanding the job opportunities at the end of their degree.
As such, the student voice is being heard right at the heart of university administration and across the faculty. Newly introduced UK websites such as Unistats (similar to the College Scorecard) allow prospective students to directly compare courses and institutions. Of course, when first introduced around two years ago, such public comparison sites were disruptive – and this perhaps echoes the current disquiet in the United States as similar plans are rolled out across the pond. Britain’s "Key Information Set" (KIS) data, which populates the site, comprises the items of information which students have said they would find most useful when making informed choices about where to study. The "empowered" student wants to know what the likelihood is of getting a job after graduation in various fields, what type of job they may get (professional or non-professional), and what salary they could expect. Nationally, total employability and a new measure of professional versus non-professional employment are both used in national university league tables, which are used by students to pick institutions and by the government to award funds.
With this public interest in outcome measures, university presidents and the wider administration are acutely aware of the potential impact on reputation, and by extension, recruitment. There are risks to both if we do not continue to produce graduates who are highly employable, who can obtain graduate-level jobs and who can deliver on the investment they have made in their education through the "graduate premium" on earnings. Placing such key institutional risks to one side, the wider public policy agenda surely means that governments, industry and indeed society at large need to pay attention to employability given the economic and indeed social impact of skilled labor in the global market place. Research consistently shows that graduates are more likely to be employed than those who left education with lower qualifications. In 2013, there were 12 million graduates in Britain and the graduate employment rate stood at 87 percent; this compares to 83 percent employment rate for those with A levels – approximately equivalent to the high school diploma.
But it’s not quite as simple as that. A degree, once considered the passport to a graduate-level career, needs to now come in a total package – "graduate plus" – as employers seek well-rounded employees who are "work-ready" with clear evidence of both job-specific skills and, prized graduate attributes. Given the fact that more people are achieving graduate status, we need to help our students develop employability attributes and skills throughout their time at university while they study. This needs careful curriculum and indeed pedagogic innovation and stewardship, including partnerships with business, industry and the professions.
This is why at my own institution, Plymouth University in Britain, we embed employability throughout the curriculum from day one and we then continue to focus on developing the entrepreneurial skills of our students through academic courses as well as support, mentoring and networking opportunities. For example, curricular experiential learning projects across the university range from business (such as management students conducting consultancy work for local businesses in a program called Inspiring Futures) to health (dental, medical and optometry students are all trained in primary care settings, ensuring they have to develop communication skills with real patients in order to better understand their needs), to the whole institution, such as the Wonder Room consultancy, which brings together students from business, arts and science to pitch for, and undertake, live projects in the region.
We are also focusing on developing internships and placements for our students to enable them to enhance their resumes and gain real work-place experience. Our Plymouth Graduate Internship Program develops graduate-level internship positions with employers where recent graduates are given the opportunity to apply a range of skills, assume real responsibilities, make an impact and progress quickly from new graduate to successful professional. Last year alone, 40 percent of our students embarked on paid industry placements. I shared this fact on social media whilst at a conference in the U.S. earlier this year and was overwhelmed by the impact of the response stateside to something that we see here as very much just "business as usual."
For us, at Plymouth, a key factor in our success has been to establish our unique "students as partners" charter which, rather than a transactional relationship that places the student as a customer, we feel that the we take joint responsibility with our students for their educational outcomes. This means that as well as supporting employment opportunities, whether through internships or placements, we recognize that we are preparing graduates for jobs that don’t even exist yet and for a career that will be multidimensional and more akin to a career portfolio. And so, in line with our focus on enterprise, we foster an entrepreneurial mindset with our students so that they are set up to thrive as socially responsible, highly employable global citizens. Testament to this success has been national success as our students and student societies win major entrepreneurial and business competitions. We are also seeing more of our graduates progress to set up their own business ventures and also to engage in community volunteering work with a social purpose. So, for students, the employability metrics impact their decision-making as they make more informed decisions.
Our faculty have embraced the employability agenda through curriculum and pedagogic innovations and by creating partnerships with employers; this in turn, has served to connect us as a university to the society we serve, leading to research opportunities and live commissions for students and staff consultancy. And for the senior administrative team around the president’s table? Well, that’s an interesting one. Of course, we always had awareness of the demand for our programs, and an interest in student satisfaction – but there’s been a real shift in emphasis and we talk a lot more about the student experience which sits comfortably alongside other top table issues such as financial sustainability and risk. We are now more acutely aware that our brand is firmly aligned to the quality of our graduates and their market value, and that employability metrics are a clear proxy measure of our university standing. So jobs for our students now sit very much as one of our jobs, too.
So, dear American colleagues, if I may be so bold – I would say please embrace employability metrics as a powerful direction of travel. Be aware that public and private supporters of higher education are keenly interested to know more about the returns on their investment and on the role universities are playing now and can go on to play in driving economic and social inclusion. Universities can respond on their own terms in powerful and compelling ways to drive the narrative around employability. We should be clear that employability is very much part of the learning continuum, and learning – well, that is our job, isn’t it?
Wendy Purcell is president of Plymouth University, in Britain.