Nearly 70 institutions are collaborating to better assess learning outcomes as part of a new initiative called the Multi-State Collaborative to Advance Learning Outcomes Assessment. The colleges and universities are a mix of two- and four-year institutions.
The initiative, funded in its initial planning year by the Bill & Melinda Gates Foundation, was announced Monday by the Association of Colleges and Universities and the State Higher Education Executive Officers association.
”The calls are mounting daily for higher education to be able to show what students can successfully do with their learning,” said Carol Geary Schneider, AAC&U president, in an announcement. “The Multi-State Collaborative is a very important step toward focusing assessment on the best evidence of all: the work students produce in the course of their college studies."
The 68 colleges and universities participating in the collaborative are from Connecticut, Indiana, Kentucky, Massachusetts, Minnesota, Missouri, Oregon, Rhode Island and Utah. Faculty at those institutions will sample and assess student work as part of a cross-state effort to document how students are achieving learning outcomes such as quantitative reasoning, written communication, and critical thinking.
All of the assessments will be based on a set of common rubrics. The project will also develop an online data platform for uploading student work samples and assessment data.
U.S. Sen. Kay Hagan, a North Carolina Democrat, last week introduced a bill that would seek to encourage four-year institutions to identify transfer students who have earned enough credits for an associate degree but never received one. Through this process, which is dubbed "reverse transfer," students at four-year institutions can earn associate degrees they failed to receive before transferring. The bill would encourage reverse transfer by creating competitive grants for states.
In their effort to improve outcomes, colleges and universities are becoming more sophisticated in how they analyze student data – a promising development. But too often they focus their analytics muscle on predicting which students will fail, and then allocate all of their support resources to those students.
That’s a mistake. Colleges should instead broaden their approach to determine which support services will work best with particular groups of students. In other words, they should go beyond predicting failure to predicting which actions are most likely to lead to success.
Higher education institutions are awash in the resources needed for sophisticated analysis of student success issues. They have talented research professionals, mountains of data and robust methodologies and tools. Unfortunately, most resourced-constrained institutional research (IR) departments are focused on supporting accreditation and external reporting requirements.
Some institutions have started turning their analytics resources inward to address operational and student performance issues, but the question remains: Are they asking the right questions?
Colleges spend hundreds of millions of dollars on services designed to enhance student success. When making allocation decisions, the typical approach is to identify the 20 to 30 percent of students who are most “at risk” of dropping out and throw as many support resources at them as possible. This approach involves a number of troubling assumptions:
The most “at risk” students are the most likely to be affected by a particular form of support.
Every form of support has a positive impact on every “at risk” student.
Students outside this group do not require or deserve support.
What we have found over 14 years working with students and institutions across the country is that:
There are students whose success you can positively affect at every point along the risk distribution.
Different forms of support impact different students in different ways.
The ideal allocation of support resources varies by institution (or more to the point, by the students and situations within the institution).
Another problem with a risk-focused approach is that when students are labeled “at risk” and support resources directed to them on that basis, asking for or accepting help becomes seen as a sign of weakness. When tailored support is provided to all students, even the most disadvantaged are better-off. The difference is a mindset of “success creation” versus “failure prevention.” Colleges must provide support without stigma.
To better understand impact analysis, consider Eric Siegel’s book Predictive Analytics. In it, he talks about the Obama 2012 campaign’s use of microtargeting to cost-effectively identify groups of swing voters who could be moved to vote for Obama by a specific outreach technique (or intervention), such as piece of direct mail or a knock on their door -- the “persuadable” voters. The approach involved assessing what proportion of people in a particular group (e.g., high-income suburban moms with certain behavioral characteristics) was most likely to:
vote for Obama if they received the intervention (positive impact subgroup)
vote for Obama or Romney irrespective of the intervention (no impact subgroup)
vote for Romney if they received the intervention (negative impact subgroup)
The campaign then leveraged this analysis to focus that particular intervention on the first subgroup.
This same technique can be applied in higher education by identifying which students are most likely to respond favorably to a particular form of support, which will be unmoved by it and which will be negatively impacted and dropout.
Of course, impact modeling is much more difficult than risk modeling. Nonetheless, if our goal is to get more students to graduate, it’s where we need to focus analytics efforts.
The biggest challenge with this analysis is that it requires large, controlled studies involving multiple forms of intervention. The need for large controlled studies is one of the key reasons why institutional researchers focus on risk modeling. It is easy to track which students completed their programs and which did not. So, as long as the characteristics of incoming students aren’t changing much, risk modeling is rather simple.
However, once you’ve assessed a student’s risk, you’re still left trying to answer the question, “Now what do I do about it?” This is why impact modeling is so essential. It gives researchers and institutions guidance on allocating the resources that are appropriate for each student.
There is tremendous analytical capacity in higher education, but we are currently directing it toward the wrong goal. While it’s wonderful to know which students are most likely to struggle in college, it is more important to know what we can do to help more students succeed.
Dave Jarrat is a member of the leadership team at InsideTrack, where he directs marketing, research and industry relations activities.
Universities, as seats of learning and powerhouses of research, are stepping up to assume a new role. In the wake of a global financial meltdown and consequent challenges to the fabric of many societies, universities are emerging as powerful catalysts and indeed drivers of socioeconomic growth – not only through research or technology transfer, but by assuming responsibility for preparing students for jobs in delivering today’s highly skilled workers and tomorrow’s innovators and leaders of industry.
That’s why the employability of our graduates needs to take center stage and why I applaud the Obama administration’s recent call to action in this regard. The emergence of new institutional rankings to compare the "value" delivered, such as graduate employment and earnings across institutions, means that employability has become "our job." And we need to take this responsibility seriously if we want to successfully compete in the global marketplace for higher education. Universities need to understand that we have a social duty and perhaps a moral one too, to help successfully launch our talented graduates into society.
Here in Britain, employability outcomes are already part of our world and feature heavily in the key performance indicators of British universities. Our Higher Education Statistics Agency collects and reports national data on our publicly funded institutions, including employment rate overall from each university and type of employment outcome. And while our American cousins are decades ahead in areas such as philanthropy and have helped our journey, Britain's experience of the employability agenda is one where we can perhaps return the favor. It's this spirit of sharing and exploring wider global education trends that moved me to share some insights into how the employability agenda is influencing behavior among our students and faculty, and in the administration team, too.
It’s clear in Britain that the move to show a return on investment through enhanced employment opportunities – the so-called "graduate premium" – is strongly correlated with the recent significant increase in student fees, or what would be considered tuition in the American context. This was a key part of a public policy shift, across successive UK governments, to recognize more overtly that graduates are beneficiaries of their education and as such should contribute to it directly, in turn reducing the public subsidy for higher education. The fees, covered by a public student loan, are repaid only once the graduate is earning a salary deemed appropriate for a graduate (approximately $35,000) and no payments are needed up-front.
A few things have happened as a consequence. The first, perhaps rather unexpected but of high value, was that we have seen a positive impact on the social inclusion agenda as more students from poorer backgrounds progress to university; analysis from the University and Colleges Acceptance Service (UCAS) indicates that compared with entry rates in 2011, the year before the introduction of higher tuition fees in England, 18-year-olds in disadvantaged areas in England were 12 percent more likely to enter in 2013. The second was, however, anticipated, and is the subject of this commentary, in that students are now much more savvy as education "consumers" and are fiercely attuned to understanding the job opportunities at the end of their degree.
As such, the student voice is being heard right at the heart of university administration and across the faculty. Newly introduced UK websites such as Unistats (similar to the College Scorecard) allow prospective students to directly compare courses and institutions. Of course, when first introduced around two years ago, such public comparison sites were disruptive – and this perhaps echoes the current disquiet in the United States as similar plans are rolled out across the pond. Britain’s "Key Information Set" (KIS) data, which populates the site, comprises the items of information which students have said they would find most useful when making informed choices about where to study. The "empowered" student wants to know what the likelihood is of getting a job after graduation in various fields, what type of job they may get (professional or non-professional), and what salary they could expect. Nationally, total employability and a new measure of professional versus non-professional employment are both used in national university league tables, which are used by students to pick institutions and by the government to award funds.
With this public interest in outcome measures, university presidents and the wider administration are acutely aware of the potential impact on reputation, and by extension, recruitment. There are risks to both if we do not continue to produce graduates who are highly employable, who can obtain graduate-level jobs and who can deliver on the investment they have made in their education through the "graduate premium" on earnings. Placing such key institutional risks to one side, the wider public policy agenda surely means that governments, industry and indeed society at large need to pay attention to employability given the economic and indeed social impact of skilled labor in the global market place. Research consistently shows that graduates are more likely to be employed than those who left education with lower qualifications. In 2013, there were 12 million graduates in Britain and the graduate employment rate stood at 87 percent; this compares to 83 percent employment rate for those with A levels – approximately equivalent to the high school diploma.
But it’s not quite as simple as that. A degree, once considered the passport to a graduate-level career, needs to now come in a total package – "graduate plus" – as employers seek well-rounded employees who are "work-ready" with clear evidence of both job-specific skills and, prized graduate attributes. Given the fact that more people are achieving graduate status, we need to help our students develop employability attributes and skills throughout their time at university while they study. This needs careful curriculum and indeed pedagogic innovation and stewardship, including partnerships with business, industry and the professions.
This is why at my own institution, Plymouth University in Britain, we embed employability throughout the curriculum from day one and we then continue to focus on developing the entrepreneurial skills of our students through academic courses as well as support, mentoring and networking opportunities. For example, curricular experiential learning projects across the university range from business (such as management students conducting consultancy work for local businesses in a program called Inspiring Futures) to health (dental, medical and optometry students are all trained in primary care settings, ensuring they have to develop communication skills with real patients in order to better understand their needs), to the whole institution, such as the Wonder Room consultancy, which brings together students from business, arts and science to pitch for, and undertake, live projects in the region.
We are also focusing on developing internships and placements for our students to enable them to enhance their resumes and gain real work-place experience. Our Plymouth Graduate Internship Program develops graduate-level internship positions with employers where recent graduates are given the opportunity to apply a range of skills, assume real responsibilities, make an impact and progress quickly from new graduate to successful professional. Last year alone, 40 percent of our students embarked on paid industry placements. I shared this fact on social media whilst at a conference in the U.S. earlier this year and was overwhelmed by the impact of the response stateside to something that we see here as very much just "business as usual."
For us, at Plymouth, a key factor in our success has been to establish our unique "students as partners" charter which, rather than a transactional relationship that places the student as a customer, we feel that the we take joint responsibility with our students for their educational outcomes. This means that as well as supporting employment opportunities, whether through internships or placements, we recognize that we are preparing graduates for jobs that don’t even exist yet and for a career that will be multidimensional and more akin to a career portfolio. And so, in line with our focus on enterprise, we foster an entrepreneurial mindset with our students so that they are set up to thrive as socially responsible, highly employable global citizens. Testament to this success has been national success as our students and student societies win major entrepreneurial and business competitions. We are also seeing more of our graduates progress to set up their own business ventures and also to engage in community volunteering work with a social purpose. So, for students, the employability metrics impact their decision-making as they make more informed decisions.
Our faculty have embraced the employability agenda through curriculum and pedagogic innovations and by creating partnerships with employers; this in turn, has served to connect us as a university to the society we serve, leading to research opportunities and live commissions for students and staff consultancy. And for the senior administrative team around the president’s table? Well, that’s an interesting one. Of course, we always had awareness of the demand for our programs, and an interest in student satisfaction – but there’s been a real shift in emphasis and we talk a lot more about the student experience which sits comfortably alongside other top table issues such as financial sustainability and risk. We are now more acutely aware that our brand is firmly aligned to the quality of our graduates and their market value, and that employability metrics are a clear proxy measure of our university standing. So jobs for our students now sit very much as one of our jobs, too.
So, dear American colleagues, if I may be so bold – I would say please embrace employability metrics as a powerful direction of travel. Be aware that public and private supporters of higher education are keenly interested to know more about the returns on their investment and on the role universities are playing now and can go on to play in driving economic and social inclusion. Universities can respond on their own terms in powerful and compelling ways to drive the narrative around employability. We should be clear that employability is very much part of the learning continuum, and learning – well, that is our job, isn’t it?
Wendy Purcell is president of Plymouth University, in Britain.
We have a propensity to draw conclusions from patterns, even when there is a lack of hard data. Thus, British soccer fans are rarely invited to join quiet lawn parties in retirement villages, and NFL football scouts have not added Rabbinical schools to their talent-search itineraries.
Epidemiologists have taken us a step further. Drawing from behavior and outcomes of large populations, they have been able to make recommendations in the fields of nutrition and health, long before the relevant science has been worked out.
None of this has proven to be particularly troublesome. We are all part of a public and we expect that our behavior and actions will be observed and sometimes recorded by others.
Sometimes records are kept of relatively private actions under tacit assumptions which have not withstood the impact of modern technology. Particulars regarding our homes, once maintained on cumbersome files located in sleepy municipal offices, are now accessible, instantly, anywhere in the world.
Health records, too, once safely housed in thick files in a physician’s office, will soon be electronically stored, one renegade’s finger push away from worldwide exposure.
All of the above record keeping is associated with voluntary actions on our part, and intended for our benefit. We want to buy homes, and want careful records maintained of our title. Similarly, physicians need details of our health history to treat us properly.
An even stronger element of cooperation enters when purchases are made by customers who willingly show their names and other personal information in exchange for discounts, diaper coupons and the like.
Duress enters with the record-keeping and dissemination associated with the criminal justice system. This has consequences. A 2011 article in The New York Times described the difficulty that people with any record of conviction – over 65,000,000 was the number mentioned – have getting an interview, let alone a job.
There are laws which require employers to take into account the “severity of an offense, the length of time that has passed, and the relevance to the job in question,” according to the article. In other words, a person with a speeding arrest should be considered differently from a violent felon.
But Internet-based search capabilities make it trivially easy to select from a large pool of applicants with no criminal records, even while (wink, wink) giving fair consideration to all qualified applicants.
All of this leads us up to the most Big Brotherly data collection of them all. Education is viewed as correlating with many key career and life outcomes: employment, family, citizenry and according to some, even health. The case has also been made for the collection of student unit records to enable sophisticated statistical tools to identify those educational policies and practices that lead to particularly successful outcomes. At first glance, the case is a compelling one.
Except that the individuals whose records are being collected have no choice, and no voice. Any student attending certain colleges in one of the 40 states gathering unit record data is subject to this data collection, with all his/her triumphs and failures permanently on file in a state education office.
This is compounded by the fact that while some states have been gathering student unit records for a quarter of a century, with a state like California housing as many records as do entire countries, there have been no enhanced educational outcomes or effective policies emerging as a result of the collection and use of these student records.
Faced with this absence of concrete results, advocates of data collection are now trying to persuade legislators of the need for more data, specifically a national unit record system.
Putting aside the astronomical costs involved and the millions of productive hours wasted, this national collection of educational records would constitute the motherlode of all databases, particularly when cross-linked to other databases such as those of health, employment, criminal record, and credit worthiness.
There is no point discussing privacy protection. Those who believe in a national database of student unit records will refer to laws and law enforcement. Those who are more skeptical will point to the "data breach of the day" column in their local newspaper.
More relevant is the fact that every person is entitled to a copy of his or her own record. Employers will know this too. And while laws can be passed making it illegal to require an applicant to submit a copy of his or her student unit record, nothing can prevent an employer from favoring applicants who "willingly" share their personal documents.
Nor is it just employers who will want to know. Venture capitalists, graduate and professional school admissions officers, and future business partners also will want to know everything about a person before making any kind of commitment or investment.
Sad. College has always been a time for growth, for experimentation, for change. Young people could make false starts knowing that they could always start fresh. Few people were watching and even fewer recording.
All this will change should we accede to pressures for a national student unit record system. The sense of freedom and independence which characterizes youth will be compromised by the albatross of a written record of one’s younger years in the hands of government.
Nobody should be sentenced to a lifetime of looking over his/her shoulder as a result of a wrong turn or a difficult term during college. Nobody should be threatened by a loss of personal privacy, and we as a nation should not experience a loss of liberty because our government has decreed that a student unit record is the price to pay for a postsecondary education.
Bernard Fryshman is a professor of physics and a former accreditor.
A recent Gallup-Purdue study on the relationship between student experiences in college and later job satisfaction concludes that what matters is not “where you go” but “how you do” college. According to the report, students who participate in what the authors term the “winning combination” -- research projects, extracurricular activities, internships and close relationships with faculty -- are more highly engaged in their jobs after college. What the authors fail to acknowledge, however, is the fact that how students “do” college is often governed by pre-existing inequalities that our higher education system does little to ameliorate.
The report glosses over the reality that students from low-income families tend to cluster in less-prestigious public colleges and universities. These institutions often have scarce resources, resulting in high student-faculty ratios and fewer opportunities for students to engage in the ways that appear to correlate with later job satisfaction.
Even at colleges with bountiful resources and a low faculty-student ratio, however, students from higher-income families are more likely to participate in activities that increase college engagement. Such students are far more likely to have contacts at the college and to feel comfortable building relationships with faculty. Lower-income students, on the other hand, often face barriers -- including the need to work -- that make it difficult for them to pursue college experiences that might help them in later life.
Beyond these structural impediments, low-income students face a problem that is more nebulous and difficult to address: cultural mismatch. Often, students from working-class backgrounds interpret the differences between their background and the middle-class norms they encounter at college as signifying that they do not belong. As Paul Tough recently chronicled in a New York Times Magazine article, these students often experience feelings of discomfort, inadequacy and exclusion, which hinders their ability to make meaningful connections with faculty, staff and peers.
Even when working-class and first-generation students partake in extracurricular activities, or meet with faculty or staff, they struggle to achieve the same benefits as their more affluent peers. My research indicates that participating in study groups and extracurricular activities, meeting with staff and socializing with faculty does not improve the GPA and persistence of low-income students, but does result in significant returns in those areas for students from higher-income families.
Why the benefits of these experiences accrue so lopsidedly is difficult to parse: one reason may be that higher-income students are better-able to leverage opportunities that arise when interacting with faculty. A recent study finding that professors were less inclined to respond to emails from female or minority students hints at another partial explanation: biases may undermine the usefulness of interactions with faculty for low-income students.
Regardless of what causes the disparate effects, colleges must acknowledge that a range of impediments prevents low-income college students from participating in and benefiting from meaningful extracurricular activities and relationships. They should assess the range of non-classroom activities available -- such as student government, student-led publications, and intramural sports -- and consider whether these opportunities should be structured differently to maximize accessibility and value for students from less advantaged backgrounds.
The “University Leadership Network” at the University of Texas -- highlighted in the New York Times article -- is an example of a program that arose from such purposeful consideration. The program is designed to both deeply engage low-income students in the kinds of activities that Gallup-Purdue correlates with later job satisfaction, and to simultaneously quell feelings of inadequacy.
While they are not a cure-all for the reproduction of inequality in American higher education, these kinds of wrap-around supports may be necessary for low-income and first generation college students to reap the same benefits from the “winning combination” of college experiences that more advantaged students already enjoy. Paired with larger policy changes to eliminate structural barriers to college access and completion, they offer a promising tool to improve equity.
Lauren Schudde is a research associate at the Community College Research Center at Columbia University's Teachers College.
Senator Claire McCaskill on Thursday stepped up pressure on a top higher education lobbying group to turn over copies of a presentation that advised colleges on how to respond to her survey on campus sexual violence.
The Democratic lawmaker sent a letter to the American Council on Education on Thursday that said the group should either provide the webinar materials she previously requested or offer a legal justification for its refusal to hand over the documents.
The letter escalates a dispute that began earlier this month when McCaskill accused the higher education association of interfering with her effort to collect information from colleges about their sexual assault policies. The American Council on Education had sponsored a webinar, conducted by a Washington D.C. law firm, that advised colleges on how they should respond the McCaskill’s survey. The group maintains that the online briefing did not discourage institutions from responding to the survey, but it has refused to provide copies of the presentation materials or a list of which institutions participated. Ada Meloy, general counsel at the American Council on Education, called McCaskill’s request for information about the webinar an intrusion on colleges “rights to association."
McCaskill’s letter on Thursday reiterated her request for a copy of the PowerPoint presentation and struck a harsher tone.
“While I appreciate your commitment to your members’ confidentially, I question how that commitment would prevent you from providing the presentation materials used during the webinar,” McCaskill wrote.
“If ACE continues to refuse to comply with my request,” she continued. “I request that you provide a written response containing ACE’s legal justification for refusing to provide a Subcommittee of the United States Senate with information necessary to carry out its constitutional responsibilities.”
A spokesman for the American Council on Education said Thursday that the group is “in communication with Senator McCaskill and her staff about this issue.” The organization declined further comment on the matter.
McCaskill is drafting -- along with Senators Kirsten Gillibrand of New York and Richard Blumenthal of Connecticut -- legislation aimed at reducing campus sexual assaults. She has said she’s interested in increasing the penalties colleges face for violating federal rules on handling cases of sexual violence on their campus.