How the Department of Education should design its ratings tools (essay)

Last week, the Department of Education walked back from its plans to develop a comprehensive college ratings system. In its place, the department plans to release “easy-to-use tools that will provide students with more data than ever before to compare college costs and outcomes.”

There are already plenty of resources and tools to help students and families research these questions, provided by government (College Navigator), nonprofit organizations (College Results Online), and media outlets (U.S. News & World Report College Rankings), and lawmakers continue to call for more consumer information about college outcomes.

But is anyone actually using these tools, and are the interfaces, graphics and user experiences designed to actually help students? And how “easy to use” will these new tools be?

To better understand what works and what doesn’t, I recently co-wrote a report with Healey Whitsett, now of the Pew Charitable Trust, that provides best practices on how to design and deliver information to help prospective college students navigate the best programs available for them. The department should turn to these principles as it develops its new tools to maximize their effectiveness.

We synthesized scores of studies on behavioral economics, information search, retention, and bottlenecks that get in the way and break down the proper processing of information – and recommended ways to design tools so students get the information they need to make more informed decisions about where to go to school, what to study, and how to pay for it. We also recommend that designers target their efforts at students from low-income families, as unfortunately, they’re the least likely spend a lot of time searching for information.

For example, designers shouldn’t try and cram too much information in one place. Cognitive research tells us this overwhelms the reader and makes it difficult to comprehend and retain the information.

In one study, researchers compared individuals' interactions with the standard mortgage disclosure form, against a redesigned, better organized prototype: borrowers using the redesigned form were 38 percentage points more likely to correctly identify the amount of the loan, and 11 percentage points more likely to correctly identify their monthly payment amount. Since students are unable to identify how much money they took out in student loans, redesigning these forms makes a lot of sense.

The literature also demonstrates the importance of personally tailored information: Consumers are much more likely to identify, remember, and use information if it is personally relevant to them. That’s why broad national rankings are probably only so useful to students and families. In this regard, the Department seems to be on the right track by making the tools “customizable” to the user.

Our research also finds that higher education stakeholders shouldn’t present students with too many options of comparison. With 7,000 colleges and universities to choose from, it’s critical that we figure out manageable ways for students to compare a limited set of schools that tailor to their interests, or they risk facing what we call the “tyranny of choice.”

A study by Judith Scott-Clayton at the Community College Resource Center at Columbia University's Teachers College showed that too many choices in community college majors or programs may overwhelm and discourage students from persisting in and succeeding at earning a credential.

Furthermore, information should be as personally tailored as possible, as individually contextualized information is much more likely to be recalled and used in decision making.

Aspiring students must also be able to compare and contrast their school and loan choices. It would be very difficult to choose which school to attend if you knew the graduation rate of one and the list of majors of the other. They must be able to compare the same variables side by side.  

Fortunately, it looks like Congressional leaders are interested in arming students with more information as well. In its white paper on proposals for reauthorizing the Higher Education Act, the Senate Health, Education, Labor and Pensions Committee called for “extensive consumer testing on what information is needed and how it should be presented” and to “[a]pply this research to any federally produced consumer tools and make the research available publicly to voluntarily inform the market.”

The House Education and Workforce Committee recently said, “Access to better information will empower students with the knowledge they need to make smart decisions in the college marketplace.”

We hope that the department will take note of this research as it designs this new tool and we also applaud department officials for committing to work with outside parties to design their own mobile apps and interfaces optimized for usability. After all, aspiring students can have all the data and information in the world, but if it’s not packaged and delivered in a way that’s useful to them, then we’re wasting our time.

Tom Allison is research and policy manager at Young Invincibles.

Editorial Tags: 

Workforce dollars in Indiana are tied to graduation rates that cause trouble for Ivy Tech

Smart Title: 

Indiana's Ivy Tech Community College is working to hold onto federal workforce development dollars as the state now uses graduation rates as an accountability measure, which the college feels is unfair.

Right to marriage ruling shows impact of humanities 'basic research' (essay)

Are the humanities useless? Or can they produce “inventions” like the natural sciences? If our only understanding of invention is a technological product, perhaps the humanities are useless. But if we include new insights into culture, insights that transform our relationship with the world around us, then the humanities have real value.

Perhaps nowhere was this made more clear than in last week’s decision by the U.S. Supreme Court that marriage is a fundamental right under the U.S. Constitution. The court’s decision, written by Justice Anthony Kennedy, upholds the historical significance of marriage and recognizes the sanctity of the intimate relationships marriage makes possible.

Early in his decision, Justice Kennedy cites three historians’ works: Nancy Cott’s Public Vows: A History of Marriage and the Nation (2000), Stephanie Coontz’s Marriage: A History (2005), and Hendrik Hartog’s Man & Wife in America: A History (2000).

These scholars’ insights -- their “inventions” of historical understanding -- did not take place in a vacuum. They were the result of countless doctoral dissertations and “useless” journal articles.

Cott, Coontz and Hartog relied on and contributed to a vibrant community of scholars who, over the past few decades, sought to make sense of the history of American families. Scholars formed intellectual networks and new journals. They criticized each other’s work and refined understanding. They debated each other in print and at conferences.

And, over time, slowly and painstakingly, they taught us that American family life has a history, that the ways in which husbands and wives and parents and children related to each other, and the legal and cultural contexts that shaped those relations, changed over time. Like everything else, families are part of culture.

In short, the Supreme Court relied on the very academic infrastructure for research that is now being undermined by public defunding and efforts to make universities more focused on workforce training. It is the kind of academic infrastructure that would be threatened by efforts, such as those in Wisconsin, to weaken the tenure and shared governance protections that sustain academic freedom.

For Justice Kennedy, scholars’ “new insights have strengthened, not weakened, the institution of marriage.” Historians have demonstrated how marriage, once a way to organize family resources or a method by which men exercised governance over their dependents, gave way in the wake of the American Revolution to something more egalitarian and more affectionate.

In fact, our ideal of marriage as a relationship between two loving people deeply committed to each other was reinforced and popularized by the American Revolution, which challenged inherited ideas of inequality not just in politics but throughout society.

Historians, of course, did not make this happen on their own. Were it not for all the same-sex couples who dared to come out of the closet, all the organizers who built a movement and all the people who brought cases in the first place, the issue would not have been before us.

But if it were not for scholars of marriage, Justice Kennedy may not have had the knowledge before him to reach his decision. The value of basic research in the humanities cannot be denied. We need to reinvest in our research infrastructure so that we can continue to generate insights that will help us make sense of our most pressing public questions. Basic research in the humanities, it turns out, has a tangible social impact.

Johann Neem is a professor of history at Western Washington University, an affiliate of the Wisconsin Center for the Advancement of Postsecondary Education, and a visiting faculty fellow at the Institute for Advanced Studies in Culture at the University of Virginia.

Editorial Tags: 

First-Generation Students and Academic Preparation

A new report from ACT and the Council for Opportunity in Education found that the vast majority of first-generation students who take ACT's college entrance exam plan to attend college, but about half of them are academically unprepared to succeed.

The report found 52 percent of ACT-tested first-generation college students in the 2014 high school graduating class failed to meet the four college readiness benchmarks set by the nonprofit testing organization. Overall 31 percent of all ACT-tested graduates failed to meet benchmarks in English, math, reading and science. More than 9 in 10 first-generation students who took the ACT said they plan to attend college.

"The upside of these findings is that as more first-generation students take the ACT, their access and exposure to the college admissions process is increasing," said Jim Larimore, ACT's chief officer for the advancement of underserved learners, in a news release. "But our research also shows that students' likelihood of enrolling in college right after high school increases based on the number of readiness benchmarks they meet."

The minimum scores students must earn on each of the ACT's four subject tests indicate that students have about a 75 percent change of earning a grade of C or higher in a typical credit-bearing, first-year college course in the corresponding area.

Scrutiny for Accreditors on Poor Student Outcomes

This has been a rough week for higher education accreditors. Days after a Wall Street Journal article raised questions about whether the agencies are doing enough to improve (or, alternatively, shut down) institutions that struggle to retain and graduate students, the committee that advises the U.S. education secretary on accreditation took up much the same theme Thursday at its semiannual session to review some accrediting bodies.

During a review of the Higher Learning Commission of the North Central Association of Colleges and Schools, members of the National Advisory Committee on Institutional Quality and Integrity (NACIQI) -- led by a new student member, Simon J. Boehme -- listed colleges and universities that are in good standing with the commission despite having federal graduation rates of under 20 percent. Some committee members suggested that there should be "bright lines" on key indicators such as retention and graduation below which accreditors punish colleges, which is not typically how accreditation works now. (The discussions Thursday were resonant of similar conversations a decade ago, when then Education Secretary Margaret Spellings pushed NACIQI to put pressure on colleges about whether their students were learning enough, and suggested "bright lines" below which institutions should be penalized.

Officials of the Higher Learning Commission and some members of the accreditation panel noted that the federal graduation rates for some institutions capture small proportions of their total student bodies and may not be representative of how most students are faring. But there was little disagreement that retention and graduation rates at many institutions could be better, and that there was a role for accreditors to play in prodding the institutions they oversee to perform better. Discussion on Friday is likely to return to how that might be done most effectively (and without inappropriate federal intervention).

Also on Thursday, the accreditation panel voted to recommend that the education secretary strip federal recognition from the Accreditation Commission for Education in Nursing, which has been locked for years in a confounding feud with the National League for Nursing over control and independence. Members of the panel expressed confusion over why the two groups had been unable to reach agreement, alternately wondering whether the two organizations were greedy and power hungry. More than one called it "embarrassing" for the nursing profession that they groups returned before the panel, for the third time in four years, having not resolved their differences.

Group's Call to Action for Accreditors

The Council for Higher Education Accreditation (CHEA) has summoned regional and national accreditors to a July meeting that Judith Eaton, the group's president, described as an urgent "call to action."

Eaton, speaking at the group's summer workshop, said a U.S. Senate hearing and a Wall Street Journal article -- both last week -- contributed to the impetus for the meeting. During the hearing, Senators questioned why accreditors had not shut down Corinthian Colleges, a controversial for-profit that disintegrated during the last year. The Wall Street Journal article used public information requests to detail how accreditors continue to recognize hundreds of colleges with low graduation rates and high student loan default rates. "Last week was a game changer," said Eaton.

CHEA does not represent accreditors, but describes itself as a "national advocate and institutional voice for self-regulation of academic quality through accreditation." Eaton said the July meeting would feature a discussion by the accreditors it recognizes about graduation rates and student debt and defaults.

"We're being called on to do something more and different about protecting students," she said.

U.S. reminder to states: You must regulate higher ed

Smart Title: 

After multiple delays, federal requirement on states to approve colleges within their borders is set to take effect -- as U.S. signals importance of state role.

Performance Funding in Massachusetts

Massachusetts' nine public universities will have some of their state support tied to a funding formula based on the number of students they graduate. The state board of higher education on Tuesday voted to approve the formula and will apply $5.6 million to it in the coming fiscal year.

The board described its approach to performance funding in a news release: "It is based on a complex formula of metrics and weights developed by NCHEMS, the National Center for Higher Education Management Systems. An institution’s share of the funding will be determined in part by its five-year graduation rates, annual head count, full-time enrollment, year-over-year increases in degrees awarded, the numbers of students who reach 30 and 60 credit hours each year (with additional points awarded for low-income students who qualify for federal Pell Grants), as well as numbers and types of degrees awarded (with additional points awarded for degrees in 'priority fields' such as STEM, health, business and education). Campuses are also awarded points for 'degree productivity,' the cost of producing a degree per $100,000 in total revenue."

Community colleges in Massachusetts have received some performance funding since 2013. While the formulas are different, the board said both seek to close achievement gaps and to improve the graduation rates of underrepresented minority and low-income students.

Nursing Accreditor Faces Loss of U.S. Recognition

An Education Department report urges the panel that advises the education secretary on accreditation issues to terminate federal recognition of the agency that accredits nursing programs and schools, known as the Accreditation Commission for Education in Nursing. The staff report to the National Advisory Committee on Institutional Quality and Integrity is the latest salvo in a long-running dispute between the commission and the National League for Nursing, the membership association for nursing educators, from which the accreditation commission separated in 1997 at the urging of the Education Department.

Then and now, the Education Department doubted whether the accrediting function operated with sufficient independence from the membership group. The two organizations have been unable to reach agreement over changes in the accreditor's bylaws that would allow it to operate independently.

In the report, released in advance of the advisory committee's June 1 meeting, Education Department staff members express concern "that ACEN could be subjected to interference in its operations by NLN or any other organization or individual other than its own Board of Commissioners," which "would severely affect the agency's compliance with the department's conflict of interest and separate and independent requirements."

SUNY Reaches K-12 on College Readiness

The State University of New York system will create and test a "universal diagnostic to assess and track college readiness among 10th- and 11th-grade students in diverse communities across the state," the system said this week. The pilot program will feature extra remedial support for underprepared students to help ensure they are ready for college when they graduate from high school. All participating students will have access to a forthcoming free online course from SUNY on college readiness. SUNY said it will soon begin accepting proposals for the universal diagnostic, which it hopes to have up and running by fall 2016.


Subscribe to RSS - assessmentaccountability
Back to Top