assessmentaccountability

Report on Use of Data to Gauge College Attainment

How one gauges the scope and extent of the problem the United States has with postsecondary attainment and crafts possible solutions depends in large part on the data used to assess the situation, argues a report being released at an event in Washington today. The report, funded by the National Science Foundation and produced by American Institutes for Research and George Washington University's Graduate School of Education and Human Development, examines how different data sources offer divergent answers to fundamental questions, such as the rate at which four-year college students earn bachelor's degrees and how big gender and racial attainment gaps are. (Note: Inside Higher Ed's Doug Lederman moderated a panel discussion at Monday's event.)

Performance-Funding Bill Advances in Texas

The Texas Senate on Thursday passed a bill that would require public colleges to meet several performance standards in order to increase tuition rates beyond the rate of inflation. Performance-based funding formulas, while controversial, are becoming more popular among state legislatures. The bill in Texas, which now goes to the state House for consideration, likely will draw national attention.

The 11 performance requirements in the proposed legislation include measures of graduation rates, student completion milestones, the number of degrees earned by at-risk students and the institution's administrative costs. 

In 2003 the Texas Legislature ceded its ability to set tuition rates at the state's public institutions. That move was a response in part to deep budget cuts, The Dallas Morning News reported. But tuition has risen quickly since then, said lawmakers who support the bill.

“The cost of college education has skyrocketed to where students are being priced out of higher education altogether or required to take out exorbitant student loans to finance their education,” said State Senator Charles Schwertner, a Republican, according to the Dallas newspaper.

Colleges begin to take notice of Common Core

Smart Title: 

A growing number of states and colleges are beginning to use Common Core-based assessments to determine student placement and college readiness. 

AACU President to Step Down in 2016

Carol Geary Schneider, president of the Association of American Colleges and Universities, announced this week that she will step down next June. The association, which includes 1,300 member institutions across a broad range of higher education, is the primary group with a focus on liberal education. Geary Schneider has led the association since 1998. Among other projects, she shepherded the creation of the Liberal Education and America's Promise (LEAP) Challenge, an advocacy and research campaign. In a written statement to announce her departure, Geary Schneider said LEAP "represents our shared view of the best way to make liberal education both empowering for every student and renewing for our society at large."

An accreditation veteran responds to the critiques of the system (essay)

First they chopped off our hands. Figuratively, of course. The U.S. Department of Education, over the past decade, has made it clear that it expects accrediting agencies seeking recognition to judge student learning and institutional quality on the basis of rubrics, metrics and measured student learning outcomes. This, rather than on the basis of peer review -- the intense, direct interaction with students and faculty members that enabled site visitors to observe directly the nature, quality and level of learning taking place at an institution.

By now, there is wide recognition that relying on these proposed quantitative measures has weakened accreditation, with collateral damage. Thus, colleges that were focused on a financial bottom line rather than on student learning found it easy to produce numbers that satisfied quantitative guidelines, but said little or nothing about the learning taking place.

Sadly, the focus on such proxies remains. Graduate earnings, for example, are still being proposed as an indicator of quality. Low student debt and gainful employment are other proxies that say nothing about actual learning -- nothing about rigor, challenge, intellectual growth or any of the myriad outcomes that characterize the transformation we associate with a postsecondary education. Because these outcomes cannot be described quantitatively, they have been forced off the accreditation agenda.

Resistance proved fruitless. One small agency, facing the federal panel that recognizes accreditors in 2007, argued strenuously that its own intense focus on critical thinking and scholarship was more effective than the recording of quantitative measures. To no avail -- the agency’s recognition depended on compliance. Ever since, it has been requiring its accredited schools to gather irrelevant numerical outcome data in addition to its continued emphasis on qualitative outcomes.

So if accreditation has been falling down on the job, we now know why. Hopefully this will change with the forthcoming reauthorization of the Higher Education Act. Perhaps this paper -- and others like it -- will engender a new attitude in fashioning legislation and regulation. Accreditors will no longer have to be contortionists, and we should see a renewed emphasis on the searching and effective peer review.

The Peer Review -- A Primer

Peer review is based on site visitors who collectively have the experience and expertise to examine an institution comprehensively. A brief illustration of the thinking of a site visitor examining a college’s laboratory program will help explain peer review.

The visitor would probably think about the number of students working at a piece of equipment. If there are more than two, he will watch to see whether all are engaged -- building, adjusting, measuring, changing -- or whether one person is active and the others are simply taking notes.

The visitor will also note whether the lab instructor is sitting behind a desk or whether she is an involved part of the learning process -- helping, challenging, explaining and working with different groups of students throughout the period. The experienced peer reviewer will watch what happens near the end of the period. Will students hand in their assignments and leave -- or will they continue to work with the equipment to see what else they can learn?

These and other observations will contribute to the in-depth and comprehensive site visitors’ report, which will prove invaluable in enabling the accrediting agency to judge the quality of the institution and help its officials address deficiencies in an effort to improve.

On the other hand, an emphasis on quantitative metrics (amount of money spent on lab equipment, number of students served per day…) reveals little of the reality of a school or its lab.

Are There Alternatives to Traditional Accreditation?

Possibly, but those proposed are so far untested and unverified. Ideas that do have a “feet on the ground” validity somehow turn out to look like accreditation.

It’s helpful to visualize what an alternative strategy for the assessment of colleges and universities might look like. Clearly we would want to examine each institution against its own mission, and to do so we would need an understanding of the student demographics and the kind of learning and teaching taking place there. To be able to determine whether students are progressing successfully, we would certainly want to have a discipline expert in our group, and we would judge the institutional resources to ensure that a supportive educational atmosphere results. We would judge the faculty’s background to ensure that teachers have requisite content knowledge, and we would examine financial and other considerations that make an institution run smoothly.

We shouldn’t be surprised that this approach resembles accreditation. Accreditation has been honed by well over a century of practice at more than 6,000 institutions in the United States, and increasingly, throughout the rest of the world.

Licensure and certification pass rates have been proposed as indicators of quality for colleges offering occupational and vocational programs. But we must consider that if students learn only what is needed to pass a licensure exam, their preparation could be obsolete by the time they graduate. Everyone needs to learn how to learn, to adapt, to cope with changes in a given field, and colleges whose programs are limited to enabling students to pass licensure exams cannot be deemed quality. A far more intense examination is called for, and that’s of course the role of the peer review and accreditation.

 

Examining Proposed Modifications to Traditional Accreditation

(i) Should There Be More Public Member Participation?

There are public members who add immeasurably to accreditation decision-making meetings and just as many who do not. It’s also important to recognize that public members do not represent any special sector of the American people, bring no special expertise and offer no unique perspective. They are representatives from the public, not of the public.

Agencies have discovered that finding a public member prepared to serve, vetting him or her, gathering relevant documents, ensuring that the broad membership approves of this person, and training him or her constitutes a burden that sometimes is not commensurate with the benefit.

Until there is clear, experimentally validated evidence that additional public participation is warranted, no legislative or regulatory change to the current requirements should be made.

(ii) Should Accreditation Be More Transparent?

Accreditation is not a secretive, mysterious process, and on occasion legislators and journalists have been invited to participate in site visits to see for themselves. At a college preparing for a visit, dozens of people are involved. At the visit itself, dozens of others participate. This broad participation is not a recipe for secrecy and a lack of transparency.

Confidentiality, however, is important for decision making, because this enables institutions to be open and forthright with site visitors in describing their limitations. In fact this is a characteristic that distinguishes between accreditation and regulation. Because schools know their internal confidential documents and problems will not be widely published, they trust accreditors. This is why regulation rarely, if ever, leads to disclosure or improvement while accreditation does.

Transparency is neither disclosure nor exposure. Complete disclosure of, for example, all the part numbers of an automobile says nothing to the consumer about its functionality. Nor is transparency exposure. Releasing accreditation reports about institutions to the public ensures that schools will hesitate to discuss their limitations and challenges; it will occasion site visitors to write defensively and result in a distorted picture of an institution being presented to the public. 

Transparency properly relates to the needs of the public and makes it possible for members of the public to receive the information that will make a difference, without at the same time compromising the accreditation process. 

This is worth reiterating. The strength of accreditation lies in the fact that colleges are prepared to disclose, to be frank, to share limitations, to open the door to examination of all aspects of their activity, including the most embarrassing situations that might subject them to disfavor.

Accreditors are not regulators. Regulators discover only what they happen to see during their visit. Trusted accreditors discover everything. 

(iii) Should There Be a Common Language?

Every step taken to unify different accrediting agencies is a step toward compromising the very diversity that makes American higher education so strong. Of particular concern is the fact that anything that is defined as common practice inevitably slides into becoming required practice. American higher education flourishes partly because there is competition; there are differences and there is an opportunity to examine the impact of different approaches in different situations.

We must ever be alert to the dangers of a unitary, “ministerial” form of higher education. Our diversity, competition and freedom to try new and different things should never be limited or compromised.

Alignment is not an overriding imperative, nor is comparability. Sometimes these considerations are helpful, but they should not be permitted to lead to a homogeneity that precludes responsible change and innovation in accreditation.

(iv) Should Accreditation Be More Proactive to Innovation?

Most so-called disruptive innovations are focused on inputs. MOOCs, experiential learning, competency-based education and technology of all kinds speak largely to different means of delivering education -- that is, to inputs.

Calling for innovation says little about quality or outcomes, and this is precisely why accreditation is so necessary in fostering innovation. It is only if an innovative approach is measured against standards, and performs well against a template of excellence, that we can agree that learning is taking place. Innovation, without the careful review associated with accreditation, risks harming students because of deficiencies that can emerge after graduation.

An innovation that does meet the standards of an appropriate accrediting body moves much more quickly and effectively into the mainstream than would otherwise be the case. In this respect accreditation can be seen to be fostering innovation.

(v) Should There Be Different Levels of Accreditation?

Perhaps. At the moment, standards operate to establish a floor, delineating the minimum quality necessary for institutional or program acceptability. It might be desirable to create gradations of accreditation status. Will such gradations make a difference? No one can tell, and careful experimentation is called for before any permanent changes are made.

Accreditation as Gatekeeper to Student Financial Aid

Accreditation as gatekeeper has become a target, and it’s important to outline the reasons that keeping this feature is extremely important to a well-functioning postsecondary educational system.

(i) Maintaining Order and Fostering a Culture of Improvement

The gatekeeping function helps create a sense of order in American higher education. All reputable institutions are recognized by agencies that, in turn, satisfy the regulations of the Department of Education and the standards of the Council for Higher Education Accreditation.

There are problems with this structure, but because of the role of Congress, the tendency of the Department of Education to overreach can be checked. Every reauthorization of the Higher Education Act has helped remind the department that it serves education; it does not guide or control it.

Now visualize a situation in which student financial aid would flow to colleges and universities whether or not they are accredited.

Since they would remain eligible for aid without accreditation, elite institutions with strong, well-established reputations will not find it necessary to seek to be accredited. Some might join together in an exclusive club, but there is not likely to be a pretense of seeking to improve or to adhere to any external standards. Such schools will continue to be able to attract excellent students and donor support on the strength of their reputations alone.

The very lowest-performing colleges in the country are not likely to seek accreditation, either. At best, some will gather to create an accrediting body with indifferent standards, as a marketing feature. Lost in all of this will be the culture of improvement that is integral to accreditation. Schools do respond to site visit recommendations, and if site visit reports are not as incisive and comprehensive as they used to be, they are still effective in charting a path for improvement.

(ii) Diploma Mills Will Proliferate

As noted, the gatekeeper function ensures that all reputable schools seek accreditation from agencies that have Education Department recognition. Indeed, the department maintains a list online of all schools accredited by recognized agencies. Diploma mills are not on this list, nor are they eligible for student financial aid. If accreditors are no longer gatekeepers, many legitimate schools will not seek accreditation yet will receive financial aid. So will diploma mills. People who create diploma mills are extremely resourceful, and will be able to meet any paper requirements needed to become eligible for SFA funds. It’s only accreditation as gatekeeper under the Education Department that prevents them from doing so now.

Further, without the department's recognition function, there will be nothing to prevent a dishonest operator from creating an accrediting body (such as the “Northeast and Southwest Association of Colleges and Schools”), listing several hundred legitimate schools as being recognized by this association, and adding a number of diploma mills to its list.

True, this fraudulent accrediting body would not be admitted to the Council for Higher Education Accreditation, but the unsophisticated and unsuspecting enrollee in a diploma mill would not know the difference. And if necessary, the same printing press that created schools out of air, and accrediting agencies out of fairy dust, can readily create a CHEA-like body as well.

(iii) Transfers Will Be More Difficult

Colleges hesitate to accept transfers from community colleges and weaker institutions without reassurance from an external body. As difficult as it is now for students to transfer credits, it is still possible to move from a weak accredited school to a stronger one. In the future, with no recognized accreditor to offer lose control over their practices, colleges will be free to institute arbitrary and unduly restrictive transfer policies.

(iv) Innovation Will Be Stifled

As noted above, innovation makes sense only if it is responsible and consistent with the standards of higher education. It is useful to students only if their efforts in an innovative mode are subsequently recognized by conventional schools and employers. Accrediting agencies, even at the risk of being accused of being obstructionist, have insisted that innovation not result in any diminution in the quality of education.

Innovators seek accreditation, because once approved, a changed structure, strategy or delivery mechanism can be assured of acceptability and wide implementation. Without a broad-based, unified accreditation structure, tied together by a gatekeeping function, acceptance of innovation will become more difficult.

(v) Public Protection

There are many emerging areas of public activity, particularly in the field of health services, where there is no government oversight and much potential for public harm. Access to financial aid encourages accreditors to seek federal recognition. This is why responsible educators and practitioners in a given field organize themselves around an accrediting structure that will meet Education Department standards. This helps create a level of assurance of quality, which serves the public.  

 

The advent of the Internet and the ease with which false and misleading structures can be created makes it important to keep accreditation intact, particularly since there have been no structured experiments and pilot programs to examine proposed alternatives. Conversation is healthy, but nothing should be done to dismantle the existing gatekeeping/accrediting relationship until it is clear that a replacement is at least as good as the original.

Bernard Fryshman is a professor of physics and former accreditor.

Editorial Tags: 

Rich New Resource on Postsecondary Attainment

A new federal report presents a wealth of data about how 2002's 10th graders fared in higher education (and not) a decade later -- potentially offering researchers and policy makers enormous insight into who attains postsecondary success and why.

The report offers a first look at new data from one of the U.S. Education Department's most important longitudinal research studies, the Educational Longitudinal Study of 2002, which followed 10th graders through to the 2012-13 academic year. Eighty-four percent of those high school sophomores went on to at least some postsecondary education within that decade, while 16 percent did not, with those variations differing, somewhat predictably, for certain demographic traits (women were more likely to go on than men, students from wealthier socioeconomic backgrounds were more likely than their peers, etc.).

For those who went on to postsecondary education, the study examines what they attained (how many credits earned, whether they earned a credential and, if so, what kind), when and in what kind of institution they enrolled (13 percent of students attended a two-year institution first and then a four-year college, and 12 percent did the reverse), how they performed in terms of grade point average and other outcomes, and how many times they stopped their studies.

The report also includes data on the proportion of undergraduate credits that students actually earned versus those they attempted, and provides a slew of information on the characteristics of students who took at least one remedial course.

New Reports on Competency-Based Education

The American Enterprise Institute's Center on Higher Education Reform today released two new reports on competency-based education, which follow a report the center released in January. The first paper uses results from a survey of hiring managers at companies around the country to learn about employers' perceptions of the emerging form of higher education. The survey found that while employers' overall awareness of competency-based education is low, those that do know about it have a favorable view.

The center's second paper seeks to describe best practices for the assessments that competency-based programs use. The report argues that the credibility of this form of higher education hinges on the quality of those assessments.

Helping students make better decisions can improve higher ed equality (essay)

The federal government has made significant investments over the last several decades toward reducing socioeconomic inequalities in college access and success: hundreds of billions of dollars in financial aid; a host of informational tools to provide students and families with better information about college quality and costs; prominent attention from the White House, including not one but two presidential summits in 2014 on expanding college opportunity for economically disadvantaged students.

Yet as we all know, gaps in college completion by family income have actually widened over time. How can this be?

Simply creating a financial aid system and college search tool kit isn’t enough. We have to make sure students and families know about these resources and can easily access them.

Think about Apple products for a minute. Why do we buy iPhones, iPads and MacBook Pros? First and foremost, they are durable, high-quality devices. We imagine some readers may be able to rattle off the technical specifications that make these devices superior to their competitors.

But there are four characteristics that really hooked us on Apple products: (1) They are easy and intuitive to use. It’s unlikely the iPhone would have gone viral if the average user needed to spend an hour reading through an instruction manual before getting started; (2) The devices have a sheer stylistic appeal -- sleek metallic casings, glossy touch screens; (3) Apple has been incredibly savvy in its marketing and advertisements -- Steve Jobs’s legendary launch events, crisp Apple television and print ads; (4) The social atmosphere of the Apple stores and skill of the Genius Bar technicians make getting help when problems arise an enjoyable experience.

Now think about the Pell Grant, or the College Navigator search tool, or federal loan entrance counseling. The maximum Pell Grant is worth over $5,000 a year, so one could argue that the product quality is in place. But the Pell Grant falls short on other dimensions that make Apple products so successful. The grant is not nearly as well marketed, so some students and families aren’t aware that it exists or that the money doesn’t need to be paid back.

And to access the money, students and families need to complete a cumbersome and confusing financial aid application. Imagine if, to get your iPhone, you had to first fill out a complicated rebate form, send it in and wait for a few months for the device to arrive.

College Navigator has literally hundreds of data points on every college and university in the country, but this is as much a problem as a benefit -- too much information overwhelms the average user. There are also design limitations -- the site is set up for people who know what to look for and how to interpret all the information they see.

There’s little guidance for students about how to structure a college search, or which data points to prioritize over others. And College Navigator has an even bigger name-recognition problem than the Pell Grant. Our guess is that a small fraction of first-generation college students in the country has even heard of the tool.

We can learn a great deal from companies like Apple. We’ve also learned a lot from the burgeoning science of decision making over the last several years. In the face of complex decisions and complicated choices -- like deciding where to apply to college or navigating the financial aid process -- people have a common set of responses.

One common behavior is to put off making any decision at all. Another common response is to use a simplifying strategy to make a decision, like choosing which college to attend based on where your friends have gone to school or a connection with a particularly charismatic tour guide. And in some cases, people don’t make any decision at all -- they just follow the path of least resistance. For middle-income students, this might mean enrolling in the nearby public university. For students whose parents did not go to college, it might be looking for a local job after high school graduation.

Research from fields like behavioral economics, psychology and neuroscience has helped us recognize what private sector companies like Apple have known and exploited for decades: just having a good product or policy isn’t enough. For policies to achieve their desired aims, we need to do what Apple does -- develop high-quality products, and then devote just as much attention to publicity, consumer engagement and customer service as we do to policy development.     

What does this mean in practice?

Nudging students about important tasks. Especially in this day and age, adolescents often balance a multitude of academic, social, work and family commitments. These responsibilities, on top of the fact that adolescents frequently struggle with organization and long-term planning, mean that even students with clear intentions to start or stay in college may miss important deadlines. Simple strategies like sending students text message reminders to renew their federal financial aid can help students translate their intentions into concrete actions.

Improving the design of publicity materials. There’s a reason Apple print ads rely on striking visuals and minimal content: people tend to glaze over dense text. Yet much student-facing communication is incredibly text heavy. By simplifying the content of letters, emails and websites and incorporating behavioral cues for students to take action, we can more effectively help students take advantage of the opportunities and resources that are available to them.  

Simplifying enrollment processes. Researchers and advocates have devoted considerable attention over the last decade to how complexities in the federal financial aid application process can deter college-ready and financially eligible students from receiving aid. Reducing hassles associated with completing the Free Application for Federal Student Aid (FAFSA) -- either by simplifying the form itself or by making it easier for students to get assistance -- can lead to substantial increases in the share of students who receive aid and enroll in college.

The student loan origination process is another important decision-making bottleneck where highly complex information may inhibit students from making informed choices about how much to borrow. Simplifying information about borrowing and increasing access to loan counseling can help students make more informed choices about borrowing levels that are a good fit for their personal circumstances.

Changing default options. Several important stages in the college-going process -- taking college entrance exams, choosing courses once in college -- require active steps on the student’s part. Failure to take action can lead students to miss important opportunities, like getting into key prerequisite courses for their intended majors.

We can change the default option so that, for example, the curriculum is laid out for students unless they actively make different choices. Several states have shifted to mandatory college entrance exam testing to increase the number of students who take the S.A.T. or A.C.T. Several colleges have employed active course mapping that provides students with a scripted set of courses to take that will help them complete their intended majors in the least amount of time possible. Students can opt out of these course maps, but only by taking active steps to meet with an adviser to discuss alternative course options.

The greatest appeals of these approaches include their relative ease of implementation, low cost and scalability. A rapidly growing number of academic, public and private-sector ventures are applying behavioral insights to improve postsecondary access and success. Some of these initiatives have been rigorously evaluated through randomized controlled trials and have generated substantial improvements in students’ outcomes. We bring together insightful essays on many of these innovative approaches in our forthcoming volume, Decision Making for Student Success.

Behavioral solutions alone won’t eliminate socioeconomic inequalities in postsecondary access and success. But for a relatively small investment in these strategies, we can meaningfully improve the efficacy of existing programs and policies and expand college opportunity for hardworking but economically disadvantaged students.

Ben Castleman is an assistant professor of education and public policy at the University of Virginia. Saul Schwartz is a professor in Carleton University's School of Public Policy and Administration. Sandy Baum is a senior fellow at the Urban Institute.

Compilation of Articles on Accountability Debates

Inside Higher Ed is pleased to release today "New Debates About Accountability," our latest compilation of articles. As with other such print-on-demand booklets, the compilation groups together news articles and opinion essays representing a range of views. The booklet is free and you may download a copy here. And you may sign up here for a free webinar on Wednesday, April 29, at 2 p.m. Eastern, about the themes of the booklet.

Chegg Expands Counseling Services With InsideTrack

The student services company Chegg on Monday announced its users will soon be able to subscribe to career counseling from InsideTrack. The counseling service, currently in beta, will launch next month. 

Chegg, which made a name for itself as a used textbook provider, has moved quickly to invest in student services. Among its recent moves, the company has launched a college counseling platform, acquired Internships.com and embedded its services in Blackboard's learning management system.

InsideTrack has provided career counseling to students (in addition to its much larger service providing academic and life coaching) through their colleges for about three years. But the partnership with Chegg represents its first major effort to provide services directly to students, using a multimedia platform it has built in the wake of its purchase last year of Logrado.

Pages

Subscribe to RSS - assessmentaccountability
Back to Top