The Decline of Testing Affects More Than Testing

Is standardized testing dead? Or, to paraphrase Mark Twain, are the reports “greatly exaggerated”?

In March 2020, about 1,000 of the 2,300 private nonprofit and public bachelor's-granting colleges and universities offered students the option to apply to their institutions without submitting SAT or ACT scores, and several dozen operated as “test blind,” wherein all applicants were reviewed without testing results. According to FairTest, nearly 1,700, or two-thirds of colleges and universities, as of October 2020 are operating with some form of test-optional or test-blind policy.

Pre-pandemic, only a handful of the most selective colleges in the country were test optional. Today, the new surge includes many of the most selective colleges in the country (e.g., the Ivy League, Caltech). In addition, the University of California system (which received nearly 180,000 applications for fall 2020) announced in May 2020 that it would be test optional for the next two admission cycles, test blind for the two years following, and in 2025 it will administer its own test or eliminate testing altogether. A lower court ruling in November now requires it to be test blind immediately.

As both test agencies canceled or modified multiple testing dates over the past six months, colleges girded for the immediate impact for the class entering in fall 2021. In the aggregate, for the past four SAT test dates, 40 percent of the registrants were unable to take the test.

More students will have no testing or just a single instance to report; hence, the 60 percent increase in the number of test-optional colleges.

We foresee a long-term structural change in the role of standardized testing in college admissions. One of our university colleagues likened College Board and ACT as the big cruise ships of the enrollment management business. The recruitment, selection and ranking of colleges have long depended on standardized testing to keep the system afloat. Perhaps Carnival Cruise Lines may sail at full capacity sometime in the future, but we anticipate that the pandemic-related impacts on standardized testing will fundamentally change their role in college admissions for decades to come.

Peter Drucker observed, “The best way to plan for the future is to plan the future.” How well is your institution positioned to build recruitment practices, selection processes and reputational indices that have significantly less reliance on standardized testing? Here are some thoughts on planning for college admission’s new relationship with testing.

The Funnel

For decades, the purchasing of student test-taker names has been fueling college recruitment’s time-honored funnel. Able to select on multiple criteria from test score to academic interest to ethnicity and more, colleges populated the top of the funnel with tens of thousands of “pre-qualified” prospects. By one estimate, the College Board sold nearly 2.5 million names to over 1,600 colleges and scholarship in 2019; at 47 cents per name, that is a decent income stream. Meanwhile, colleges built continually more sophisticated communication campaigns to move these prospects down the funnel into applicants, admits and enrollees.

It is likely that the demand for testing will decrease significantly in the aftermath of a pandemic-driven move away from requiring standardized testing for admission. College Board and ACT income will suffer, and colleges will need to look elsewhere for prospects. Of course, the testing companies are not the only players in the game. Over the years, entities like NRCCUA (now owned by ACT), Cappex, RaiseMe, College Express and others bombarded admissions deans with promises of perfect prospects for their colleges. Expect the marketplace of student name purveyors to get more crowded in the coming years. And, of course, we can also anticipate more stealth activity on the part of students; the coming generation of prospects will be more apt to “pull” the information they need from the internet rather than wait for the “push” of communications by colleges.

Considering these developments, a funnel plan needs to be dynamic and intentional, characterized by, among other things:

• An institutional website that considers prospective students as its primary audience; admissions must be front and center, navigation must be intuitive, and the CRM must capture the traffic pattern and, perhaps with the help of a third-party, can identify who is searching your site.
• Prospect sources that are diversified and negotiated to ensure quality over quantity.
• A first contact source that must be consistently captured in a CRM.
• A rigorous analysis on how effective different prospect sources are in yielding applicants and enrollees.
• Partnerships with CBOs and other agencies that work closely with students who have been undeserved in the name-buying patterns of the past.
• An aggressive transfer strategy that makes it easy for two-year college students to transfer without losing credit.
• The inclusion of adult students, particularly those who have had some college, along with programs that will serve their needs.

The Selection Process

The use of standardized testing in the selection process has long been a topic of intense debate. Over time, numerous studies have demonstrated a high correlation between test scores and family income. Highly selective college that operate “holistic” admissions and require testing balance test performance bias against other factors, especially high school performance, motivation and socioeconomic-related opportunity. Still, many colleges (along with the National Merit Scholarship Corporation) use testing as a strict criterion in the granting of merit aid. If, as we expect, reliance on standardized testing will never return to pre-pandemic levels, how will institutions pivot to a new selection paradigm tailored to their mission and circumstances?

Of course, there are trailblazers who have long assessed students without standardized testing and show longitudinal evidence that nonsubmitters perform at or above test submitters in various measures of success (i.e., grade point average, retention, graduation). As Bucknell University did when it decided to adopt a test-optional policy in 2018, assessing the experience of such institutions reassured campus stakeholders that the academic integrity in the admissions process would not be compromised. Admissions leaders can also channel the resources of initiatives like Making Caring Common, the Character Collaborative and the Character Lab to incorporate important noncognitive traits in the selection process. The major players in the application forms business, Common Application and the Coalition for College, have changed their templates minimally over the years, but their respective membership is likely to push for more innovative ways to collect evidence of talent.

In the meantime, here are some thoughts on designing a selection paradigm that is less reliant on or completely discounts standardized testing.

• Conduct a rigorous analysis of currently enrolled students to see how various factors (e.g., testing, GPA, academic interest, socioeconomic factors, distance from campus, etc.) impact student success measures. Does this reveal insights that can inform the selection process?
• Look at the attrition and retention of current students with test scores below profile. Were there any identifiable patterns that differentiated those who left the institution vs. those who continued toward graduation?
• Ask the question: Does the selection process reflect the institution’s mission? It is likely that the mission does not cite an average SAT score as a marker of importance.
• The efficacy of a test-optional policy cannot be judged in only one or two admission cycles. At the very least, the performance of at least two entering cohorts must be measured, particularly with respect to first-to-second-year retention. Also, in a time of pervasive uncertainly, college counselors and college-bound students will be leery of a swift shift in a college’s selection criteria.
• Whatever the selection paradigm, we would advise institutions to provide a very explicit explanation of what matters in the admissions process and how students can best present themselves.

Reputation (Rankings)

Few higher education leaders believe that college rankings offer an unquestioned benefit in the college search process. Over the more than three decades of providing its annual ranking of colleges, U.S. News & World Report has periodically tinkered with its formula to address changing circumstances or, as the skeptics might contend, to produce sufficient ranking disruption to generate annual sales. Such adjustments as eliminating yield data because it could be manipulated by colleges and introducing a social mobility factor (weighting Pell recipients and their retention) have not produced much change to an order that overwhelmingly favors wealth and “reputige” (reputation and prestige).

U.S. News has been particularly skeptical about colleges’ motivation to operate a test-optional admissions process. For example, because weak test takers will disproportionally opt out of submitting test scores, colleges can generate a stronger testing profile by excluding them. For colleges whose entering class includes more than 25 percent nonsubmitters, U.S. News applies a penalty (the SAT/ACT percentile distribution value used in the rankings is discounted by 15 percent) to hedge against profile manipulation. While it is too early to tell how many more colleges will report fewer than 75 percent test submitters in the next ranking cycle (and beyond), the U.S. News wheels must be turning as to how its formula will adapt. (On a related note, there should also be consternation on how to account for financial resources in this time of emergency spending.)

From our vantage point, rankings will not go away in higher education’s post-pandemic reincarnation. But for colleges that may have been purposely or incidentally using the U.S. News formula as a tactical playbook, now is the time to develop and communicate a distinctive narrative. Here are some thoughts:

• Explicate the profile of nonsubmitters (i.e., GPA, course rigor, talents, etc.).
• Double down on retention efforts.
• Pay as much attention to outputs (e.g., job placement, student debt, default rate, etc.) as inputs (e.g., selectivity, diversity, school-based achievement, etc.).
• Narrow the message of differentiated excellence to two or three core elements rather than five or more.
• Communicate what families want to know about programs, outcomes and price, and recognize how that might change over time.

The Last Word

For over 40 years in college admission, we relied on the tried and true admission equation: high school rigor + grades + test scores + personal qualities = student excellence. But truth be told, at the highly selective institutions we were privileged to serve, test scores took on a disproportionate role in the final decisions. Blessed with strong, self-selecting applicant pools, test results often became the differentiators.

While we do not believe that standardized tests will disappear, their role in the admission process will most certainly change. As we all work through the challenges of a global pandemic, we would do well to reassess our decision paradigms toward less reliance on tests and more emphasis on a holistic process that takes into account the student’s performance in the context of their environment. The suggestions we have offered above should help enrollment managers and admission officers move in that direction.