When institutions and organizations begin to identify with processes instead of intended outcomes, they become vulnerable. They lose sight of their real missions and, when faced with challenges or disruptive innovation, often struggle to survive.
Eastman Kodak, once the dominant brand in photography, identified too closely with the chemical processes it used and failed to recognize that its overarching mission was photography rather than film and film processing. Swiss watch manufacturers likewise identified too closely with the mechanical workings of their watches and lost market share to companies that understood that the real mission was the production of reliable and wearable instruments to tell time. If railroads had viewed their mission as transportation of people and goods rather than moving trains on tracks, we might have some different brand names on airplanes and vehicles today.
In retrospect, it seems that the decisions made by these industries defied common sense. Although the leaders were experienced and capable, they were blinded by tradition, and they confused established processes with the real mission of their enterprises.
Higher education today identifies closely with its processes. In open-access public institutions, we recruit, admit and enroll students; assess them for college readiness; place or advise those who are not adequately prepared into remedial classes; give others access to a bewildering variety of course options, often without adequate orientation and advising; provide instruction, often in a passive lecture format; offer services to those who seek and find their way to them; grade students on how well they can navigate our systems and how well they perform on assignments and tests; and issue degrees and certificates based upon the number of credits the students accumulate in required and elective courses.
We need to fund our institutions, so we concentrate on enrollment targets and make sure classroom seats are filled in accordance with regulations that specify when we count our students for revenue purposes.
At the same time that American higher education is so focused on and protective of its processes, it is also facing both significant challenges and potentially disruptive innovation. Challenges include responding to calls from federal and state policy makers for higher education to increase completion rates and to keep costs down, finding ways that are more effective to help students who are unprepared for college to become successful students, making college information more accessible and processes more transparent for prospective students and their parents, explaining new college rating systems and public score cards, coordinating across institutional boundaries to help an increasingly mobile student population to transfer more seamlessly and successfully from one institution to another and to graduate, dealing with the threat to shift from peer-based institutional accreditation to a federal system of quality assurance, and responding to new funding systems that are based upon institutional performance.
Potentially disruptive innovations include the increasing use of social media such as YouTube and other open education resources (OER) for learning, the advent of massive online open courses (MOOCs), the quick access to information made possible by advances in technology, and the potential for a shift from the Carnegie unit to documented competencies as the primary way to measure student progression.
One of today’s most significant challenges to higher education is the increased focus on student success. In response to calls and sometimes financial incentives from policy makers -- and with the assistance provided by major foundations -- colleges and universities are shifting their focus from student access and opportunity to student access and success. Higher education associations have committed themselves to helping institutions improve college completion rates. The terminology used is that we are shifting from an “access agenda” to a “success agenda” or a “completion agenda.”
This identification with outcomes is positive, but it raises concerns about both loss of access to higher education for those students who are less likely to succeed, and the potential for decreased academic rigor. The real mission of higher education is student learning; degrees and certificates must be the institution’s certification of identified student learning outcomes rather than just accumulated credits.
Faculty and academic administrators, perhaps working with appropriate representatives from business and industry, need to identify the learner competencies that should be developed by the curriculum. The curriculum should be designed or modified to ensure that those competencies are appropriately addressed. Students should be challenged to rise to the high expectations required to master the identified competencies and should be provided the support they need to become successful. Finally, learners should be assessed in order to ensure that a degree or certificate is a certification of acquired competencies.
What would we do differently if, rather than identifying with our processes, we identified with our overarching mission -- student learning? When viewed through the lens of student learning, many of the processes that we currently rely upon and the decisions we make (or fail to make) seem to defy common sense. The institution itself controls some of these policies and practices; others are policies (or the lack of policies) between and among educational institutions; and some are the result of state or federal legislation.
A prime example of a detrimental institutional process is late registration, the practice of allowing students to register after orientation activities -- and often after classes have begun. Can we really expect students to be successful if they enter a class after it is under way? Research consistently shows that students who register late are at a significant disadvantage and, most often, fail or drop out.
Yet, many institutions continue this practice, perhaps in the belief that they are providing opportunity -- but it is opportunity that most often leads to discouragement and failure. Some institutional leaders may worry about the potential negative impact on budgets of not having seats filled. However, the enrollment consequences to eliminating late registration have almost always been temporary or negligible.
Sometimes institutional policies are developed in isolation and create unintended roadblocks for students. When I assumed the presidency of Palomar College, the college had a policy that students could not repeat a course in which they received a passing grade (C or above). But another policy prohibited students who had not received a grade of B or higher in the highest-level developmental writing class from progressing to freshman composition. Students who passed the developmental class with a grade of C were out of luck and had to transfer to another institution if they were to proceed with their education. The English faculty likely wanted only the best-performing students from developmental writing in their freshman composition classes, but this same objective could be accomplished by raising the standards for a C grade in the developmental writing class.
Higher education institutions rely on their faculty and staff to accomplish their missions, so it is important for everyone to understand it in the same way. A faculty member I once met told me that he was proud of the high rate of failure in his classes. He believed that it demonstrated both the rigor of his classes and his excellence as a teacher. If we measured the excellence of medical doctors by the percentage of their patients who die, it would make as much sense. Everyone at the institution has a role in promoting student learning, and everyone needs to understand that the job is to inspire students and help them to be successful rather than sorting out those who have challenges.
"The mission of higher education is student learning, and all of or policies, procedures, and practices must be aligned with that mission if our institutions are to remain relevant."
It is important for faculty and staff to enjoy their work, to feel valued by trustees, administrators, peers, and students -- and for them to feel free to innovate and secure in their employment. As important as our people are to accomplishing our mission, their special interests are not the mission. Periodic discussions about revising general education requirements are often influenced by faculty biases about the importance of their disciplines or even by concerns about job security rather than what students need to learn as part of a degree or certificate program. Before these discussions begin, ground rules should be established so that the determinations are based upon desired skills and knowledge of graduates.
Too often, students leave high school unprepared for college, and they almost always face barriers when transferring from one higher education institution to another. The only solution to these problems is for educators to agree on expectations and learning outcome standards. However, institutional autonomy and sometimes prejudice act as barriers to faculty dialogue across institutional boundaries. It is rare for community college faculty and administrators to interact with their colleagues in high schools -- and interaction between university and community college faculty is just as rare.
Why should we be surprised when students leaving high school are often not ready to succeed in college or when the transition between community college and university is not as seamless as it should be for students? If we are serious about increasing the rates of success for students, educators will need to come together to begin important discussions about standards for curriculums and expectations for students.
Despite the best intentions of legislators, government policies often force the focus of institutions away from the mission of student learning. In California, legislation requires community colleges to spend at least 50 percent of their revenue on classroom faculty. Librarians, counselors, student advisers, and financial aid officers are “on the other side of the Fifty Percent Law.” The ratio of student advisers or counselors is most often greater than a thousand to one. Research clearly demonstrates that investments in student guidance pay off in increased student learning and success. Despite the fact that community college students are the most financially disadvantaged students in higher education, they are less likely to receive the financial aid they deserve. Yet, the Fifty Percent Law severely limits what local college faculty and academic administrators can do on their campuses to meet the needs of students in these areas. Clearly, this law is a barrier to increasing student learning and success. Perhaps state legislators and the faculty unions that lobby them do not trust local trustees and administrators to spend resources appropriately, but this law, in its current form, defies common sense if our mission is student learning.
At the federal level, systems of accountability that track only students who are first-time, full-time freshmen to an institution do not make sense in an era when college students are more mobile than ever and in an environment in which most community college students attend part-time. A few years ago, I met with a group of presidents of historically black universities and encouraged them to work with community colleges to increase the number of students who transfer to their institutions. The presidents told me that doing so could lower their measured student success rates because transfers are not first-time freshmen, and the presidents were not willing to take that risk. Fortunately, officials in the U.S. Department of Education are aware of this issue and are working to correct data systems.
There are many other examples of policies and procedures that seem senseless when viewed through the lens of student learning rather than cherished processes and tradition, just as it seems silly that Eastman Kodak did not recognize that its business was photography or that the Swiss watch manufacturers did not understand that their business was to manufacture accurate and affordable wristwatches.
American higher education today is increasingly criticized for increasing costs and low completion rates. Higher education costs have risen at an even faster rate than those of health care; student indebtedness has skyrocketed to nearly $1 trillion; and college completion rates in the United States have fallen to 16th in the world. In addition, new technologies and innovations may soon threaten established practices.
Challenging the status quo and confronting those with special interests that are not aligned with the mission of higher education can be risky for both elected officials and educational leaders. But given the challenges that we face today, “muddling through” brings even greater risks. Every decision that is made and every policy that is proposed must be data-informed, and policy makers and leaders need the courage to ask how the changes will affect student learning, student success, and college costs. Existing policies and practices should be examined with the same questions in mind. Faculty and staff need to be free of restraining practices so they can experiment with strategies to engage students and to help them to learn.
Colleges and universities are too important for educators to deny the challenges and demands of today and too important for policy makers to pass laws because of pressure from special interests or based on their recollection of what college used to be. Decisions cannot be based on past practices when the world is changing so rapidly. The mission of higher education is student learning, and all of our policies, procedures and practices must be aligned with that mission if our institutions are to remain relevant.
George R. Boggs is the president and CEO emeritus of the American Association of Community Colleges. He is a clinical professor for the Roueche Graduate Center at National American University.
Justice Department halts investigation of discussions among private college leaders on ways to shift spending from non-need-based to need-based aid. Some say the review -- even now that it's over -- has made institutions too nervous to go forward.
The new president of the College Board, David Coleman, has written a letter to College Board members proposing to redesign the SAT. He wants to fix it so the test will "focus on the core knowledge and skills that … are most important to prepare students for the rigors of college." The shift may seem unremarkable but it represents a paradigm revolution in relation to the original test. The old SAT, introduced in 1926, was supposed to be an IQ test, measuring innate ability, not hard-earned subject-specific knowledge of anything. For eugenicists, the IQ argument was a winner; for private colleges, it gave them bragging rights for selecting students with a nationally normed device that coincidentally had a powerful linear relation with family income. Administrative complacency, faculty ignorance, and business office economics have kept the test in play. Why fiddle with a winner?
Between 1926 and today, the test was "redesigned" only once, in 2005. When the University of California threatened to dump the old SAT because it was statistically weak and socially biased, the College Board kept them hanging on by promising a better test – one that would be predictively more powerful and without the social disparities of the old test.
Instead, the 2005 SAT has been a failure on all counts. The new SAT dropped the dripping-with-social-bias verbal analogies and added an easily coached writing section. It took more time, was more expensive, predicted even less well than the old one, and managed to magnify social disparities. Racial, gender, and socioeconomic status test score gaps widened, instead of narrowing. Nonetheless, the College Board proclaimed the new SAT a success; everything was supposedly rocket-science perfect, until Coleman’s letter last week.
But why does the SAT need fixing if it is already, as Coleman says, “the best standardized measure of college and career readiness currently available”? The administrators of the ACT would dissent and slightly more of America’s high school seniors now agree with them. Clearly, part of the reason the SAT needs a remake is in response to a decline in market share. But, paradoxically, another source of pressure on the test comes from new developments inside its true archrival, America’s high schools.
The institution that has done the most to uphold academic standards for generations of America’s college-going youth has not been the College Board; it has been the American high school. Coleman’s formulation on the SAT being "the best standardized measure" is a misleading half-truth; the best statistical predictor of college performance is, and always has been, high school grades in college preparatory courses. It is a myth that America’s high schools are so unreliable (but, of course, not our colleges) that their grades are inflated and meaningless measures of academic achievement.
Even the College Board stipulates in its technical literature that high school grade-point average is the variable that holds the highest statistical correlation with first year grades and with cumulative grades. And high school G.P.A. is the best predictor of who will finish a college degree. High school G.P.A. alone performs better than test scores alone, whether one uses the SAT or the ACT; when combined with high school G.P.A., test scores increase our statistical power by one percentage point, as found at DePaul University, using the ACT, or at the University of Georgia, using the SAT. For me, a variable that raises one’s adjusted r-square in a statistical model by one point contributes diddly to our predictive powers. And what it contributes that isn’t diddly is the transmission of social inequality. There is no correlation between high school G.P.A. and family income; the same cannot be said for the SAT/ACT.
America’s high schools, in reaction to No Child Left Behind and the Obama Administration’s push for transparency and accountability, have given birth to a "common core" standards movement in math and English that has been adopted by 45 states and the District of Columbia. Coleman is intimately familiar with the common core, as one of its architects, and my hat is off to him for that. But one of the consequences of getting a more nationally uniform curriculum is that high school grades will end up predicting even more powerfully than before how well one will do in college, and aptitude tests will be left further behind. America’s schools are where our youths learn the "knowledge and skills" needed for college level work; test-prep for a Saturday morning’s experience filling in the blanks cannot ever do that job. As America’s schools become more uniform and transparent, the fears of unreliability that the test industry preys upon will dissipate.
Another reason the SAT is on the drawing board again is the success of the test-optional movement in higher education. Pioneered by Bates College, and championed by many others, including my own Wake Forest University, more than one-third of America’s colleges do not require the SAT or ACT of an applicant. It is a myth that we need the SAT/ACT to select youths who are prepared to make the most of an opportunity to get a college degree — just as it is a myth that we have perfected a statistical science for doing college admissions. According to the College Board, our statistical models capture about 22 percent of the variance in college grades; the University of Georgia, where the SAT contributed one point, managed to get a model that explained 31 percent of the differences in undergraduates’ first year grades.
Most of what matters to undergraduate performance, 70 to 80 percent of what’s going on, isn’t captured by our best statistical modeling. Admissions remains more art than science, and colleges who look at the whole applicant in search of the best fit between individual and campus do a valuable service. Test-optional colleges have to look beyond the numbers. The ranks of test-optional colleges have grown in the last four years. A tipping point will come when everyone will rush to jump on board, and the admission by the College Board that its 2005 version of the test was a failure brings that day closer to us.
First generation college students (or FGS) comprise a student population that is routinely overlooked at American colleges and universities. These students, whose parents have attained neither a bachelor’s nor an associate degree, are more likely to encounter academic, financial, professional, cultural and emotional difficulties than are students whose parents attended college. However, at a time when budget space is at a premium, many colleges and universities do not have campus programs to help first generation students matriculate. This could be problematic for student recruitment and retention at the nation’s college and universities.
Years ago, one could learn a trade, make enough to support a family, and even comfortably retire. Not today. Even specific trades require additional education. Today’s graduates also report that a college degree does not necessarily guarantee an interview during the current economic recession, when well-qualified and older workers are unemployed and seeking entry-level positions.
However, a college degree is increasingly necessary for an initial interview. As a result of these circumstances, first generation students are increasing at the nation’s colleges and universities. Nearly one in six freshmen at American four-year institutions are first generation, according to a 2007 study by the University of California at Los Angeles’s Higher Education Research Institute.
The topic of first generation college students is often linked to race, ethnicity and affirmative action on many campuses. Programs that assist such students are frequently housed in multicultural or diversity education offices because minority students are also commonly first generation. A 2005 study by the National Center for Education Statistics found that 36 percent of minority students are also first generation. Certainly, research indicates that FGS usually have to traverse additional cultural boundaries such as race and ethnicity, as well as class, when they enter predominantly white institutions.
Some might argue that race and ethnicity matter in a way that economic status does not. Education researchers assert that race, ethnicity, and class are all important, but do not always impact students’ college experiences in the same ways. One recent study found that minority and first generation students have somewhat different challenges. FGS are less involved in fine arts activities, science/quantitative experiences, course learning, and engage less with students who are different from them, but they have greater academic learning gains than minority students.
Given these and other findings, educators must recognize what factors can be attributed to first generation status and/or race/ethnicity, and then target those students’ specific needs. FGS programs could address targeted areas in existing campus initiatives to save costs. For example, orientation workshops could teach first generation students how to effectively take notes and participate in class, just to name a few academic skills. Residential programs and even class assignments could encourage minority students’ learning and involvement with campus activities. I tried this latter possibility in my first-year seminar course at Hope College a few years ago, when I required students to attend campus events and discuss how such events could affect their overall college learning.
Regardless of whether first generation students are racial or ethnic minority students or not, studies show that FGS often lack reading, writing and oral communication skills at the levels of the children of college graduates, which frequently lead to poor retention rates. I was a strong high school student, and even was class valedictorian, but soon after I arrived at Oberlin College I realized that my preparation for college-level work lagged that of my classmates. I learned in high school to study for memorization rather than for analysis. I learned study and time management skills through Oberlin’s Student Support Services (now Student Academic Services). After a semester of hard work, I caught up and later progressed toward graduate studies.
First generation students struggle with more than just academics. With the burden of outside work hours and less parental involvement, such students take part in fewer extracurricular organizations, campus cultural programs, internships, and career networking activities than their peers from middle- and upper-class economic backgrounds. If FGS participate in extracurricular activities, they often delay their involvement until they feel they have their studies under control. Although the hurdles may seem insurmountable, FGS must use campus resources to build social and professional connections.
Campus involvement was not an issue for me because I enjoy helping to make decisions that impact my community. I quickly joined Oberlin’s campus newspaper staff and other organizations. Research has shown that social networks are crucial for academic success. Campus involvements helped me better acclimate to campus culture by giving me a sense of belonging and thus a stronger desire to succeed academically.
Although I quickly found a niche in campus organizations, internship searches and other aspects of professional development were foreign terrains. Non-FGS may be able to consult their parents for assistance with making professional contacts and preparing for interviews. First generation students often do not have an immediate role model for such activities and have to look outward for such support. Because my parents had factory or service jobs, I did not understand the process of applying for a white-collar position. Thankfully, Oberlin’s career office provided excellence assistance in job search strategies, résumé and cover letter writing, and interview preparation.
Adding to the academic, social, and professional challenges that FGS face, these students typically straddle home and school class cultures. The home culture is typically working-class, whereas the academic culture is traditionally middle and upper class. When I was at Oberlin and even long afterward in graduate school, visits home were very stressful because I had to abruptly leave my growing middle-class identity on campus and jump into a working-class one. I sometimes spent school breaks on campus because I did not wish to experience the anxiety of straddling two class cultures.
The stress of managing two class cultures was especially acute during my early college days. I compare my immersion into campus culture to being dropped into a foreign country without a map. To navigate unfamiliar territory, I required certain types of cultural capital that needed to be learned. It would be erroneous to argue that either the working- or middle-/upper-middle class milieu lacks cultural capital. Rather, each social class has its own cultural capital that someone must know in order to function well within that class. Difficulties occur when someone lacks the cultural capital necessary for successful self-management within an unfamiliar social class.
My college experiences were a lesson in acquisition of cultural capital. As a first generation college student in the early 1990s, I was not prepared for the culture shock that I would experience at Oberlin. I thrived in Oberlin’s academic environment, where I befriended other students who were curious about the world, politically- and social justice-minded, and enjoyed learning. However, the campus’ cultural class aspect was initially more challenging because many classmates came with previous assumed experiences that would have been unthinkable in my working-class upbringing. I learned a new cultural grammar for relating to others and just being: What conversational topics were appropriate for dinners with professors, how to make an airline reservation, what outfits were appropriate for professional interviews, and many other aspects of this new middle-class life. Most of these cultural rules were unspoken, making the puzzle of learning them even more maddening.
Many of my Oberlin classmates, professors, and staff were welcoming and eager to help, but I especially experienced class disparities while studying abroad. During the spring of 1993, I planned to study abroad through an Oberlin program in London. I worked five jobs during the summer of 1992 just to save for the travel. For many peers, this was not their first trip abroad. I was dumbfounded to discover that some of my friends’ families even swapped houses with people in other countries or rented cottages abroad. My classmates had an assumed cultural capital of fluency in European languages, travel experiences, and knowledge of art and theater. These would be unthinkable luxuries in my working class upbringing in which vacations, let alone plane trips, were rare. I bonded with several other working-class students enrolled in the London program, and together we stumbled through American and British middle-class culture.
The cultural challenges that first generation students face are real, but they can be the most difficult for institutions to identify because they typically result from unspoken cultural expectations. The baseline of knowledge between first-generation students and upper-middle class students is sometimes vastly different. FGS typically lack certain types of cultural capital, such as exposure to cultural arts that wealthier students might take for granted. And first generation students are often employed for more hours and commute rather than live on campus, which make it even more difficult for them to engage in the campus cultural activities that would acculturate them.
To compound these difficulties, the cultural straddling between school and home is often colored by shame and guilt.
I used to feel guilty about leaving my family behind and ashamed of my working-class origins as I worked and traveled all over the world. Other working-class academics speak about having this "survivor’s guilt" even though they logically know their family is proud of their accomplishments. It took me a few years to learn how to gratefully receive opportunities so that I could as a teacher help others realize their goals through education.
My former shame is now similarly replaced by thankfulness. I am grateful for the diverse class experiences that have given me the bilingual-type ability to fluently speak in different class languages. Cultural straddling helps me relate to people from highly varying class backgrounds and different ways of thinking. As I share my class experiences in the classroom, I find that my self-disclosures give my first-generation and/or working-class students the courage to also speak. I always tell these students that the same skills that enabled them to attend college will help them make their way in the world. After all, we are survivors.
Certainly, institutions can assist FGS by recognizing how contextual factors such as class impact academic performance and cultural transition into college. At a time when every student matters, retention of first-generation college students should be a top priority for colleges and universities. I urge colleges that have no programs to support first generation students to start small and dedicate resources toward specifically helping these students. This can be anything from helping these students navigate study abroad opportunities, extra help in explaining financial aid options and, of course, a bit of hand-holding when it comes to different cultural events.
Researchers have found that even existing programs such as first-year experience courses can be fine-tuned to benefit first generation students. A focus on helping such students graduate not only can strengthen a college or university’s bottom line, but also enables more FGS to create stories of success.
Many educators believe that standardized tests, such as the SAT and the ACT, do not fully measure the spectrum of skills that are relevant for college and life success, and yet, year after year, the same tests hang around with only cosmetic changes. What is to be done?
First, one needs to understand the persistence of standardized tests, and then one can address what can be done. Consider each in turn.
In the beginning of the 20th century, tests like the SAT made reasonably good sense. Most test-takers were upper-middle class or upper-class white males, and so there was little variation in socioeconomic status. Differences in test scores thus largely represented differences in achieved academic skills rather than differences in opportunities to achieve those skills. But as the testing population broadened, test scores more and more reflected differences in opportunity — socialization in the home, school quality, and willingness and ability of parents to invest in the education of their children.
Standardized tests have great staying power for several reasons, described in more detail in my 2010 book, College Admissions for the 21st Century. Here are some of them:
The Appearance of Quantitative Precision. The tests give an exact set of numbers as scores. They thus appear to be very precise. And people are enchanted with the appearance of precision. Many end-users do not fully take into account the concept of standard error of measurement — that a score represents not a pinpoint-accurate level but rather a fairly wide range of levels of possible achievement. End-users may take into account even less the standard error of estimate — that test scores predict only quite imprecisely the criteria they are supposed to predict.
Similarity. To get to a position in which you make decisions about applicants, in part on the basis of their test scores, you had to do well enough yourself on the tests to get into that position of decision-making authority. The fundamental principle of interpersonal attraction is that we are attracted to others like ourselves. So the decision-makers select those who, like themselves, did well on the tests.
Accountability. Admissions officers may be afraid that if they admit students with lower scores and the students do not perform well in college, it will make them — the admissions officers — look bad. If they admit students with high scores and the students do not do well, they always can blame the tests.
Publication Mania. Colleges are evaluated by U.S. News & World Report and other media in part on the basis of standardized test scores. So the system becomes self-perpetuating: Colleges admit students to get high ratings, and to keep their high ratings (or to rank higher), they need to rely ever more on the tests.
Sunk Costs. Society has invested so much in testing — whole admissions and scholarship systems are built around it -- that it is hard to admit that something that once, long ago, may have worked fairly well is today far from optimal.
Rewards for True Believers. As is the case with pharmaceutical companies, those researchers and practitioners who show they are true believers or, at least, supporters, can reap from testing companies financial rewards through funding of research, research awards, prestigious committee memberships, invitations to conferences, and the like. (Disclosure: My colleagues and I were funded by the College Board for the Rainbow Project described below, and by the College Board and the Educational Testing Service for another project on Advanced Placement, funding for which we remain grateful; we could not have done the research without such funding. The results of the both projects were positive — showing that supplementary tests could provide substantial benefits -- and the results were originally published in leading journals -- Intelligence [one article] and Contemporary Educational Psychology [two articles], with secondary publications elsewhere. The College Board stopped funding the research once we got those positive results because, we were told, the research would not lend itself to upscaling. The research has since been upscaled using modified measures, with funding from other sources.)
Superstition. College personnel may see that everyone or almost everyone who succeeds at their college has scores above a certain level. So they conclude that, to succeed, students need scores at or above that level. If they do not admit students with lower scores, they may never find out if the students with lower scores could have succeeded.
Faith. To some people, standardized test are virtually a religion. They believe! They have faith in the tests and do not react well to and may become enraged by those who question their faith, although they might not admit this to themselves. Moreover, the true believers are organized and may become angry, en masse. For some of them, tests are measures of their value as a person: they are their test scores. Because these individuals view themselves as smart — they typically did well on the tests — they see their position as reasoned, not as based on faith.
Free (for institutions)! Students pay for the tests, not colleges and universities, so the institutions get information at no cost.
Tests Work to Some Degree. The standardized tests are predictive, at low to moderate levels, of many outcomes for many groups under fairly diverse circumstances. Whether they add statistically and practically significant prediction of college success over and above that provided by high school grade-point average is less clear and varies from one institution to another. End-users may decide the levels of prediction the tests achieve are good enough for their purposes.
There are sound reasons to wish for more than scores on the standardized tests as they now exist. The tests measure academic knowledge and analytical reasoning skills. They do not measure many of the skills that are truly important for success, broadly defined, in college and life: ethical reasoning, creative thinking, practical problem solving, leadership skills, emotional intelligence, motivation, initiative, self-discipline, ability to delay gratification, belief in self-modifiability, character, belief in one’s ability to succeed in one’s work, and the like. It is understandable that, when the tests were created in the early 20th century, these and other attributes would not be measured: psychologists did not yet know how reasonably to measure them.
Today we have come a long way, but nevertheless are using measures that are roughly the same as those used a century ago. Changes are mostly cosmetic. Imagine if contemporary medical tests still resembled those of the early 1900s! Tests of these other complementary constructs are not as well-developed as conventional tests of knowledge and analytical thinking — but tests of at least some of these other constructs could add to power of prediction beyond what one can obtain merely from conventional tests and potentially reduce ethnic and other group differences. They could provide useful supplements to existing measures.
In our own research on the Rainbow and Kaleidoscope Projects (conducted on participants across the country while I was IBM Professor of Psychology and Education at Yale University and then when I was dean of arts and sciences at Tufts University), my colleagues and I have shown in various refereed publications how tests of creative, practical, and wisdom-based skills can improve prediction of academic and extracurricular success above what one obtains from SAT or ACT alone, and also decrease ethnic-group differences. We found that such tests also increase applicant goodwill toward the assessment process, because applicants feel and report that the assessments measure more than narrow test-taking skills. These broader assessments make a public statement that a college values much more than the academic skills assessed by conventional standardized tests. At Oklahoma State University, we are implementing broader assessments of creative, analytical, practical, and wisdom-based skills in our Panorama Project to predict which undergraduate applicants have the leadership skills potentially to make the world a better place in which to live.
What’s to be done? First, we can dispense with the notion that testing companies represent some kind of "evil empire." They don’t. They are businesses like any other, and their primary end-goal is to make money. They are no better or worse than other typical large companies in the United States. For-profit companies use the profits in part to pay shareholders; nonprofits plow the money back into the business. But they all try to maximize their bottom line. If people keep using their tests as they now exist, test-makers will keep making the same old same old. Why should they change what they believe to be a winning game? And that is the basis for the solution.
Colleges must demand more. When the University of California system threatened to stop using the SAT unless changes were made, the College Board made the changes that the University of California (in particular, Richard Atkinson, at the time president of the UC system) wanted. The College Board presumably did not want to lose such a valuable customer. If end-users demanded more of testing companies, they would get more. If they take what they get and don’t act, they will continue to get what they are getting. Testing companies will change when their consumers demand change — the same as any other kind of company. Organizations, like individuals, can be modifiable if they need to be.
Most colleges are not as large as the California system. So they need to organize, at least, informally. Businesses listen to their customers, or they cease to be businesses. Testing companies are no exception. Like all businesses, what they listen to most is the metaphorical ringing of the cash register — the effect of customer actions on their bottom line. Customers can make all the differences, but they have to act, not just grouse. When they talk, companies may or may not listen; when they act collectively, companies listen. If the companies don’t change the tests, colleges should go test-optional, or introduce supplementary measures, or abandon the tests. It may require a year or two or three to change, but so what? Many universities have taken one or the other path, and they are still surviving and even thriving. The world will not end. There are other measures of academic skills that institutions can use, most notably school grades, which typically are the best single predictor of college academic success.
My hope, of course, is that the testing companies will act, but their past record is not encouraging. At the same time, they have not experienced much consumer pressure at the cash register, except in the past through the University of California system. They need to hear a message that is clear, direct, and that makes clear the consequences of their endless surface-structural change without deep-structural change.
There is one metaphorical fly in the ointment: Many people and organizations have a vested interest in testing as it now exists. They may write items for the tests, publish the tests, publish articles on the tests, feel comfortable using the tests, feel that their self-esteem is threatened if the tests are deemed not to be the measures of human worth they believe them to be, run test-preparation companies, etc. Some of them form a vocal "amen chorus" for tests as they now exist.
The existing standardized tests are quite decent for the kind of limited assessment they represent. But these tests are narrow and incomplete in what they measure. They show high correlations with socioeconomic status and IQ. They thus produce large socioeconomic (as well as ethnic and other) group disparities. Moreover, scores on the tests frequently are over-interpreted — they often are used as though they indicate far more about an applicant than they really do. These tests could be and should be supplemented with modules, chosen by students or colleges, measuring additional characteristics that are important for college and life success.
Why not get rid of the tests altogether? Because they provide potentially valuable, if limited, information. That is why they were created in the first place. The issue is not whether the tests are any good — they are. The issue is that our colleges, and our society, deserve better. And we can get better if we act to force the testing companies to give us the better products we need and deserve.
Robert J. Sternberg
Robert J. Sternberg is provost, senior vice president, and Regents Professor of Psychology and Education at Oklahoma State University. He was president of the American Psychological Association in 2003 and is president-elect of the Federation of Associations of Brain and Behavioral Sciences.