Unlike novelists and playwrights, journalists tend to lead with the best material we’ve got. So here’s mine: In the controversy between U.S. News & World Report and Sarah Lawrence College, a college is alleging that a national magazine is making up data, so as to allow said magazine to continue conducting its university rankings. But in Canada, one of the country’s leading universities is making up data — in an attempt to prevent a national magazine from conducting university rankings. Readers of Inside Higher Ed were last week urged by Indira Samarasekera, the president of that university, the University of Alberta, to follow the Canadian lead. American university presidents might want to learn a bit more about what is actually going on in Canada before taking that advice.
The example set by a number of Canadian universities in 2006 involves a level of information suppression that I suspect neither the American public nor students, not to mention university administrators, would be able to countenance. It is one thing to decline to turn over SAT scores to U.S. News, when your college is no longer gathering SAT scores to evaluate students. It is quite another to refuse, as some Canadian universities did, to make public your university’s average entering grades, or its graduation rate, or the number of international students, or its faculty count. Yet that is what happened at many Canadian universities in the fall of 2006.
Rankings were intended for students, and to put the focus on students and on quality. Unfortunately, it is administrators, in both Canada and the United States, who have become the most engaged and obsessively implicated audience. That result has not been for the better for either rankings or universities. At Maclean’s, we have good reason to believe that students use our rankings as intended: as one of many sources of information about universities, and one source – an important source, but not necessarily the most important source – to help them and their families make their university decision. Students give our rankings the appropriate weight. And university administrators? Perhaps we should be flattered, but they often seem to accord our rankings a weight that can only be described as disproportionate.
Maclean’s, Canada’s main national newsmagazine, has been conducting a ranking of Canadian universities since 1991. The goal is to provide a broad evaluation of the quality of undergraduate education at 47 ranked universities. All are public institutions. Maclean’s constructed its original methodology in cooperation with Canada’s universities, and has modified and improved it over the years, in consultation with the universities. For example, in 2003, at the request of many universities, Maclean’s added retention rate as a new rankings measure. In 2002, the magazine modified its class-size measure and introduced a grade distribution chart, both steps taken in light of feedback from universities. In her article, Dr. Samarasekera says that Alberta high school grades cannot easily be compared to those from other provinces, in part because Alberta has grading system, where a “student’s final achievement level is defined by a graduation exam not used in other provinces.” However, the Maclean’s ranking for a number of years has attempted to take account of that difference. We ask Alberta universities to provide their first-year students’ average high school leaving grades. We do not request provincial exam results, even though those exam results are used in Alberta university admission decisions. That change was made at the behest of Alberta universities.
Dr. Samarasekera writes that she has “just learned” about Maclean’s introducing a new issue devoted to research, graduate schools and professional school. In fact, I wrote Dr. Samarasekera and other university presidents to announce the issue last August. It is a response to complaints from many large research universities that the rankings, which evaluate undergraduate education, leave readers with the mistaken impression that undergraduate education is all a university does. In that letter to presidents, Maclean’s also introduced its new online Personalized University Ranking Tool. Dr. Samarasekera and others had argued that, since every student is unique, each should be able to build a unique assessment of universities, based on her own preferred criteria. We agreed. Since last November, students and parents have been able to go online and instantly create a personalized ranking of Canadian universities, picking and choosing from Maclean’s data to build an assessment of what matters most to them.
In the rankings, published each November, we group universities into three categories – medical-doctoral universities, comprehensive and primarily undergraduate. Universities are evaluated based on 22 to 24 indicators, depending on the category. And though the rankings give some weight to the university’s reputation, as judged by a survey of knowledgeable observers and peers, it is considerably less than the weight in the U.S. News survey: 16 percent vs. 25 percent.
If universities were like governments or publicly-traded corporations, the data we seek (and so much more) would all be public. Journalists would focus on the job of covering universities, rather than the task for trying to pry loose some snippets of basic comparative data. The task of the editor of the Fortune 500 -- and I used to be the editor of a magazine that publishes the Canadian equivalent -- is relatively simple, at least in terms of information-gathering. Performance measures for public corporations are all easily available to anyone with a computer and a few minutes to spare. You don’t have to ask companies for permission to publish their top-secret earnings-per-share numbers. The information isn’t top secret.
Some of the data points that Maclean’s uses in its evaluation can be gathered by Maclean’s itself: the aforementioned reputation survey, along with the number of national awards won by students and faculty at each university, from Rhodes Scholarship and Fulbright Awards to 3M Teaching Fellowships and Guggenheim Fellowships. Some of the information is available from third party sources: four measures of library size and expenditure; each university’s operating budget per student; the percentage of budget spent on student services and scholarships and bursaries; and the number and value of research grants obtained from the three national granting councils covering medical grants, science and engineering, and social science and humanities.
And then there is the information that we have always had to request directly from universities. This information is largely quite basic, and I suspect that many readers will be stunned to discover that a number of Canadian universities in 2006 simply refused to make this information public. The key information that Maclean’s must ask universities for directly, and which the boycotting universities refused to provide includes:
The number of first- year students.
The percentage of first-year students from outside the province.
The percentage of first-year students who are international students.
The percentage of graduate students who are international students.
The average high school grade of first year students, on a scale of 100.
The proportion of first-year students with a high school leaving grade of 75 per cent or higher.
Graduation rate (percentage of full-time, second-year students who go on to graduate within the expected time frame).
Alumni giving, as percentage of alumni who give.
Number of faculty.
The percentage of students in enrolled in each of six class-size ranges (1 to 25, 26 to 50, 51, to 100, 101 to 250, 251 to 500 and over 500).
The percentage of first-year classes taught by tenured or tenure-track professors.
The percentage of faculty members with Ph.D.'s
Universities that refused to make such this information public invoked a variety of justifications, and since we don’t have the space to consider them all, allow me to examine some of those cited here.
According to Dr. Samarasekera, comparing university entering averages across Canada is unfair and inaccurate, because Alberta high schools “employ a different grading system -- believed to be more rigorous” than that in other provinces. This is something that Alberta universities have long asserted. What Dr. Samarasekera failed to mention is that for students commencing studies in fall of 2004 -- the most recent year for which University of Alberta entering grades are available -- the University of Alberta, a very large university with more than 35,000 students, had one of Canada’s highest entering grade averages. Alberta’s 2004 entering grades, as displayed in the 2006 Maclean’s rankings, were in fact higher than those of 15 out of 17 ranked Ontario universities, including the University of Toronto, a research powerhouse that until last year was the perennial number one finisher in the medical-doctoral category. And if one compares the 2004 Alberta entering average to the 2005 entering averages of Ontario universities, one finds that the University of Alberta has a higher entering average than any Ontario university. Given these facts, it is hard to argue that Alberta is suffering because of grade inflation elsewhere.
Readers may be wondering: Why I am citing the University of Alberta’s entering grades for students admitted in the fall of 2004? That’s a long time ago; those students are now juniors. Surely more up to date information is available? No, it isn’t. The University of Alberta has released an average entering grade for the first-year class admitted in the fall of 2005, but it is a made up number. The data is an invention. The University of Alberta knows what the real entering average of the 2005 freshman class is, but it has instead chosen to publish a fabricated number. You can view it on the university's Web site.
According to the university, the entering average in 2005 was 88 per cent. However, it puts a big asterisk next to the 88 per cent, noting that this is not the actual average high school grade admitted students received, but a modified grade, which has been “adjusted for Canadian inter-provincial variables: grading practices, age group participation rates, student preparedness (international PISA survey by the Organization for Economic Cooperation and Development).” So what is the average high school grade of Alberta students? The real, pre-adjusted grade? How has it been adjusted? We don’t know.
Last year, I on two occasions met with Dr. Samarasekera, and I met at other times with her provost and other senior officials. At these meetings, the university demanded that Alberta grades be adjusted. Maclean’s asked them to propose a formula by which this could be done. Don’t just make a demand, we said, make a proposal. Let’s get a national dialogue going. The response: nothing. But the university has, apparently, come up with a formula. They just won’t tell anyone what it is. We have asked the University of Alberta for the formula since last November, and have received no answer. Maclean’s isn’t making up information, but a taxpayer-funded university is. Is this a model U.S. universities wish to follow?
Dr. Samarasekera also complained about a Maclean’s survey of recent graduates, which we conducted in the spring of 2006. She says that one of the reasons the university is unwilling to provide data for the Maclean’s ranking is this graduate survey. She says that “in the case of the graduate survey, we argued that surveying alumni reflects an institution’s past, not its present, particularly in a province such as Alberta, where the government has poured billions of dollars into postsecondary education in the last few years.” Just to be clear, the graduate survey was not part of the fall ranking; it was included in an issue published in the spring. The existence of a graduate survey conducted six months before the rankings may seem like a strange reason to pull out of the rankings. It will seem even more implausible when you learn that, when it comes to student surveys vs. alumni surveys, I listened to Dr. Samarasekera and we have largely followed her advice.
In 2006, in lieu of participating in our graduate survey, Maclean’s asked the University of Alberta and other universities to make public the results of two national student surveys, the Canadian Undergraduate Survey Consortium (CUSC) and the National Survey of Student Engagement (NSSE), which has become popular in both Canada and the U.S. Both are surveys of current students. In 2007, we did not conduct a survey of graduates, but instead asked all universities to make public their most recent CUSC and NSSE results. In other words, we published surveys of current students, commissioned by the universities, imposing no additional costs in time or money on the universities, and providing potential students with information previously accessible only to university heads. Alberta participates in both CUSC and NSSE, and its results, along with those of nearly every other university in Canada, are featured in the Univesity Student Issue, an issue of Maclean’s now on newsstands.
Many universities were unhappy with Maclean’s publishing these student surveys. Some universities do not have the most fantastic results. But as I repeatedly explained to every member of the press who talked to me about the data, the fact that the University of Alberta does relatively poorly on both CUSC and NSSE does not make it a bad university. It does not mean that you should not enroll there. But surely it is a piece of information the students might want to know, and have a right to know. The same goes for this fascinating finding from NSSE: every ranked Canadian university — all 28 of them — has a NSSE benchmark score for “student-faculty interaction” that is lower than the average of American colleges. Does this make these institutions to be avoided? No. But it does raise interesting questions.
On my blog, you’ll find a more detailed critique of Dr. Samarasekera’s article and arguments. But I want to end on a hopeful note. I believe that there is reason to be hopeful. Out of the battles of 2006, even those Canadian universities that were most opposed to the Maclean’s rankings have come to realize that their way of solving this “problem”– namely through suppression of basic information— is no solution at all. Many universities, including the University of Alberta, are taking part in discussions to create what is being called the Common University Dataset-Canada. If this is done right, and that remains a big if, it would mean that the public – journalists included – would have access useful statistics on each university’s operations and performance, measured on a common basis. Discussions are still in the early stages, but the ideal is a sound one, and it is achievable.
As Dr. Samarasekera wrote earlier this week, in her final bit of advice to American universities, “remember to stay united, don’t put anything in writing you don’t want FOIAed, involve your stakeholders – students, faculty, governing boards – and make your data public and easily accessible for any who wish to find it.” If Canadian universities merely would follow the precepts of the last item on that list, as many of them did not in 2006, all else would be unnecessary. Journalists like me could then spend most of our time covering higher education, instead of engaging in an endless paper chase. We could focus on our jobs. So could university presidents.
Tony Keller is managing editor for special projects at Maclean’s.
The United States is part of the Americas. Hence not all Americans are citizens of the United States -- and it is a sign of imperial hubris to treat those terms as synonyms.
Or so runs a bit of routine language-policing, as practiced by many well-intentioned people. By many well-intentioned Americans, one should say – meaning "citizens of the United States." I know because I used to be one of them. Then, a few years ago, while on vacation in Canada, my wife and I had an odd conversation with the woman who ran the place we were staying. When she used the expression “you Americans,” our half-baked cosmopolitan reflexes kicked in.
“You’re an American, too,” we insisted. “Canada is part of America!”
Our landlady thought this was crazy. Being Canadian, she was too polite to say so. But no amount of argument could persuade her that a person born and raised in Toronto could be an American. The very idea was absurd. And from paying attention to the news media, we soon learned that she was not being idiosyncratic. On that side of the border, the word “American” applied only to someone from the south. (And not too far south, either. While Mexico is undeniably on the North American continent, the expression norteamericano is not one that Mexicans use to describe themselves.)
So which is the worst case of verbal imperialism? Is it the unthinking use of “American” to mean someone from the United States? Or is it forcing the word upon people who emphatically do not consider themselves Americans? Endlessly absorbing as this miniature paradox of political correctness may be, I’ve found my interest shifting in the course of subsequent trips to Canada. Where is the real line of distinction between the countries -- apart from the border, obviously?
One joke has it that a Canadian can be defined as an American with health insurance and no guns. By that standard, my wife and I are already Canadians, and from time to time we discuss moving there at some point in the next couple of decades. Especially if there is ever a president named “Jeb.”
But even while daydreaming of relocation, you know there is a stronger sense of Canadian national identity than that -- resting on differences in history and culture that are large, but unclear, at least from this side of the divide.
Actually, even that may be a misleading way to put it. In fact, almost nobody here in the States thinks about the difference. Our default outlook is best summed up by “Blame Canada,” a rousing number in the "South Park" movie, which contains the line “It’s not even a real country anyway.”
This is satirical, of course -- a send-up of how fast arrogant indifference can turn to belligerence. But the sardonic and foul-mouthed "South Park" lyricists have perhaps tapped into something that Canadian cultural critics themselves have, by turns, celebrated and deplored: the idea that the national identity is hard to grasp because Canada isn’t a “standard” nation-state. Political power is fairly decentralized, with the provinces retaining a lot of authority, if not autonomy. The legal category of Canadian citizenship only came into existence 60 years ago; before that, one had simply been a British subject living in Canada. The ethnic and linguistic composition has always been heterogeneous. And while its expanse makes it the second largest country in the world, most of the land is very thinly populated.
That’s not quite the same thing as saying “it isn’t a real country anyway,” by any means. But it makes for a relatively ad hoc and open-ended situation in defining the national consciousness. In 1970, Allan Smith, now an associate professor of history at the University of British Columbia, published an influential paper called “Metaphor and Nationality in North America.” (It is reprinted in an interesting collection of Smith's papers.) He contrasted the American idea of the national “melting pot” and the preferred Canadian trope of the “mosaic” of different cultures.
“American nationalists have seen their nation as a vessel containing a single, virtually unblemished way of life,” wrote Smith, “and their language has, accordingly, been confident and assured. They have known who they were and what they believed, and their vocabulary has reflected the pride and security that this knowledge has brought. Canadian nationalism, in contrast, has been less exuberant and more diffident because it recognizes how fragile and uncertain is the structure it tries to celebrate, and how delicate must be the touch of they who would work all of its parts into a cohesive whole.”
While in Montreal last week, I picked up a new book called The Unfinished Canadian: The People We Are (McLelland & Stewart) by Andrew Cohen, an associate professor of journalism and international affairs at Carleton University, in Ottawa. As one reviewer there said, it “has already become a Canadian best-seller, which means that more than 5,000 copies have been sold.” (Now there's a national trait: the Canadian knack for self-deprecation is quite well-developed.)
One complaint lodged against Cohen’s book is that it merely recycles discussions of national identity that are familiar to any well-informed Canadian. For the clueless American reader, that actually qualifies as a recommendation. Most of us did not know, for example, that one of the major nonfiction books up north during the past few years was called Fire and Ice: The United States, Canada, and the Myth of Converging Values (2003).
The author, Michael Adams, is a well-known public-opinion analyst, and the findings from his polls of Canadians and Americans in 1992, 1996, and 2000 suggested the emergence of a growing gap between the countries. And the fact that his book appeared two months after Canada declined to join the Coalition of the Willing certainly made this a timely claim.
Asked whether they agreed with the statement “The father of the family must be master in his own home,” Adams reported that 49 percent of Americans over the age of fifteen did, while the figure from Canada was just 18 percent. A quarter of Americans believed that “non-whites should not be allowed to immigrate to this country,” while only half as many Canadians agreed. Minivans outsold SUVs by two to one in Canada; the ratio was reversed in the U.S.
“Canada is becoming the home of a unique postmodern, postmaterial multiculturalism,” wrote Adams, “generating hardy strains of new hybrids that will enrich this country and many others in the world.”
Fire and Ice won rave reviews and prizes; and in 2005, the Literary Review of Canada named it one of the top 100 Canadian books of all time. In The Unfinished Canadian, Cohen agrees that Fire and Ice “cast a light on a corner of our national character,” but not in quite the way its enthusiasts believed. It revealed, he says, “our ambivalent and tortured relationship with the Americans, our struggle to understand them, our moral superiority in dealing with them.”
Cohen cites what he calls a “devastating critique” of the book’s statistical methodology by Joel Smith, a professor of sociology at Duke University, in The American Review of Canadian Studies. “At best, this is an op-ed piece spun into a book,” wrote Smith. “Despite its pseudo-scientific trappings, the basic message is only Adams’ personal views on where the two countries are heading.” As for respective sales in the minivan vs. SUV as proof of a deep-seated Green awareness, Cohen cited David Frum’s argument that the minivans probably sold better in Canada because they were cheaper and Canadians had less money. (Frum, who one tends to think of as part of the inside-the-Beltway conservative punditocracy, is himself Canadian.)
Cohen’s argument seems to be that his fellow Canadians are too prone to emphasizing that they are profoundly different from the Americans -- while at the same time neglecting their own history, and otherwise remaining very loose about defining what counts as a Canadian citizen. He complains that students can leave school in most provinces without studying more than a very little of the nation’s history. By contrast, the license plates in Quebec bear the words Je me souviens (“I remember”): a nationalistic slogan of disputed origins, though the Acadian Expulsion in 1755 is doubtless at the top of the list of things not to forget.
“Unlike the United States,” writes Cohen, “which encourages and underwrites presidential libraries as repositories of artifacts and papers, Canada has no such practice.” He quotes a member of Parliament who said, “Visiting Washington, D.C., you would find a plaque anywhere George Washington sneezed, but we’re more modest.” Important prime ministers remain largely unstudied, while the winter 2006 catalog of the University of British Columbia Press lists titles such as Nutrition Policy in Canada, 1870-1939 and The Culture of Flushing: A Social and Legal History of Sewage.
As if monographic torpor did not threaten the nation enough, there is also the policy on dual citizenship. Cohen thinks it has gone from generous to latitudinarian. Late last year, when Stephen Dion became the leader of the Liberal Party -- hence potentially the country’s prime minister -- it came out that he also held French citizenship. “For a day or so,” writes Cohen, “his dual citizenship unleashed a frenzy of teeth-gnashing and forelock tugging in Parliament. Then it went away.”
It’s the culture war, then, under the maple leaf flag. What worries Cohen are “the elements of our character: the failure of memory, the weakness of citizenship, the tolerance of ethnic nationalism, the willingness to compromise one too many times.”
Some of Cohen’s complaints -- about multiculturalist excess, and disrespect for the national founding fathers, for example -- sound to an American ear like rallying cries of the right. But then he calls for taxes on the wealthy to be raised, and for them otherwise to be shamed into generosity. (“We criticize many things in Canada,” he writes, “but we rarely give our reluctant rich a hard time. We should.”) He’s also happy that Canada remains independent enough to be able to tell the U.S. that a war it is contemplating is a terrible idea.
For an American, reading about the national identity crisis there is a little like visiting Canada: A lot sounds familiar, though the accent occasionally falls on an unexpected syllable. But I'm struck in particular by a difference from the cultural polemics down here -- the lack of our usual, almost apocalyptic stridency, which probably echoes the Puritan sermons from generations past.
"Once again," writes Cohen about the future of his country, "Canada will have to find a way to muddle through, whatever its endless existential questions. Every generation has had to face down the forces of disintegration and every one has." This sounds like a tone bred by facing one terrible winter after another for two or three centuries. It is neither wildly optimistic nor bitterly pessimistic; it just chops wood and waits. I can't help wishing we could pick that tone up and make it American.