“What Do You Think?”

As an instructor, I was never entirely sold on the multiple-choice format for tests. I would sometimes have a small section of it on an exam to test really basic factual knowledge -- how many senators does each state get -- as a sort of reality check, but it wouldn’t count for much. Life rarely comes at you in multiple-choice format, so it seemed kind of silly.

I’m beginning to appreciate the wisdom of the format, though.

This week I was in a meeting at which the discussion turned to ways the college website could be improved. There’s an effort under way to make it more useful and effective, and we were discussing a few items on our personal wish lists. (To be clear, the meeting was not of the group in charge of the site.) As the discussion turned to ways to solicit input, someone suggested putting out a “what do you think?” question far and wide.

In my experience, questions as broad and open-ended as those tend to turn off most people, ceding the field either to the local experts or to a few true believers. It’s just such a wide-ranging question that unless you’re really focused on it, it’s hard to know where to start.

Most of us who have taught classes have had the experience of throwing an open-ended question out there and seeing it land with a thud. Sometimes it reflects apathy, but sometimes it reflects a sense of being overwhelmed. If I were asked, abruptly and point-blank, “Quick, how do you think we could achieve peace in the Middle East?” I probably wouldn’t do much better than a shrug. There’s so much in there, so much at stake, that a knee-jerk answer probably would do more harm than good.

But a multiple-choice question is far less overwhelming. Instead of asking, “How should we improve the website?” this might take the form of showing and demonstrating, say, three alternatives, and then asking which one is the most appealing.

The multiple-choice approach is both more and less democratic. It’s more democratic to the extent that it enables broader participation; it’s less democratic to the extent that elites are the ones delineating the options (and thereby choosing what to exclude). Democratic theorists have lobbed tomes at each other on this question. Walter Lippmann and John Dewey wrestled over this a century ago, with no clear winner. There’s no shortage of “voters are ignorant” studies, showing that, say, more of them can name three Kardashians than three Supreme Court justices. Now there’s an “epistemic turn” in democratic theory in which some argue that individual ignorance is canceled out in large groups, like in “guess how many jelly beans are in the jar” contests. The guesses tend to range widely, but they tend to cluster around the right answer. So it goes with voting, argue these theorists.

In a campus context, in which the numbers are relatively small and people are distracted with other things, I’m thinking that the multiple-choice approach has merit. Yes, it relies on leaders to structure the options, but in the case of something like a website, some level of technical expertise is necessary to make a viable decision. Having people with that technical knowledge help delineate the choices can prevent people wasting their opportunity for input by wishing for things that are beyond the capacity of the college to provide. Restricting the options to things that could actually happen may seem, well, restrictive -- and it is -- but it also increases the likelihood of the group input (or vote) leading to a viable outcome.

In politics, we use parties for that. Parties narrow down the options, and voters choose from among them. Parties are flawed instruments, heaven knows, but they serve a purpose. They move elections from essay questions to multiple choice.

With websites, of course, running two or three or four samples simultaneously may be a bit of a heavy lift. But the idea seems right.

Wise and worldly readers, is there a way to make an open-ended question like “What do you think we should do with the website?” effective across hundreds of people?