The Customer Is Always Right

ARLINGTON, VA. – The Female Offender: Girls, Women, and Crime is an unusual library request for Mandy Watts, a third-year physical therapy doctoral student at Marymount University. She has limited interest in either criminology or women’s studies, but that’s what brought her to the library’s help desk on November 9.

After a pleasant conversation with the librarian and some confusion about the book title and whether it was in the library’s collection, the librarian located Watts’s book in the database and directed Watts downstairs into the stacks with a call number in hand.

Watts noted that, in an ideal world, the librarian would come down into the stacks and help her find the book. But she said that’s probably not what an average student would expect. Had the directions proven confusing or the book not been where the librarian said, then she might have returned to the desk.

But, after only a minute or two of searching, she found the book right where the librarian said it would be. After glancing at the book, she put it back on the shelf, and walked out. “That went pretty much exactly like it was supposed to,” Watts said, exiting the building. “There won’t be much to report about that.”

Watts, who usually spends her days on the university’s campus in Ballston, is here on the Marymount’s main campus on a mission that showed up in her e-mail inbox a few days earlier. Watts is part of a group of students who, at the behest of Marymount’s president, Matthew D. Shank, surreptitiously tested various campus offices to evaluate service quality.

“Mystery shopping," while common in retail, hospitality and entertainment fields, is not a tool regularly employed by higher education administrators. Shank hopes to change that.

“Inside the classroom, students are students, but outside it they’re customers,” Shank said, explaining the motivation behind the mystery shopper program. “Marymount is engaged in a broader initiative to improve service quality on campus, not only for students, but for all internal customers. We’re asking everyone, unit to unit, as employees, ‘How do we improve services to have a better impact on students?’ ”

Shank and Marymount’s efforts highlight an often-overlooked aspect of university administration that can have a profound effect on the student experience – the myriad interactions students have with university officials outside the classroom. Shank said such interactions, while not the focus of a student’s time at the university, can shade his or her view of the experience, thereby making him or her less likely to recommend the institution to others or preventing him or her from engaging with a particular campus office. In the case of something like the library or career service, it could have a significant effect on that student’s educational or professional outcome.

The Actual Experience

The mystery shopper idea is the brainchild of Shank, whose own background was in marketing before he entered academe and university administration. “We’re so familiar with the idea of mystery shoppers in retail settings,” Shank said. “Why not bring it in to the university environment? It could help you get a sense of what the culture of the university really is.”

Shank said he pushed the idea when he worked at other colleges, but none adopted the idea. He joked that now that he’s in charge, he can finally follow through on it.

In all seriousness, Shank said, Marymount is engaged in a broader initiative focused on service quality, and he viewed it as a perfect opportunity to try out the program. He let this reporter observe some of the secret shopping in action, on the condition that this article not appear until the university completed the initiative.

Last year the institution surveyed students about their experiences with the various service offices. The response was generally positive, but administrations said there was something lacking about the feedback. “We were able to get a lot of data back, but it lacked qualitative things,” said Michael Schuchert, executive director of institutional effectiveness, who is overseeing the mystery shopping initiative. “It really didn’t capture what the actual experience was.”

Since the survey tended to return responses that were either very positive or very negative, Schuchert and Shank saw the mystery shopping initiative as a way to capture the experience for the average user.

Actually organizing the initiative took several months. Once they realized they wanted to launch the mystery shopper initiative, Schuchert asked Chris Macomber, an undergraduate involved in student government, to recruit a diverse group of students to serve as mystery shoppers, help develop the case scenarios and manage the initiative.

The university decided to use real students rather than trained mystery shoppers because many offices require student identification numbers before they can provide any service. The students were trained in how to go about the mystery shopping and what to look for in the process.

Shank said there is robust literature on service quality. He said there are broad objectives that have been refined through research and that some specialized types of service, such as libraries, have their own expectations.

Administrators selected six offices to test based on the results of the survey: the registrar, student accounts, financial aid, the IT help desk, library circulation and career services. The heads of all the departments were brought in to consult on what types of student requests they see most regularly.

Andy Brantley, president of the College and University Professional Association for Human Resources, cautioned against evaluating the performance of an institution based on the opinions of a handful of students. “One student’s definition of outstanding customer service might be different than another’s,” he said. He said any approach should be rooted in research, and that evaluators should be trained in what to look for to ensure that the evaluations are not arbitrary.

David Wilmes, executive director of Marymount’s career services, was one of the administrators eager to be a part of the program. He said he was interested in finding ways to improve his program to ensure that students find value in working with his office. “I think there’s a certain resistance to thinking of students as customers, but for many services offices, they are customers,” he said. “If we don’t serve them well, they have the option of not using us.”

While campus administrators are aware that the mystery shopper program is going on, Schuchert said, most of the staff members of those offices were unaware. The idea behind the program was not to be punitive for employees who might not be serving students well, but to identify areas where there might be problems and find ways to improve service across the board.

The Average Experience

If capturing the average student experience is what the program was after, that’s just what they’re getting. “Average” is the general experience that Watts and other mystery shoppers said they had during the course of the initiative. Most of their interactions with service offices were relatively routine. They would inquire about a bill, or an internship, or financial aid, and the staff would more than likely have the answer.

Watts’s other task for the day she visited the library was to inquire at the student accounts office about a fee on her bill for health insurance – whether the item listed on the bill covered the semester or the year.

Aside from a hiccup when she first asked to see her bill – the records office employee said paper copies of the bills had been sent out just yesterday – Watts simply asked about the fee and got a response. The entire interaction lasted about a minute. She said that was standard.

The mystery shoppers had 18 total interactions each over the course of the nine-week initiative. Some were in person, like Watts’s two assignments that week. Other times the students inquired by e-mail or phone about a specific issue.

After each assignment the students fill out a questionnaire about their experience, noting things like whether or not the office was able to answer their question, whether the employee was friendly and professionally dressed and whether the employees focused on the student making the request or had divided attention.

Watts said that, in general, her experiences have gone smoothly. She did have one gripe about the university’s services, noting that the library at the Ballston campus was frequently closed when the signs said it should be open, stifling her efforts to print assignments before class.

“Programs like this are the only way we are going to be able to fix things like this,” Watts said.   

What Will Come

The initiative wrapped up last week, so administrators have not yet tabulated comprehensive results. But the shoppers and administrators noted that a few trends have already emerged.

Macomber noted one difficulty relating to student services that he’s observed over the course of the initiative – the individual nature of student problems. Unlike offices such as the library where students’ inquiries tend fall along similar lines and can thus be triaged, students come into offices such as financial aid and the registrar’s office with unique sets of problems. Macomber said it’s hard to make recommendations about how to adjust services in those departments.

Others point out that some offices, such as student judicial services, are often faced with the challenge of giving students bad news rather than providing help, which could result in a negative perception no matter how good the service. “Something we’re really trying to wrestle with is when a good service sometimes has to deliver bad news,” Schuchert said.

Shank and others also wants to use the results to celebrate and learn from those offices where the experience was above average, something Brantley applauded.

Shank said that if this first test of the mystery shopper program works well and provides valuable feedback, he would like to expand and refine it and expand it in future years, incorporating other services and re-evaluating some services tested this time to measure improvement.

He said one area that won't be mystery shopped would be the classroom. "We know that student satisfaction in the classroom can lead to greater outcomes in student learning, and that bad service and bad interaction can influence learning in negative way," he said. But he noted that the relationship inside the classroom has a different set of expectations from what is evaluated through the mystery shopper experience.

And unlike the services that were mystery shopped this fall, the college already has numerous procedures in place to evaluate quality in the classroom.