Your Interview With AI

Miguel Santiago, a senior at Baruch College in Manhattan, is graduating soon and already considering his next move -- maybe to a job at Goldman Sachs or somewhere else in banking.

In at least six of his interviews, he's been questioned by a computer and not a live person.

“They’ve basically replaced the first round with the HireVue,” he said, referring to the video and artificial intelligence platform increasingly being used by employers for job interviews.

When a candidate applies to a job at a company that uses HireVue, they are asked to go on to the platform, allow use of their webcam and respond to interview questions on video. The candidate’s answers are recorded and then saved to the platform.

Santiago helped bring Goldman and Bank of America employees to his campus to discuss the HireVue process when he was events director for Baruch’s chapter of the Association of Latino Professionals for America.

“What they really want the students to know is that the camera that they’re speaking to when they’re answering the questions from HireVue, it’s just another person,” he said. The banking employees told him that a human would be reviewing the footage, he said.

But according to HireVue’s own advertising materials and recent reporting by The Washington Post, that’s not the only way employers use the platform. Employers can choose to have the recorded answers evaluated by artificial intelligence. If the proprietary technology that HireVue uses to evaluate the recordings concludes that a candidate does well in matching the demeanor, enthusiasm, facial expressions or word choice of current employees of the company, it recommends the candidate for the next round. If the candidate is judged by the software to be out of step, that candidate is not likely to move on.

Although colleges are ostensibly preparing students to enter the workforce, many institutions appear to be unfamiliar with or unprepared for this latest trend in the job market. Unilever, Atlanta Public Schools, Hilton Hotels and Resorts, and nearly 100 other employers now use HireVue, but little advice concerning HireVue interviews can be found on university websites.

To many people who study workforce and AI issues, the software is something of a black box. What the artificial intelligence is looking for -- a smile, a loud speaking voice or a particular keyword -- can’t really be ascertained. Helping prepare students for AI-assessed interviews is challenging, as a result.

“No one is really sure what they look for when it comes to who moves on and who doesn’t,” Santiago said. “It’s definitely kind of a weird experience to have.”

Alex Engler, a former data scientist and a current fellow at the Brookings Institution, where he focuses on AI, is doubtful about the AI's qualification assessment capabilities.

Engler contrasted HireVue's evaluation with other AI-based hiring software such as résumé scanners, which he said are concerning but are based on a plausible connection between résumé qualifications and job readiness.

“HireVue is doing something that goes past that, which is looking at how candidates act in interviews. Their gestures and pose, if they’re leaning with their arms on the table, their tone and cadence,” he said. “Inferring personality traits from a 25-minute video interview, I think, is probably incredibly difficult or impossible. And to tie it to outcomes like, can you be an investment banker or an accountant, is to me very far-fetched.”

HireVue says its assessments are based on "100 percent validated science."

"We follow leading psychological research showing the behaviors, skills, traits, thinking styles, and competencies that predict success at work," a HireVue spokesperson said via email.

Supporters of such AI-based hiring methods say traditional in-person interviews are even more unfair, and that human evaluators might judge job candidates on arbitrary or illegal criteria such as their physical attributes, dress style or ethnicity.

“People are rejected all the time based on how they look, their shoes, how they tucked in their shirts and how ‘hot’ they are,” Loren Larsen, HireVue’s chief technology officer, told The Washington Post. “Algorithms eliminate most of that in a way that hasn’t been possible before.”

Ifeoma Ajunwa, a professor of labor and employment law at Cornell University who has previously written about AI-based hiring, says that line of thinking is misguided.

“We have to remember that automated hiring platforms are still created by humans,” she said. “The same biases that humans have would also be transferred to any platforms they create.”

HireVue works by having current employees answer the questions on video and then evaluating the candidates on how well they match those employees.

Annelies Goger, a fellow at the Brookings Institution who focuses on workforce development, said the use of the platform could create anxiety for job seekers trying to access industries that are historically segregated and exclusive.

For example, she said, in an occupation primarily filled by young, white men, the system is likely trained to recognize success as young, white men exhibit it.

“If I’m a woman of color and trying to get in,” she said, “I would be pretty anxious about, is this system really going to assess me fairly?”

The HireVue spokesperson said the company's AI assessments are actually increasing diversity at companies, because the algorithms don't notice appearance.

"Each algorithm or assessment model is trained not to 'notice' age, gender, ethnicity, and other personal characteristics that are irrelevant to job success, so it helps to level the playing field," the spokesperson wrote in the email.

But to critics, the things that the AI does notice -- faces and gestures -- are the root of the problem, not appearance. Motions like these can vary widely based on culture and ability status, they say.

"You have to exhibit similar qualities to the people already in those jobs and performing well," said Engler. "That doesn’t actually have to mean qualities that are actually useful for the job -- you just have to share the same qualities."

The effects for people with disabilities could be drastic, he said.

“The way that disabilities can affect people is very broad, and as a result some of the characteristics that people with disabilities exhibit are unlikely to exist in the AI’s training data.”

"HireVue offers various accommodations for people with disabilities and is actively working with international disability groups, as well as with Integrate Autism Employment Advisors to ensure that the tools and processes are fair and accessible," the company spokesperson wrote.

Goger said no established "legal infrastructure" exists to protect a person facing discrimination from AI.

“Who’s going to check for ADA compliance in these systems?” she said.

Trying to Prepare

Some colleges are trying to prepare their students with the little knowledge that career counselors do have about how HireVue's AI evaluation works. Oftentimes that advice doesn’t look very different from the guidance they provide for in-person interviews.

Michael Kalish, associate director of on-campus recruiting at Baruch’s career center, says career counselors commonly suggest that students dress in a full suit and use industry-specific lingo, since it has been suggested that HireVue scans a candidate's answers for keywords.

“We instruct students in general that when they’re interviewing they should always be using that kind of terminology regardless,” Kalish said. “Not even just to be prepared for [HireVue] interviews, but just, in general, to show the interviewer that they are prepared, that they’ve done their homework, that they’re generally interested in that field.”

Students can also practice for interviews using a mock interview platform called Symplicity that asks industry-specific interview questions and records their answers via webcam. This software predates the widespread use of virtual interviews, Kalish said. Unlike with HireVue, students can rewatch their answers on the Symplicity software.

“They can actually watch it and learn from their mistakes,” Kalish said. “They can meet with a counselor. We can watch it with them and give them constructive criticism and feedback on areas that they should improve upon.”

At Duke University, a document from the economics department lists typical HireVue questions (“Tell me about a time you worked on a team?” and “What does integrity mean to you?”) as well as a few tips for students. The suggestions range from the general interview advice (“Try to give structured concise responses”) to the more technical (ridding the screen of your own image makes it easier to look into the camera), but they don’t really touch on how to nail the mannerisms of past employees other than to “act natural.”

Brigham Young University Idaho is one of the few colleges that advertise mock interviews specifically for HireVue on the university's website. Students can schedule a HireVue mock interview and meet with a career mentor to hear feedback, according to the website. BYU Idaho career center staff declined to discuss these resources, which they said are still in their early stages.

The University of Colorado at Boulder also uses HireVue, but not to help prepare students for interviews. The college uses the software for nearly all of its own hiring.

Andrew Horovitz, an assistant director of talent acquisition at Boulder, said that HireVue makes the process more efficient and lends flexibility to both hiring managers and candidates who don't have to travel or take off work for the video interview. The college does not use HireVue's AI assessment tool, only the video interviewing, he said. The AI feature is an add-on that the university would have to pay for.

"I don’t foresee us using that any time in the near future," he said of the AI tool. "We do a lot of work on our campus around mitigating bias in the interview and hiring process as it is already, and so we want to make sure any tool we introduce or functionality we introduce is in line with that.”

Yajin Wang, a professor of marketing at the University of Maryland, says that most of the things students can do to prepare for HireVue job interviews will make them better public speakers.

The University of Maryland’s business school posted an article in June of this year featuring Wang giving advice to students on how to do their best on HireVue and other platforms.

“Ace an AI interview by incorporating keywords and phrases that explain what you can contribute, echoing the exact language of the job posting,” the article said. “Use gestures, smile, and nod frequently.”

Wang suggested candidates record themselves and watch their answers to prepare.

HireVue advised that students visit the company candidate center and focus on giving examples from relevant work.

"We encourage candidates to 'show what they know,' give specific examples of successful projects when asked, and 'be themselves,'" the company spokesperson said in the email.

The HireVue Effect

Frederick Hess, director of education policy studies at the American Enterprise Institute, said while he is “not impressed” by HireVue's platform, which he called “dystopian” and “pseudoscience,” the overall trend toward assessment-based hiring may undermine the economic value of a college degree. He said if companies begin to hire based on skills tests that are, unlike HireVue, non-prejudicial and legally sound, they will stop using college degrees as a proxy for knowledge and employability.

“I think these efforts to build new tools, hiring platforms, hiring systems, which will stand up to legal scrutiny because they are specific and clearly attached to the job you’re going to do, and can be defended that they are non-prejudicial, that stuff should worry the heck out of colleges,” he said. “If you can apply without having to go through all the stuff for the degree, then employers can pay less, and you still feel like you’re getting enough.”

“In terms of value as a proxy for skill and talent, I think college degrees themselves are limited, and they’re one instrument that is probably overused in the labor market,” said Goger, the Brookings fellow. But HireVue “is replacing one flawed instrument with another.”

She said colleges could focus on providing students more opportunities for face-to-face interaction and getting a foot in the door.

“They can help candidates understand how to communicate their value to an employer in a specific role,” she said.

Ajunwa, the Cornell professor, said job candidates being assessed by AI should use keywords from the job description, avoid hinting at gender or ethnicity on their résumé, and explain employment gaps by saying what they were specifically doing, such as having a child.

“I worry that these systems might be sending a wrong message to students,” she said. “They’re so anxious and focused on cracking the résumé or cracking the CV to beat the system and then feeling perhaps that their work experience or their skills that they acquired in school aren’t that important. College should be a time to explore and should be a time to learn various skills … I want students to really focus on that, more than worrying about beating these automated hiring systems.”