Facial Analysis During Job Interviews

Artificial intelligence (AI) seems to be everywhere these days. This technology is the reason we have so many cool modern technologies such as banking apps, self-driving vehicles, Uber, and Netflix.

One specific application of AI, facial analysis, is even being used by some companies during job interviews. The practice of using facial analysis software during job interviews is controversial. And like any controversial topic, it has supporters and detractors.

What is Facial Analysis Software?

The facial analysis software being used in most corporate interview rooms was developed by Utah-based company HireVue. Their AI technology scans a candidate’s facial expressions and analyses their word choice and tone of voice while on camera.

For example, the technology may scan for how long a candidate can maintain steady eye contact, whether they furrow their brow, and how often they smile. Do they widen or squint their eyes when speaking? Do they tilt their chin up or down?

On the language analysis side, the AI will listen to a candidate’s rate of speech and preference for using passive or active language. It may be listening for how often an “uh” or “um” pops into the candidate’s responses or how long it takes a candidate to start answering after the question is asked.

Of course, those are just a few small examples of the type of information facial analysis software is scanning for in job interviews. HireVue has a database of more than 25,000 snippets of facial movements and 350 bits of language information that was gathered from thousands of successful interviews.

This huge swath of information is what was used to “teach” the company’s algorithm a very specific lesson. That lesson is: “This is what a successful candidate looks and sounds like.” The algorithm can then predict which candidates would be good hires for the role. The final decision gets passed to a human hiring manager, of course, but the technology whittles down the screening process considerably.

What are the Benefits of Facial Analysis?

HireVue, obviously, touts the benefits of this technology. They claim it can speed up the overall hiring process by 90% in some cases. That means getting workers into roles faster, which obviously saves companies the time and money it would take to screen and interview candidates the old-fashioned way.

In fact, more than 700 companies have used HireVue including big names like Unilever, Urban Outfitters, and Hilton. And the results are impressive. Each company reports hiring highly qualified candidates in less time than they would if they had not used the software.  

But there’s another side to the story.

Are There Risks of Facial Analysis?

Not surprisingly, this technology has its fair share of critics because of the risks involved with bringing facial analysis into job interviews.

UCLA professor and noted author Safya Noble, makes the argument that algorithms are not created in a vacuum. They are created, whether intentionally or not, with the inherent biases, prejudices, and values of those who created it.

In her book. Algorithms of Oppression, Noble points out that when you conduct an image search for the term “successful manager” in Google, it returns images of middle-aged, white men. Search the term “housekeeping” and the results are predominantly women.

In short, that means, the AI that powers Google has learned that successful managers are white men and housekeepers are women. It’s not only learned that stereotype but will continue to reinforce it with each search.

Facial analysis in job interviews has another big failing, beyond enforcing outdated stereotypes. It has to do with the fact that our cultural and economic backgrounds influence how we speak and act from a very early age.

As noted by famous sociologist Erving Goffman, those with less privileged upbringings may be more timid, quiet, and reserved, while others who grew up with more resources and opportunity may have more confidence, poise, and self-assurance. The more privileged candidates tend to be more successful during interviews whether they have the best skills for the job.

Of course, this was all true in job interviews before facial analysis came along, but AI only reinforces this problem. In fact, if the HireVue technology works by relying on the facial movements and speech patterns of successful candidates in the past, companies are more likely to hire the same types of employees they’ve always hired.

Hiring a pool of employees that are too similar to one another could lead to decreased risk- taking and less diverse points of view, which can, in turn, stifle a company’s growth and innovation.

Finally, reliance on facial analysis also takes away the “gut feeling” factor. Sometimes a manager just gets a feeling that they should take a chance on a candidate.

With AI in place, that candidate may not get a chance to speak to the hiring manager face to face because they didn’t raise their eyebrows at the right time or speak in the right way. AI could actually result in companies missing out on talent. That’s the exact opposite of what the developers of the technology intend it to do.

Surprise! Here’s the Plot Twist

Every good story has a plot twist.

It turns out, as of January 2021, HireVue has discontinued the use of the facial analysis component of its software. An internal study conducted in March 2020 by the company concluded in part, that “visual analysis no longer significantly added value to the assessments.”

HireVue CEO and Chairman Kevin Parker made this statement:

“Over time the visual components contributed less to the assessment to the point where there was so much public concern to everything related to AI that it wasn’t worth the concern it was causing people”

Those who have been fighting against facial analysis in job interviews are hailing this as a huge win. Although, it should be noted that HireVue is still using language and speech analysis in their software, which is in and of itself, controversial.

Empire Resume Will Help You Get Hired!

• We’re an industry leading certified professional resume writing company that offers LinkedIn profiles, cover/follow up letters, and detailed job search guidance.
• We have a phenomenal success rate landing our clients interviews.
• We offer a 100% satisfaction guarantee.
• Click here to learn more and get started today!

Maria Gold is a Content Manager/Writer for Empire Resume. She is dedicated to helping educate and motivate people with the latest career articles and job search advice. Her interests range from writing to programming and design. She is also passionate about innovation, entrepreneurship, and technology.

Related Articles: