Usage of facial coding to detect emotional responses


You can tell by your face

Using facial coding at the Deloitte Neuroscience Institute

With facial coding the Deloitte Neuroscience Institute can detect subtle emotional responses to content, such as presentations or communications materials.

Based on these results, we help our clients messages and presentations so that the desired effect can be maximized. The rationale of facial coding is that: "The face is the window to the soul", as the renowned US psychologist and emotion researcher Paul Ekman put it.

The Deloitte Neuroscience Institute offers – among other things:

  • Analysis & optimization of sales calls to increase the chance of success
  • Measurement & prediction of emotional response to communication materials and contents
  • Measurability of unconscious emotions regarding products 
  • Design user-friendly software processes

The theory of facial coding is based on 40 years of research and diagnostics in psychology. The method is technically based on a combination of image recognition, machine learning and deep learning algorithms.

A self-learning algorithm for pattern recognition detects key facial characteristics such as the mouth, eyes and eyebrows. Changes in these features are used to classify emotional states. The machine learning algorithm improves the accuracy of the results continuously. Very small changes in facial expressions, so-called micro-expressions, can be detected and interpreted. These are usually not visible with the naked eye and at best, can be detected by experts with years of training. Digital recording, analysis and interpretation of these changes in facial expressions allow us to draw conclusions about the emotional state and its effect on behavior and decisions.

Facial coding thus provides an understanding of whether emotional reactions are positive or negative. At the Deloitte Neuroscience Institute, we complement these results with additional neuroscientific methods, such as EEG, eye tracking, implicit association testing and galvanic skin response measurement. This enables us to visualize both the mental and emotional response of participants.