Traditionally, brands have used panel discussions or self-reporting to determine how their marketing influences consumer preferences, action and behaviour. It’s not a bad approach, but it’s difficult to scale or identify which ads will work and which won’t. And marketing doesn’t come cheap.
Experts at the Mars Marketing Laboratory and software company Realeyes have taken a different approach. The team used webcams and the latest computer vision and machine-learning technologies to analyse facial expressions to determine “how people felt” whilst they watched adverts for brands including Dolmio, Galaxy, Uncle Ben’s and Bounty.
I spy … with 75% accuracy
The team first measured the micro-movements in the faces of willing consumers and used computer vision and machine learning to analyse them. They focused on expressions of happiness, surprise, confusion, disgust, engagement, as well as behaviours such as how and when people move their head. These emotions data were then cross-referenced with Mars’ known sales lift data for each ad.
Their analysis – involving 149 ads across 35 brands and 22,334 people in six countries – created the “largest emotional dataset linked to real business outcomes currently in existence”.
They discovered that “emotions data could be used to correctly identify whether the ads tested had a ‘no to low’ or ‘high’ impact on sales 75% of the time”. The accuracy was “nearly identical” regardless of the product category, they noted in their paper due to be published in the journal Image and Vision Computing. For chewing gum ads it was 76%, whilst in food and confectionery the associations were 75% and 70% accurate.
Splitting results by country there was more variation, however. The best results were in France (87%), Russia (82%) and the US (80%). UK associations were 79% accurate, but in Germany they were only 64%. The researchers are not sure why there was such a discrepancy.
The full study can be read here.
Adspend saver
The team also discovered that successful ads “grow happiness” for longer than unsuccessful ones, whilst the absolute level of happiness matters less than the dynamics of it developing over time. For instance, an advert that steadily increases happiness up to a level of seven out of 10 is likely to do better than one in which levels hit eight at some point but fluctuate more.
The findings are good news for brands. Analysing adverts in this way could ensure marketing budgets are spent more effectively, said Realeyes CEO Mihkel Jäätma. “Being able to identify strong creative with high sales impact enables advertisers to push these ads, and avoid putting media spend behind those with low, or worse – no sales impact,” he explained.
The “promising results” may lead to a new generation of automated, cost-efficient, behavioural cue driven market research tools for analysis, the authors explained.
“Just think – an algorithm can detect how people feel about an advert by tracking their facial expressions, and that can tell us whether that ad will sell or not,” explained Realeyes CEO Mihkel Jäätma. “That’s exactly what our scientists been working to achieve.”