A Closer Look at Our Facial Emotional Recognition Tool

At Will & Agency/PUBLIKUM, we are constantly exploring how new technologies can strengthen the way we understand audiences. Not just what people say, but what they actually feel in the moment.

Over the past few months, our tech team has been developing a new AI-powered Facial Emotional Recognition Tool designed to bring us closer to the emotional layer of human behaviour in qualitative research.

From raw video to structured emotional insights

The tool works by processing video material collected through our mobile ethnographic platform. Once respondents upload their videos, the system automatically retrieves and processes them into a structured dataset. From there, each video is analysed frame by frame, typically at four frames per second, allowing the system to detect subtle changes in expression, tone, and language over time.

Understanding emotion across multiple layers

What makes the tool particularly interesting is its ability to analyse emotional signals across different dimensions at the same time. Emotion is not treated as a single signal, but as something that emerges across:

• Text-based emotion detection (28 emotion types)

• Facial expressions (10 emotion types)

• Word choice and language patterns (7 emotion types)

By combining these inputs, the tool creates a more layered understanding of what is being expressed and how it is being expressed.

From themes to meaning

Beyond emotional detection, the tool also identifies recurring themes and keywords across the material. These are then summarized and ranked based on frequency, helping us highlight what stands out across respondents — both in content and emotional tone.

We are excited to continue developing it, testing its potential, and exploring how it can deepen the way we work with audience insights.