Analyzing Customer Behavior from Video

A lot of customer service functions involve complex interactions between employees and customers. Studying and analyzing these interactions is complicated and time consuming. In the retail world, however, there is often a steady source of high quality video of those operations … surveillance videos.

Dartfish can help to take surveillance video and extract operational insights. The following video clip (borrowed from this YouTube video) illustrates how one of Dartfish’s tools could help retailers to help understand complex customer behaviors.

In this situation, a customer has entered a hobby shop and is uncertain about what to buy. The blue line follows the customer as he journeys around the store trying to find answers. The yellow line follows the lead server (presumably the one with the answers) as he tries to find a solution. Note how the intensity of the line color more or less tracks the movements of each subject.

The single result, by itself, has modest value. It does illustrate a few observations about the store layout and server actions that might be worth studying:

  • The perimeter counter layout forces the server to travel long distances around the store. The customer also travels a fair bit, but some of that is due to indecision. The server moves because the layout forces him to take the long route.
  • The server engaged the customer over each individual product.  We don’t know why the customer was interested (or the server thought he was interested) in items all over the store. If it related to a specific hobby topic, it is strange that the relevant parts are scattered all over. One wonders if a different item layout might have been more efficient … concentrating interest and discussion in one place.

Of course, with one example and no audio, these are mere speculation. Consider, however, what might happen if we took a few hours or a day of surveillance footage and isolated a random sample of customer visits. it might look something like the following diagram.


Then consider that you decided to organize and plan a systematic investigation of in-store behavior. You could define some categories and criteria and design a custom tagging panel to record when suitable situations occurred in a video. Dartfish allows you to play through a large video file at up to 16x normal play speed. Then do the following:

  • Watch for situations that fit your criteria.
  • Mark them as continuous duration events, with suitable keyword descriptors

With a little practice, an observer should be able to review and tag an 8 hour video in an hour or so. When suitable lengths of video have been tagged and cataloged, you can select segments according to keywords and publish playlists of the relevant subclips. Consider what you might learn if you could quickly watch:

  • 10 or 15 instances where a (possibly younger) shopper came into the hobby shop, looking for a suitable way to start building models.
  • 10 or 15 instances where a shopper came into the hobby shop trying to find something that was a gift for a friend.
  • 10 or 15 instances where an experienced hobbyist came into the shop looking for the pieces that would allow them to build an imagined device.

Would they navigate the store differently? Would they ask similar questions (assuming the surveillance camera has a microphone)?  Would they tend to approach the same staff member? Would they spend time on their own or asking questions? There are so many things that you might be able to investigate with surveillance footage and Dartfish.