This project with LEAP was a blast. It came about before the official SDK for the Kinect was released so we had to get pretty creative.

The graphics were all done using the XNA framework along with the Farseer physics engine for all of the bouncing objects and falling particles that users interacted with (Writing particle modifiers for the Mercury Particle Engine is just plain fun!).

For the Kinect, we used a special build of the Primesense drivers along with the .net wrappers for OpenNI and EmguCV for OpenCV. We were located in an area that got direct sunlight for several hours a day, which rendered the Kinect useless during that period. To get around the issue we used a high resolution Microsoft LifeCam with auto focus and impressive brightness control.

For those times that the Kinect had trouble getting data, we had the camera pointed directly at the ground and used an image matching algorithm to determine if people were standing in front of quiz options. For the more interactive screen, we fell back to a passive mode with timed events rather than 1:1 interactions.

In the couple of months that the display was up we got over 30,000 interactions (people stopping to look not just walking by) and close to 1,000 people stopped for around 4 minutes to take an entire quiz! "Digital out-of-home", as it's called in the industry, is an incredibly effective way to engage users.

From the tech side, I'm not super thrilled with this video (not just because I only make a 4 second cameo either). Most of the video was done very early on, and not enough of the fun interactions made it into it.