Translation and Interpreting in 150+ Languages
Four portraits of a woman gesturing with her hands
Decoding Hand Gestures with Kinect
July 3, 2014 - By: - In: In the News / Awards, Language - Comments Off on Decoding Hand Gestures with Kinect

While I previously thought that Kinect was just about badly losing to my sister in a string of Xbox videogames, it turns out that researchers have found a better use for Microsoft’s motion sensing device Kinect. Hao-Chuan Wang and his team of researchers from Taiwan’s National Tsing Hua University are using it to study the role of hand gestures in communication.

His team used Kinect to capture participants’ hand gestures as they talked to each other: in person or face to face in the same location so they could see, hear and be near each other; at a distance over video chat but still with the ability to see and hear each other; as well as with audio only, just allowing them to hear each other’s voices. They used one sensor per person, which produced a record of both speech and hand gestures with time stamps.

You would probably think that people would only gesture if others could see them, but interestingly, the team found that people gesture nearly as much regardless of whether others can see them or not. They also found that gestures were not used to help the other person understand the communicator better, but were used as part of the communicator’s own understanding of a conversation.

Wang and his team concluded that hand gestures and body language are not necessarily used to express and communicate messages to others, but rather as a self-reinforcement of the communicator’s own comprehension and intention.

Since the study depended on the use of Microsoft’s Kinect, I was unsurprised to learn that Wang and his team worked with Microsoft Research Asia on it. It makes me wonder what other kinds of research Microsoft is doing at their Asian headquarters in Beijing. Perhaps how to improve Windows 8?

LiveZilla Live Chat Software