At MIPT, a Computer Was Taught to Understand Gestures
It is possible that in the near future the computer mouse will become history. This moment is being progressively brought closer by inventors around the world. Among them, is a post-graduate MIPT student Alexei Kurakin - he developed a new method of data entering, using a web-camera and his own hands.
A web-camera, a tripod and a laptop – that is all you need for an engineering marvel. MIPT post-graduate student Alexei Kurakin almost never uses a computer mouse. Instead - his own hands. A few simple gestures, and the picture on the screen changes.
"I need to move an object in three-dimensional space. With my hand, I can do it in six directions, plus I can turn my hand. With a mouse, I can track it in only two dimensions," - shows the inventor.
On the basis of a palm silhouette an algorithm builds a hand model , and, based on the number of visible fingers, determines what to do with the picture. The image can be moved, rotated and stretched. And if you add a second web-camera, with your bare hands you can work with three-dimensional objects.
To break into the third dimension, the researchers simulated binocular vision. Two cameras, located at a short distance from one other, fix the movement of control points, i.e. the palm and the fingertips. The program then combines the images from the cameras and determines which command to execute. For example, turn off the lights.
Three-dimensional manipulator is useful not only designers. It will solve other problems, for example, it will help to recognize sign language of the deaf. Twins Alexei and Nikolai Ivanov lost their hearing in early childhood. Nevertheless, both play the keyboard and drums, and even sing. They do doubt the fact that a machine will learn to understand the world's most emotional language - sign language. Their mentor Valentina Petrovna Kamneva, who heads the laboratory of sign language, does not agree with the brothers. She believes the new invention can bring many benefits.
"My dream is to have children's cartoon characters talking in sign language,” says Valentina Kamneva. “You cannot even imagine how much deaf children need that."
Experts have realized the fact that the good old mouse is gradually receding into the past a few years ago. A boom that accompanied the emergence of tablet computers indirectly confirms that users want to communicate with the computer in the language of touch. Many research centers are working on data entry systems that use hands. The original feature of the Moscow institute development is that no additional devices other than the PC and the camera are required.
"I am working to give up the use any of gloves and just use the image of the hand or the two images from two cameras to determine its configuration" - says Kurakin.
While the prototype currently recognizes only five gestures, in the future that number will be increased.