Facebook have just announced some pretty revelatory virtual reality and augmented reality advances at their F8 developer’s conference. One of the most noticeable advances on the agenda was hand tracking powered by AI, which could spell the end for VR controllers.
Prefer to game at a desk? Here are the best gaming monitors going.
The hand tracking tech was presented by head of core tech product management, Maria Fernandez, and targets complex interactions with objects. The hand tracking is made possible through an AI that learns to recognise hand patterns.
Facebook’s dev boffin’s collect marker based tracking data, which is then used to collect the data required for controller-free tracking. Gloves with motion capture markers record the necessary data, which is subsequently fed through a convolutional neural network called deep marker labeling. The AI hive-mind can then use this data to track fully-articulate human hands without any gloves, markers… anything.
With this new tech, Facebook’s AI can not only recognise human hands, thanks to FB’s neural network crash course, but track natural hand movement, too. From here it opens the door to complex object interactions. Props will no longer need to be ‘smart’ and packed with sensors, they only need to be shape-mimicking objects that the user can interact with naturally for a far more instinctive gaming experience. Essentially, foregoing the need for products such as HTC’s Vive sensor entirely.
Facebook believe this new method will increase success rates drastically, not only for single-hand tracking, but also two-hand tracking, and hand-object interactions. I’m a little pessimistic of the numbers at this point, as FB believe this technology will be 100% successful when it comes to single-hand tracking, 99.29% successful in two-hand tracking, and 98.26% successful in hand-object tracking.
For VR games, that could change everything – literally. While VR controllers have been fairly successful, they do drastically reduce the immersion factor while gaming in a virtual space – especially when patting around on the floor for a controller that’s fallen through the floor texture.
We’ve had quite a few new VR headsets launch recently, from HTC’s Vive Pro to the standalone Oculus Go that launched only days ago. However, none have truly changed how we approach, and interact with, VR – at least not yet anyways. With FB showing off their latest virtual tech, truly game-changing innovation might not be too far off across the entire industry.