By cobbling together an Oculus Rift and three Kinects, Oliver Kreylos has built something stunning. When you stand in Kreylos’ “mixed-reality theatre” the three Kinects work to create a a “3D video” of you in virtual space. The virtual reality headset lets you look into this virtual space as though it were real. Raise your hand in real life and you will see it through the headset.
Because the environment is virtual, Kreylos can load in new sets and objects easily. Someone else with a similar hardware set up could load themselves into the same virtual space, even if they were on the other side of the globe.
You’ll need to watch the video to understand quite how incredible this is.
What we’re seeing isn’t a 3D model but a 3D video of Kreylos walking about the virtual office. “One of the things we’ve noticed since we started working with 3D video to create “holographic” avatars many years ago was that, even with low-res and low-quality 3D video, the resulting avatars just feel real, in some sense even more real than higher-quality motion-captured avatars,” explains Kreylos. “I believe it’s related to the uncanny valley principle, in that fuzzy 3D video that moves in a very lifelike fashion is more believable to the brain than high-quality avatars that don’t quite move right.”
And the video is fuzzy. The three Kinects Kreylos has set up in a triangle around his office are generation one Kinects, those released for the old Xbox. The second generation Kinects have a much higher resolution camera and sensor, incorporating those into the mixed-reality theatre would improve the fidelity of the 3D video.
Something Kreylos has discovered is that, because he is using the Kinect in “raw” mode, using direct input data from the device, not running it through an Xbox, “latency of the 3D video is either not noticeable, or not a problem. Even when waving my hands directly in front of my face, they feel completely like my hands, and whatever latency is there does not lead to a disconnect.”
Kreylos has hit on a technique that, while producing a rough-looking result, instills a sense of presence. Something Valve see as the holy grail of VR.