Microsoft’s research team have been working on a way to combine projectors and Kinects to turn your living room into a holodeck. RoomAlive may only be a prototype but its the closest we’re going to be to Star Trek’s tech for a good while.
Once you’ve rigged up the frame of projectors and cameras RoomAlive can repaint your walls with virtual, dynamic textures - turning your floor into a river, for instance; spawn enemy moles that you have to whack back into the walls; and you can even sit down on your sofa and guide a character around your room, shooting invading robots.
It’s all very much proof of concept but it’s a hint of some of the game experiences we might have in the future.
What’s most impressive is the low barrier the team have aimed for. The hardware RoomAlive uses is a standard Kinect and an off-the-shelf projector. You can use multiple pairings to create a rig that can cover the entire room but you need no other kit except a frame to hang them from. The rest of the setup is handled by the software - this is what really sets RoomAlive apart from other projection mapping setups which have to be manually calibrated. You can leave RoomAlive’s cameras to create a 3D map of your room and detect the viewing range of other cameras in the rig.
The rig makes a 3D map of the space and exports it to Unity where a developer can do pretty much whatever they like to it. They can retexture it, animate the walls, create AI characters to stalk about your room - having them hide behind your furniture when you get near.
The full paper on the project can be found on RoomAlive’s project page.