Nvidia’s brand new robotics research lab has created the ‘mobile manipulator’, a robotic arm on wheels that utilises deep learning protocols to help you out in the kitchen. While it lacks the nuclear element from Fallout’s floating butler, Nvidia’s Mr. Manipulator (as it shall henceforth be known only to me) has the ability to open and close kitchen drawers, move objects, help you cook, and even clear a dining room table.
“We really feel that the time is right to develop the next generation of robots,” Dieter Fox, lab lead, says. “By pulling together recent advances in perception, control, learning, and simulation, we can help the research community solve some of the world’s greatest challenges.”
I believe when Fox refers to the “world’s greatest challenges”, he doesn’t mean how to make a decent apple pie. But while the team of researchers has grander ventures in mind for its sentient Segway, they thought it best to give their domestic buddy a run a test run in the kitchen before setting it loose on more complex affairs.
At an open house event on January 11 (via Designboom), the lab demonstrated how its domesticated robot buddy can operate in the real world utilising real-time environmental tracking powered by Nvidia GPUs. Specifically, Mr. Manipulator comes equipped with Nvidia’s Jetson platform for navigation and multiple Nvidia Titan GPUs for inference tasks necessary for manipulation.
Read more: Here’s the scoop on the Nvidia RTX 2060
The deep-learning perception system was trained using the cuDNN-accelerated PyTorch deep learning framework. Here’s a video of Nvidia’s robot in action:
One of the goals of the lab is to create fully integrated robotic systems out of independent projects. As such, there’s also a heap of underlying technologies brought together within the robot.
Included are systems to keep track of a robot’s environment, detect the orientation of objects, implement libraries of simple and complex actions, and utilise photorealistic simulation tools to train the robot without it breaking everything, or itself, in the process.
So while Mr. Manipulator isn’t quite at the level of nuclear automatons, fully-equipped with buzzsaws and fantastically posh accents, we’re one step closer to living the sassy robot-butler dream. And hopefully we’ll get there without any catastrophic nuclear events happening in between, too.