In a cluttered open-plan office in Mountain View, California, a tall, slender robot on wheels is busy serving as tour guide and unofficial office assistant—thanks to a major language model upgrade, Google DeepMind revealed today. Robot uses the latest version of Google Gemini Big Language Model to parse commands and find paths.
For example, when told by a human to “Find me somewhere to write,” the robot would obediently go, leading the person to a clean whiteboard located somewhere in the building.
Gemini’s video and text processing capabilities—coupled with its ability to take in large amounts of information in the form of pre-recorded video tours of the office—allow the “Google Assistant” robot to understand its environment and navigate accurately when given commands that require some common sense reasoning. The robot combines Gemini with an algorithm that generates specific actions for the robot to take, such as turning around, in response to commands and what it sees in front of it.
When Gemini was introduced in December, Demis Hassabis, CEO of Google DeepMind, told WIRED that its multimodal capabilities could open up new possibilities for robotics. He added that the company’s researchers are working to test the model’s robotic potential.
IN a new sheet of paper In outlining the project, the researchers behind the work said their robot demonstrated up to 90 percent reliability in navigating, even when given difficult commands like “Where do I put the roller coaster?” DeepMind’s system “dramatically improved the naturalness of human-robot interactions and significantly increased the usability of the robot,” the team wrote.
With the help of Google DeepMind
Photo: Muinat Abdul; Google DeepMind
The demo clearly illustrates the potential of large language model to reach out to the material world and do useful work. Gemini and others chatbot Most operate within the confines of a web browser or application, although they are increasingly capable of handling visual and audio input, as well as both Google And OpenAI has demonstrated recently. In May, Hassabis demonstrated a Gemini Upgrade able to understand office layout through smartphone camera.
Academic and industrial research labs are racing to see how language models can be used to enhance the capabilities of robots. May programme for the International Conference on Robotics and Automation, a popular event for robotics researchers, lists nearly two dozen papers related to the use of visual language models.
Investors are pour money into startups that aim to apply advances in AI to robotics. Several researchers involved in the Google project have left the company to form a startup called Bodily intelligencereceived initial funding of $70 million; the project is working to combine large language models with hands-on training to give robots general problem-solving capabilities. AI Skillsfounded by robotics researchers at Carnegie Mellon University, has a similar goal. This month, the company announced $300 million in funding.
Just a few years ago, a robot would need a map of its environment and carefully chosen commands to navigate successfully. Large language models contain useful information about the physical world, and newer versions trained on images and videos as well as text, called visual language models, can answer cognitively demanding questions. Gemini allows Google’s robot to analyze visual instructions as well as verbal ones, following a sketch on a whiteboard showing a route to a new destination.
In their paper, the researchers said they plan to test the system on a variety of robots. They added that Gemini would be able to understand more complex questions, such as “Do they have my favorite drink today?” from a user with multiple empty Coke cans on their desk.
