Get your hands on Google’s Gemini-powered Smart Glasses, Android XR, and Project Moohan Headset
Of course, you can work in mixed reality environments with a connected Bluetooth keyboard and mouse, and you can place yourself in an immersive environment if you want to concentrate, or enable see-through mode to make sure your co-workers don’t take pictures and giggle while you wear a ridiculous headset to get work done. It’s unclear whether you’ll be able to connect a headset to your laptop to turn your work into mixed reality, a feature available on the Apple Vision Pro.
Gemini in XR
A tap on the side of the headset brings up the app launcher, and this is where you can turn the Gemini on if you want it to stay “on.” Once turned on, there will be an icon at the top of the virtual space letting you know that everything you say and see is being registered by Gemini.
In see-through mode, you can get up close to an object and ask Gemini about it—a Google employee demoing the headset (before I tried it out) walked up to another person wearing an FC Barcelona shirt and asked asked Gemini to find “this object’s chart”. team.” Gemini quickly registered the team name and presented search results with league standings and scores from recent matches.
You can ask Gemini anything similar and they will answer with visual results displayed on the headset. I asked it to “take me to Peru” and it opened a 3D version of Google Maps. I was able to move around and focus on Lima, and in cities where Maps already has multiple 3D models, you can explore areas in more detail. You can keep chatting with Gemini during these experiences, so I asked questions like when is the best time to visit and got quick answers.
In another example, I peeked inside a New York City restaurant to take a virtual tour of the space. Google says it can use AI to stitch together images of a place’s interior and display it so you feel like you’re there. It worked out pretty well, and I asked Gemini if the place took reservations without having to specifically say the name because I was staring at the restaurant name. It do takes reservations, but Gemini can’t actually make a reservation for me. (That integration may come later.)
Next, I watched some YouTube videos, where the 2D content looked sharp and colorful. Stereoscopic content is even better; My senses feel surrounded. I watched some hikers walking along the trail and asked Gemini where all this was and they replied, “New Zealand.” I can’t verify that, but it seems like the correct answer. I’ve seen some 2D videos play back more spatially where the virtual player adds depth and layering to make them feel 3D. I switched to the Google TV app and turned on “Cinema Mode” to launch a virtual cinema to watch movies and shows, just like on other VR headsets.
Circle to searchGoogle feature launched earlier this year on Android phonesalso available in Android XR. Just walk up to an object near you, press the top button on the headset, then pinch and draw a circle around the thing you want to know more about. You’ll get a Google Search page with results.
Smart glasses
Project Moohan feels like Google and Samsung are playing catch-up with the rest of the VR market, although the integration of Gemini gives their effort a layer of uniqueness. However, I have to admit that I was a lot more excited to try out the smart glasses, where Gemini felt it could be even more useful. They did not disappoint. I walked into another room and there were several pairs of glasses in front of me. Some are sunglasses, others have clear lenses. Like headphones, you can get them loaded with your prescription. Google did not provide a name for the prototype glasses.
