Almost a decade Previously, Google introduced a feature called Now on Tap in Android Marshmallow—tap and hold the home button and Google will display useful contextual information related to what’s on the screen. Talking about a movie with a friend over text? Now on Tap can give you headline insights without having to leave the messaging app. Looking at a restaurant on Yelp? The phone can display OpenTable suggestions with just a tap.
I’m fresh out of college, and these improvements feel exciting and magical—the ability to understand what’s on the screen and predict what actions you might want to take feels forward-looking. It’s one of my favorite Android features. It gradually morphed into Google Assistant, which is great but not quite the same.
Today, at Google I/O Developer Conference in Mountain View, California, the new features Google is introducing in its Android operating system feel a bit like the old Now on Tap—which lets you tap into contextual information around you to help with usage. your phone a little easier. Except this time, these features are backed by a decade’s worth of advances in major language models.
“I think what’s exciting is that we now have the technology to make really cool assistants,” Dave Burke, vice president of engineering on Android, told me over a Google Meet video call. “We needed to have a computer system that could understand what it saw, and I don’t think we had the technology back then to do that well. Now we do.”
I got a chance to talk with Burke and Sameer Samat, president of the Android ecosystem at Google, about what’s new in the world of Android, the company’s new ecosystem. Gemini AI Assistantand what it all means for the future of operating systems. Samat calls these updates “a once-in-a-lifetime opportunity to reimagine what phones can do and rethink Android as a whole.”
Circle to search… Your homework
It starts with Circle to Search, which is Google’s new way to approach mobile Search. Like the experience of Now on Tap, Circle to Search—which The company launched a few months ago before—more interactive than just typing into the search box. (You literally circle what you want to search for on the screen.) “It’s a very intuitive, fun, modern way of searching… It’s also more youthful because it’s fun to use,” says Burke. taste”.
Samat claims Google has received positive feedback from consumers, but Circle to Search’s latest feature especially comes from pupil comment. Circle to search can now be used for physics and math problems as users circle them—Google will provide step-by-step instructions for completing the problem without the user leaving the app curriculum.
Samat made it clear that Gemini is not just provide answers but showed students how to solve the problem. Later this year, Circle to Search will be able to solve more complex problems such as diagrams and graphs. All of this is powered by Google’s LearnLM models, fine-tuned for educational purposes.
Gemini has more context on Android
Gemini is Google’s AI assistant, which in many ways overshadows Google Assistant. Really—when you enable Google Assistant on most Android phone today, there is an option to replace it with Gemini. So naturally, I asked Burke and Samat if this meant the Assistant was headed Google’s graveyard.
“The way to look at it is that Gemini is an opt-in experience on the phone,” Samat said. “I think it’s clear that over time, Gemini is getting more and more advanced and developed. We don’t have anything to announce today, but there are some options for consumers if they want to use this new AI-powered assistant. They can try it out and we’re seeing people doing that and we’re getting a lot of great feedback.”
