Google Search AI Mode: the best facts grounded chatbot nobody knows about
At yesterday's Google IO conference, Google announced that they will overhaul the Search experience, making the difference between normal search and AI mode smaller, adding working agents that can scan for news, and "generative UI" essentially creating dynamic web sites for your replies, and people are already losing their minds.
I used to also think that "Google Search AI Mode" is one of those "let's just tack AI on all our products" kind of moves, till I realized that Google accidentally built the best web search integrated chatbot there is, and it looks like they finally realized what they have.
Google Search AI Mode is different from your usual LLMs with a WebSearch() tool, because it works more like supercharged RAG. The first step is to take your text and generate a number of potential search terms from it, then doing all those queries in parallel and then assembling context for the chatbot's answer. And because it is powered by the Google Search index, it is essentially the best facts grounded chatbot that I've come across. No more "I think this was after your training data cutoff, please do a websearch", no more "Let me search for this... interesting, let me search for one more thing... OK, now I have the full picture." This is the only chatbot I came across that you can ask about an event that happened just today without extra prompting.
The problem is, of course, that Google Search doesn't seem to be aware that they built a chatbot, but it feels more like they think they built a search with "generative result presentation." It doesn't have a shred of personality, and depending on what comes back in the result set, it might tell you a completely different story than in the turn before. And it has the worst "might I interest you in something else" engagement prompting I've ever seen!
But it's definitely a full fledged chatbot that can be used to gather data, discuss theories, and just have a chat. One time it even offered me to help me write a draft to which I thought "ok, interesting, let's see how that goes given that you don't have access to any tools or documents." Sometimes, some guardrails kick in, and Google search seems to wonder if you're still searching for stuff. And the best thing: as this comes out of Google Search, it's free, paid for by ad revenue from the classical result page.
When Google Search AI Mode was announced a year ago, they also announced a couple of coming integrations, like giving it access to your emails and personal context, or shopping, but I think that was never really scaled out. Once or twice I have seen actual product cards, but while it's happy to talk about any kind of products, the integration isn't really there. Maybe they're waiting for Google's Universal Commerce Protocol to roll out, who knows. Other companies like OpenAI have tried to push agentic shopping but rolled back in March after lack of adoption.
Google Search AI Mode (we really need a shorter term for this!) is now the third Google Product (after Google Workspace and the Gemini chatbot) that tries to integrate with your personal data, and I think this is an example of Conway's law, every org shipping its own product. And you can see the Google Search DNA in the way it is executed technically.
I actually don't think this is wrong, this is how big organizations work, and it also creates some competition and parallel exploration. What really seems to work for Google, however, is the investment in foundational AI technology. All these products are based on Gemini models, and other recent developments like TurboQuant, and their own TPU technology to run Gemini models at scale. As so often, product development and launching products is not their strongest suit, but in terms of technology, their strategy seems to work out. As recent as two years ago people were still saying that Google lost the AI race, but those days are behind us.
Now it seems Google is realizing what they have. With 1 billion monthly active users on AI mode, my original claim that "nobody knows about this" is apparently not true anymore. But I also hope that the good old ten blue links results page will not completely vanish!