Age of the AI agents: OpenAI's GPT-4o and Google's Project Astra, plus an exclusive sit-down with Google CEO Sundar Pichai

AI has moved into a new era from chat bots to something out of the movie Her. Good morning, Theodore. Good morning. You have a meeting in 5 minutes. You want to try getting out of bed? Google and Open AI both debuting AI assistants that can emote. Whoops, I got too excited. Reason That image of a skull reminds me of Hamlet. Make jokes a lullaby about majestic potatoes. Now that's what I call a mash up. Translate. The objects you're showing are una mansana and in Spanish even. Remember where you left your glasses? Do you remember where you saw my glasses? Yes, I do. Your glasses were on the desk near a red apple. I'm Deirdre Bosat. And this week on tech. Check the rise of the AI agents. A new race kicked off in Gen. AI this week between Open AI and Google AI agents capable of having instantaneous real time conversation. A huge step forward from the AI that we've seen over the last 18 months and an even bigger leap from the series. Alexa's Hey, Googles that we're used to, which are slow, stunted, unbearable to actually talk to. No, no, no, you're not listening, I said Wolf's Glen restaurant in Westwood. A wolf's den is a habitat that provides wolves with you, stupid, an idiot. It started with Open AIA demo of its GPT 4 OAI assistant helping with math problems. Perfect. Now what do you get when you subtract 1 from both sides? Coding. This code fetches daily weather data for a specific location and time period and storytelling Once Upon a time in a world not too different from ours. There was a robot named Bite Altman confirming what many were thinking in a social media post after the event by simply posting her referencing a sci-fi movie from 2013 in which a man falls in love with AI that acts and sounds remarkably human like. The next day, Google answered back, showing off its Project Astra with similar capabilities. What neighborhood do you think I'm in? This appears to be the King's Cross area of London. It's a major departure from the chat bots that are designed for simple interactions. AI agents use sophisticated machine learning algorithms and natural language processing to understand context, learn from interactions, and perform more complex tasks. Yes, I spotted one just now. It's heading your way on the left side of the road. Get ready to wave it down. They can adapt to new situations autonomously. I spoke to Google CEO Sundar Pichai exclusively after the demo. Here's how he described it. I think you started seeing examples today across our keynote of what we think of as agentic capabilities. Project Astra itself is one right to be able to process the real world in front of you and constantly process it and answer it intelligently. You're not typing into a text box, waiting for a response, and then reading the output. You're actually interacting with the AI through voice, just as you would a human, so speed is a huge factor. The model is real time responsiveness, so that means that you don't have this awkward 2 to 3 second lag before you wait for the model to give a response. Open AI notes that the new GPT 4 O can respond to audio inputs in an average of 320 milliseconds. That's similar to human response time. You can also now interrupt the model as it's speaking another facet of real life conversations. That wasn't the case with chat bots 123 Hey, actually that's that's a little slow. Could you count faster? Sure thing. 12345678910 and Open AI says it's model can also now detect emotion, breathing in and breathe out. That's it. How do you feel? I feel a lot better. Plus, the model itself can be as emotional as you ask it to be. Let's amplify the drama Once Upon a Time in a world not too different from ours. Of course, there are caveats. Google's showcase of Project Astra during its IO keynote was prerecorded, and it was only two minutes long. Open AI's demonstration was live, and we counted at least 10 minutes of the Open AI team interacting with the model, not including more videos posted online. After Open AI's demo. While live, it also had its share of glitches, though, when the AI seemed to cut itself off or lose its place. Can you give me feedback on my breaths? OK, here I go. Whoa, slow a bit there Mark. You're not a vacuum cleaner breathing or accounted for. But while the AI agents aren't perfect, neither were chat bots, chat, TPT, or Gemini when they were released. They still aren't, but they've led to a wave of technological advancements and innovation that is only getting started. Sundar Pichai telling me that he expects a wide roll out of Astra sometime in the next year. It will be quality driven, just like with Google Lens. We are going to test it out, give it to more people, but then roll it out widely. That's what we did with search and so we know how to do it and scale it up. Meanwhile, Open a Eyes GBT 40, it's already available to many paying subscribers slowly rolling out for free in coming weeks. And the voice feature it's set to be available for free later this summer. What we're seeing from the models now clearly just a glimpse of what is to come. We are working at the cutting edge technology and bringing it as fast to our products as possible. The problem with comparing AI agents to the movie her you're kind of nosy, am I? You'll get used to it is that it doesn't take into account how that movie ends. As Wired magazine points out, it's not until the AI leaves that the protagonist confronts his own messy human relationships. Simple acts of being human are deferred because of an enabling AI. As users interact with these agents or assistants in a more vulnerable way, are they more at risk to be manipulated or weaponized by AI? From a privacy standpoint as well, AI agents open up a ton of questions like will they know too much about us? Do we want them seeing and hearing everything around us? Take that Google demo of Astra with the AI recording everything around you, even remembering where you left your glasses. Imagine what a hacker could do with that data, especially if you're recording in a corporate office setting. Another recent trend in AI the embrace of a move fast and break things mentality. Not long ago, generative AI was thought to be too risky, too consequential to deploy too quickly. It's why Open AI was established as a nonprofit. Elias Setzkever was an Open AI Co founder who was known for sounding the alarm on AI safety and pushing back against Sam Altman's drive to develop it quickly. For every positive application of AGI, there will be a negative application as well. One of my motivations in creating Open AI was in addition to developing this technology was also to address the questions that are posed by AGI. The difficult questions, the concerns that we raised, except now sets cover, has left Open AI after heading up the team that was responsible for steering and controlling AI systems much smarter than us. His departure coming just months after he tried to oust Sam Altman, reportedly over concerns that Altman was moving too fast and being reckless. Even Google, which has promised to balance boldness and responsibility in its development of Gen. AI, it's moving faster. So we leave you this week with our full conversation with Google CEO Sundar Pichai as the Gen. AI arms race moves into a new phase. Sundar, thank you so much for making the time After that. Fantastic, You know, So great to be here. Thank you. So this is pretty much the biggest overhaul of search that we've seen in, what, 2 decades? This new experience will be available to over a billion users by the end of the year. Why did you wait until now? In some ways, we've been evolving it continuously. The good thing about search is people comfortably use it. They take it for granted. We've been answering questions for a while, but with generative AI, we can do it a lot better. We've been testing it for a while and we now feel it's the right moment to roll it out broadly. And feedback has been good, right from the user engagement has been positive. The feedback has been great. I think it makes the product much better and so it's a great direction. What about advertisers? Because this will change the business model. In some cases you're going to get links from a traditional search. In some cases you're going to get a generative AI answer, which would move those links lower down on the page. Are they ready for this moment? What are you telling them about their ability to reach your users? You know, the great thing is users still value commercial information. Our ads work based on intent and quality and relevant at the right time. We've been able to test that in the context of AI overviews and it's working well as we expected it to. So I think it'll be a smooth transition and that's what we're seeing. I think I heard Liz Reed say that it's leading to more searches, but the generative AI or AI overview as you're calling it, is leading to more or less clicks in general. We find it's both overall increasing usage and when we look at it year on year we've been able to grow traffic to the ecosystem. So we are compared to most of the players. We are prioritizing you know approaches which will generate traffic as well. So we are working hard on that. Does it change the business model? How are you thinking about that? You know, I think about a year ago people had questions on whether this would cost too much to serve. You know we've brought down costs 80%. I don't think that would be a concern. I think the way we've been at work and the way we are rolling it out, you know, I feel like we are set up in a pretty good way and we can build on from here, right. Let's talk about costs. You brought it up. Semi analysis estimates that a single chat with a ChatGPT could cost up to 1000 times as much as a simple search. As you said, you've brought that cost down, but bringing out AI overviews to everyone in America, to over a billion users by the end of the year, that has to raise the cost on your side. You know, it's still, you know, maybe more expensive than a traditional query, but not by much. You know, where, you know, just in the last year we've made our models maybe overall about 80 times more efficient. And so you know this is what Google was set up for. You know for 25 years we've built our own infrastructure from the ground up and you know it's an area I feel super comfortable that we can actually do it well. Is that because you're using your own in house custom Tpus or do you still use GPUs for the AI overviews the search searches? I mean we are, we are a close partner of Nvidia's and we definitely use both GPUs as well as our own in house hardware. But it's more about it. It's the entire end to end, what we called as AI hyper computer, how we put it all together and run it super efficiently, right. So not material costs that go up from this form of searching that. That's right. Critics, you know, have said for some time that search has become more cluttered over the years with AI overviews, they're kind of adding more on to that. And competitors like Perplexity, to name one, have sprung up that have rethought the entire user interface, the whole entire user experience to a lot of fanfare. Why not use this moment to completely overhaul the search experience instead of adding new layers on top? No, in in some ways that's what we're doing. You know when you say AI overviews, we're kind of organising it for you. It has links in it. So it's not like something that just goes on top alone. You know, today we showed good examples where it almost organises the page for you. And so I actually viewed as we are simplifying the experience over time. I mean our feedback has been very positive as we tested. People actually find the experience getting better. So I think, I think it's an exciting direction. Would simplifying it the most be just putting it straight to Gemini? I mean, especially as users get used to other chat bots and going directly to them, why not just kind of go all in with a Gemini which was such a huge focus of the keynote in the last year? You know what? What Search does is unique in the sense that it takes the intelligence of Gemini, and we ground it with what Search knows about the world. What people really value is accurate, trustworthy information. And I think that's part of part of even in this moment, I think people find Google search very valuable and they also care about what's out there on the web. So sometimes they're looking for a quick answer, sometimes they actually want to go out and learn more. So getting that balance right is also what search does well, I think. And now you're letting technology make that judgement whether to get links directly to other websites or give a generative AI answer. How are you explaining that again to like your advertisers and merchants, you know, I mean they see it in their data, right? There are advertisers who are part of this AI overviews as we are rolling it out. I think they'll see it in their performance. Every time we have this transitions, people are a bit uncertain. I don't think we have done this from desktop to mobile. You know, when local and social content became much more available, we integrated it seamlessly in search. We're doing the same with AI and we've been doing it for a decade. So I viewed as we'll be able to build upon on it. I want to get to Project Astra because that was certainly one of the most exciting parts of the keynotes technology that we haven't really seen before. We saw a little bit of it yesterday from Open AI and its new ChatGPT 40, but it feels like broadly we're moving out of the era of chatbot and into the era of an AI agent. How do you make sure that Google wins that sort of next phase of generative AI that users are going to be increasingly using? I think you started seeing examples today across our keynote of what we think of as agentic capabilities. Project Astra itself is one right to be able to process the real world in front of you and constantly process it and answer it intelligently. We are building, you can go to Gemini and ask it to plan a trip. In search we announced Multi Step reasoning. You can write a very, very complex queries behind the scenes. We are breaking it into multiple parts and composing that answer for you. So these are all agentic directions, very early days. We're going to be able to do a lot more. I think that's what makes this moment one of the most exciting I've seen in my life. And the demo, you know, captured a lot of people's imaginations. Are those products, those features available right now? When are they available? Project Astra is something we are working to bring to Gemini, but we'll do it sometime this year. It will be quality driven, just like with Google Lens. We are going to test it out, give it to more people, but then roll it out widely. That's what we did with search and so we know how to do it and scale it up. Is that fast enough? When ChatGPT shows a demo or Open AI shows a demo a day before IO and now some of those features are being used right now. Can you guys move faster? I don't think they've shipped their demo to their users yet too. I don't think it's available in the product. So I think all of us are, you know, we are working at the cutting edge technology and bringing it as fast to our products as possible. I think I think it's good to be in that moment. But you know we we have a clear sense of how, how to approach it and we'll get it right. You've said before that Google's competitive advantage in Gen. AI is the quality of your data, not just the quantity of it. There was a report that Open AI trained GPT for on millions of hours of YouTube videos. Would you sue Open AI for violating your terms? Like, I think it's a question for them to answer. I don't have anything to add. We do have clear terms of service. And so I think normally in these things, we engage with companies and make sure they understand our terms of service and we'll sort it out. Are you doing anything to determine if they broke your terms? We have processes to do that. I'm not exactly familiar. OK, back to the Astro demo. The experience was better through glasses than through the phone. I think that was obvious to everyone could see what phone was that and what kind or what kind of glasses were those and what kind of leap and hardware do we need to really integrate AI agents into our lives. What we're showcasing is we build Gemini to be multimodal because we see use cases like that. Project Astra shines when you have a form factor like glasses. So we're working on prototypes but through Android, you know we've always had plans to work on AR with multiple partners and so over time they'll bring products based on it as well a lot of anticipation over how Apple is going to integrate open AI or sorry generative AI into its phones. What are you doing to make sure that you're in pole position in generative AI on the iPhone like you have been in search on the iPhone? You know throughout both, you know we've we've had a great partnership with Apple for the years. Over the years we have focused on delivering great experiences for the Apple ecosystem. It is something we take very seriously and I'm confident we we have many ways to make sure our products are accessible. We see that today AIO views have been a popular feature on iOS when we have tested and so we'll continue including Gemini, we'll continue working to bring bring that there. We last spoke at IO about two years ago. Do you expect to be in the same position at IO in 2025? Look, I I feel we are at an inflection point. Things seem to be happening faster. So by 2025, I think we'll make a lot of progress. A year from now, what do you hope to accomplish? You know, things like Project Astra would be something you take for granted when you use Google and you know, it'll be able to see the world around you. A wide roll out of Project Astra. Wow by by this time at IO. Yeah, absolutely. Across the US and even more, would you be at the same space as you are your same stage as you are in search rolling it out to over a billion users? Obviously, we'll be quality driven, but that's the kind of aspiration we are working towards. OK, Well, Sundar, thank you so much for taking the time. Thank you. Appreciate it.