Big Tech focuses on in-house chips

The growing need for cost effective AI semiconductors is leading some of the biggest names in tech to craft chips designed for their AI workloads. Hoping to avoid having their margins devoured by premium silicon from NVIDIA and others, Christina Parts and Nevelis is at Amazon’s Austin Chip Design and Testing lab in Austin, TX with the details. Christina done. It’s it’s also not only about high cost, it’s also about the short supply. We’ve seen that with certain companies and trying to get a hold of their chips. That’s why so many mega cap tech companies have announced that they’re going to be you know creating further iterations of their in house chips or like open AI creating possible in house chips in the future should they be able to do so. All in an attempt to build it in house. So they’re spending billions of dollars and then reducing their reliance on 3rd party chip companies like AWS. That’s where I am right now, surrounded by all these really smart people, PhD way smarter than me. Anna Perneta Labs focusing on in house chips. They’ve been working on their chips for about 11 years and yet they still have to maintain a partnership with NVIDIA very similar to Google. Google as well has a similar case. They have their tensor processing units TP us and they are gaining in popularity. Listen it they’re seeing tremendous interest and demand from our customers. We’re also using them significantly internally for in house machine learning workloads. So for example, the Gemini models that we’ve recently announced, we’re all trained and are being served on TP US. So Google’s using their TP us and it’s very competitive. It’s very difficult to create chips in house. You’re validating a chip right now, right? This is what this oscillation machine does. Not going to pretend that I know what it’s doing, but what we do know is that not everyone gets it right. Researchers have recently found security flaws in Apple’s new chips. the OR chips I should say the M ones, the M2, and the M threes. You also have Microsoft that’s been working for years on their custom AI chips and only recently announced them to the public. We’re not sure exactly when they will be launched for sale, but it just shows how long it’s taking. But overall, John, what we’re seeing is companies like this mega cap tech companies spending billions of dollars to build and design in house so that eventually they can steer their own destiny and not be so reliant on 3rd party chip companies. Yeah, Christine, it’s interesting in a way. This is a playbook as you mentioned, that’s been going on for more than a decade. There were cloud providers, the hyperscalers taking ARM chips and applying them to things like e-mail, the very basic functions in the data center trying to drive down costs so they could grab share, remain competitive. It looks like that’s playing out in AI as well. Maybe not competing with the very highest end functions of NVIDIA and others, but those basic AI tasks, if they can handle those with their custom silicon, at least to start, then they have a profit advantage, right? Yes, exactly. And it’s a it’s a matter of being a little bit more specific than general purpose GPU’s that could do a lot of tasks. Instead they can create chips that are very, very specific and that’s how you can lower the price. That’s how you can say it’s more cost efficient. And so that’s part of one of the angles for AWS themselves is they’re offering more power efficient, cost efficient. Google said the same, I’m sure meta one and they much theirs is going to stay the same. It’s all about bringing the best quality to customers. The problem is how many customers are going to want to switch to the in house options when they’re already, you know, stuck in that NVIDIA ecosystem that Jensen Wong loves to talk so much about Or they’re using that NVIDIA developer or software named CUDA. So it’s a matter of getting the customer to switch over and then take advantage of maybe sometimes the cheaper yet more efficient options. Yeah, great. Deep dive, Christina. Thanks for bringing it to us. It’s pretty amazing. We’re just talking about it relatively recently, but to her point, years in the making for a lot of this.