Google boosts AI agents strategy with next-gen chips to take on Nvidia

After years of building chips that could both train AI models and run them in real time, Google is now breaking that approach apart. The company is introducing dedicated processors for each task, a strategic pivot aimed at improving efficiency and sharpening its challenge to Nvidia in the booming AI hardware market.

The shift comes with Google’s eighth-generation tensor processing units, or TPUs, which will roll out later this year in two distinct forms. The move reflects how rapidly AI workloads are evolving, especially with the rise of autonomous systems.

“With the rise of AI agents, we determined the community would benefit from chips individually specialized to the needs of training and serving,” Amin Vahdat, a Google senior vice president and chief technologist for AI and infrastructure, said in a blog post.

More from Tech

Google's new chip strategy for the AI era

Google’s new TPU 8t is designed specifically for training large-scale models, while the TPU 8i focuses on inference, delivering faster responses for applications such as AI agents. The company says the training chip delivers 2.8 times the performance of its previous generation at the same cost, while the inference chip offers an 80 per cent improvement.

A key change lies in memory architecture. The TPU 8i uses significantly more SRAM, with 384 megabytes per chip, triple the amount in the prior Ironwood version. This allows for faster data access and reduced latency, crucial for real-time AI interactions.

The infrastructure is built to scale aggressively. Google says it can cluster 9,600 chips into pods for training, expand those to over 1,30,000 chips, and ultimately connect up to one million chips for the most demanding workloads.

The architecture is designed “to deliver the massive throughput and low latency needed to concurrently run millions of agents cost-effectively,” Sundar Pichai, CEO of Google parent Alphabet, wrote in a blog post.

Despite these advances, Google is not directly positioning its chips against Nvidia’s offerings, even as competition intensifies. Nvidia recently highlighted its own upcoming hardware designed to accelerate inference, signalling a broader industry shift towards specialised AI silicon.

Enterprise AI takes centre stage

The chip overhaul is part of a larger push into enterprise AI, where Google sees the most reliable revenue opportunities. At its annual cloud conference in Las Vegas, executives made it clear that AI agents, rather than experimental tools, are now at the heart of its strategy.

"The experimental phase is behind us, and now the real challenge begins," said Thomas Kurian during the opening keynote.

Google is consolidating its AI offerings under “Gemini Enterprise”, expanding tools like Vertex AI to help businesses build and deploy custom AI agents. These agents can autonomously plan and execute tasks, but also raise concerns around governance and safety, prompting Google to introduce new controls and oversight features.

The company is also committing heavily to infrastructure. Pichai reaffirmed plans to spend up to $185 billion this year, with more than half of its machine learning investment directed towards cloud services.

Competition is intensifying on multiple fronts. While rivals like OpenAI and Anthropic are pushing application-layer tools, and Microsoft and Amazon continue to invest in custom silicon, Google is betting on a vertically integrated approach, spanning chips, models and enterprise software.

That strategy appears to be gaining traction. Google Cloud has grown its market share to 14%, supported by adoption from major organisations, including financial firms and US national laboratories.

“There’s definitely a strategic shift as the models become much more sophisticated,” Kurian said.