The Novel Era of Agentic AI: Why CPUs are Making a Comeback
For years, the narrative around artificial intelligence has been dominated by the GPU. While graphics processing units remain essential for training large-scale models, a significant shift is occurring in how AI infrastructure is built. The industry is moving toward “agentic AI”—autonomous systems capable of reasoning, planning, and executing complex, multi-step tasks.
Unlike the massive data crunching required for training, agentic AI creates a surge in demand for CPU-intensive workloads. This includes real-time reasoning, code generation, search, and the orchestration of complex workflows. What we have is precisely where custom silicon, such as AWS Graviton, enters the spotlight.
The Pivot to “Always-On” Reasoning
The distinction between training and inference is becoming more pronounced. While Nvidia GPUs are the gold standard for training AI models on vast datasets, CPUs are increasingly preferred for “always-on reasoning workloads.” These are tasks that require constant decision-making and efficient execution at scale.
For a company like Meta, which serves billions of users across Facebook and Instagram, the ability to run content recommendations and AI interactions continuously and cost-effectively is critical. By shifting specific workloads to Graviton processors, companies can reduce the immense compute costs associated with running AI for a global user base.
Diversifying the AI Hardware Stack: Beyond the GPU Hype
The current trend in AI infrastructure is the “portfolio approach.” No single piece of hardware is suited for every task. To maintain a competitive edge, tech giants are diversifying their compute portfolios to balance performance, cost, and energy efficiency.
Meta’s strategy exemplifies this diversification. While they have made combined infrastructure commitments of $48 billion with CoreWeave and Nebius to access Nvidia GPUs, they are simultaneously integrating AWS Graviton CPUs. This hybrid approach allows them to use the right tool for the right job: GPUs for the heavy lifting of model training and Graviton for the agility required by agentic AI.
The Rise of Custom Silicon in the Cloud
The race for AI dominance is no longer just about who has the best model, but who controls the silicon. Hyperscalers are increasingly designing their own chips to lower costs for customers and reduce dependency on external vendors.
- AWS: Has developed a robust chip portfolio including Graviton CPUs, Trainium accelerators, and Nitro EC2 NICs. The annual revenue run rate for this business has surpassed $20 billion.
- Google Cloud: Is expanding its custom chip business, utilizing Broadcom as a co-designer to power models like Gemini.
- Microsoft Azure: Is also developing its own custom chips to compete in the cloud infrastructure space.
This movement toward custom silicon allows cloud providers to offer specialized hardware that is purpose-built for specific AI demands, such as the Graviton5 cores which provide the faster data processing and greater bandwidth necessary for autonomous agents.
Future Trends in AI Compute Infrastructure
As we look forward, the integration of Arm-based architectures will likely accelerate. As Graviton chips are based on Arm architecture, they offer a combination of performance and energy efficiency that is vital for data centers operating at a massive scale.
We can expect to spot more “agent-first” infrastructure. As AI evolves from simple chatbots to agents that can actually do work—like booking travel or managing software deployments—the demand for high-performance CPUs that can coordinate these multi-step workflows will only grow. This shift will likely lead to further price competitions among cloud providers as they strive to offer the most cost-effective “reasoning” compute.
For more insights on how hardware affects software, check out our guide on optimizing AI workloads.
Frequently Asked Questions
What is agentic AI?
Agentic AI refers to autonomous systems that can reason, plan, and execute complex, multi-step tasks independently, rather than just responding to prompts.
Why use CPUs instead of GPUs for AI?
While GPUs excel at training models, CPUs (like AWS Graviton) are often more cost-efficient and scalable for “reasoning” workloads, post-training refinements, and real-time AI interactions.
What is AWS Graviton?
Graviton is a custom, Arm-based CPU designed by Amazon Web Services to provide faster, cheaper, and more energy-efficient cloud computing.
How is Meta diversifying its AI hardware?
Meta uses a mix of its own data centers, custom hardware, and partnerships with cloud providers. This includes using Nvidia GPUs via CoreWeave and Nebius, as well as AWS Graviton chips for specific AI workloads.
Join the Conversation
Do you think custom silicon will eventually replace the dominance of general-purpose GPUs in the AI space? Let us know your thoughts in the comments below or subscribe to our newsletter for the latest in tech infrastructure!
