AT&T’s AI Shift: From Billion-Token Bottlenecks to a Future of ‘Small’ Language Models
AT&T is undergoing a significant transformation in its approach to artificial intelligence, moving away from reliance on massive language models (LLMs) towards a more efficient, cost-effective architecture built on a stack of smaller, specialized models. This shift, driven by the sheer scale of their AI needs – averaging 8 billion tokens per day – is already yielding impressive results, including a reported 90% cost reduction.
The Problem with Substantial: Scaling AI at AT&T
The challenge wasn’t necessarily the capability of large language models, but their practicality. Pushing all AI tasks through LLMs proved unsustainable, both financially and in terms of performance. Andy Markus, AT&T’s chief data officer, and his team realized a different approach was needed.
Enter the Multi-Agent Stack: Orchestrating ‘Super’ and ‘Worker’ Agents
The solution? A multi-agent system built on LangChain. This architecture utilizes “super agents” powered by LLMs to direct smaller, more focused “worker” agents. These worker agents handle specific, concise tasks, dramatically improving speed and responsiveness. This orchestration layer is key to AT&T’s success.
Small Language Models (SLMs): Accuracy Without the Expense
A core tenet of AT&T’s new strategy is the belief in the power of small language models. Markus believes the future of agentic AI lies in “many, many, many small language models.” Interestingly, they’ve found SLMs to be just as accurate – and sometimes more accurate – than LLMs when applied to specific domain areas.
Ask AT&T Workflows: Empowering Employees with No-Code AI
This re-architected AI stack powers “Ask AT&T Workflows,” a graphical drag-and-drop agent builder now available to over 100,000 AT&T employees. The tool allows employees to automate tasks without needing coding expertise, leveraging proprietary AT&T tools for document processing, natural language-to-SQL conversion, and image analysis. The system focuses on analyzing AT&T’s own data to drive decisions.
Human Oversight: Maintaining Control in an Automated World
Despite the increasing autonomy of these AI agents, human oversight remains crucial. All agent actions are logged, data is isolated, and role-based access controls are enforced. A human “check and balance” ensures the process remains reliable and secure.
Beyond Build vs. Buy: A Pragmatic Approach to AI
AT&T isn’t focused on building everything from scratch. Instead, they prioritize using “interchangeable and selectable” models, avoiding unnecessary reinvention. They actively evaluate off-the-shelf options and will replace homegrown tools as industry standards mature. Their Ask Data with Relational Knowledge Graph, for example, has achieved top rankings on text-to-SQL accuracy leaderboards.
AI-Fueled Coding: Reshaping Software Development
The principles of breaking down tasks into smaller components are now influencing AT&T’s software development process. “AI-fueled coding” leverages agile methods and function-specific build archetypes to produce code that is “very close to production grade” in a single iteration. This approach is dramatically shortening development timelines. The team built an internal data product in 20 minutes using this technique, a process that previously took six weeks.
Employee Adoption and Productivity Gains
Early results are promising. More than half of the 100,000+ employees with access to Ask AT&T Workflows use it daily, with active users reporting productivity gains as high as 90%. Surprisingly, even technically proficient users are gravitating towards the no-code interface, highlighting the importance of user-friendly AI tools.
Frequently Asked Questions
Q: What is a ‘small language model’ (SLM)? A: An SLM is a language model with fewer parameters than a large language model (LLM). They are often more efficient and cost-effective for specific tasks.
Q: What is LangChain? A: LangChain is a framework for developing applications powered by language models. AT&T uses it to orchestrate its multi-agent AI system.
Q: Is human oversight still necessary with AI automation? A: Yes, AT&T emphasizes the importance of human oversight to ensure accuracy, security, and responsible AI practices.
Q: What kind of tasks are AT&T employees automating with AI? A: Tasks range from network troubleshooting and customer reconnection to code generation and data analysis.
What does this mean for the future? AT&T’s experience suggests a move towards more specialized, efficient AI systems. The focus will likely be on orchestrating a network of smaller models, rather than relying solely on massive, general-purpose LLMs. This approach promises to unlock significant cost savings, improve performance, and empower a wider range of users to leverage the power of AI.
What are your thoughts on the future of AI orchestration? Share your insights in the comments below! Explore our other articles on [link to related article 1] and [link to related article 2] to learn more about the latest trends in artificial intelligence. Subscribe to our newsletter for regular updates and expert analysis.
