The AI Scaling Struggle: Why 95% of GenAI Pilots Fail and How to Beat the Odds
Despite billions invested in generative AI, a startling statistic emerged in July 2025: 95% of enterprise GenAI pilots deliver no measurable return. This finding, from an MIT NANDA report, sent ripples through the tech world. Eight months later, companies are grappling with the complexities of scaling AI, facing challenges from vendor underperformance to data quality issues and user adoption hurdles.
The GenAI Divide: A Persistent Problem
The initial excitement surrounding GenAI has given way to a more pragmatic assessment. While the potential remains immense, simply deploying AI tools doesn’t guarantee success. The core issue isn’t the AI models themselves, but rather the “learning gap” – a disconnect between technology and effective enterprise integration. Generic tools like ChatGPT thrive with individual users due to their flexibility, but struggle within organizations lacking tailored workflows.
CIO Insights: What’s Working in AI Implementation
InformationWeek spoke with three CIOs across diverse industries to uncover strategies for successful AI scaling. Their experiences highlight common themes crucial for moving beyond pilot projects.
1. Identify a Workable Use Case: Focus on Pain Points
Sean McCormack, CIO at First Student, emphasizes the importance of understanding core business processes. His team conducted a thorough technology walkthrough, identifying pain points for drivers and dispatchers. This led to the development of Halo, an integrated AI platform streamlining transportation operations from contract acquisition to payroll.
Brian Schaeffer, CIO at OceanFirst Bank, focused on the time-consuming process of Bank Secrecy Act (BSA) and Anti-Money Laundering (AML) due diligence. AI was identified as a solution to accelerate entity checks, reducing tasks that once took half a day to mere minutes.
Padma Sastry, CIO at Lowell Community Health Center, prioritized patient needs. Recognizing the challenges faced by a diverse patient population – over half speaking a language other than English and nearly 90% below 200% of the federal poverty level – she explored AI-powered voice systems for call triage and language support.
2. Small and Steady Wins the Race: A Layered Approach
All three CIOs advocate for a measured rollout. Lowell Community Health Center initially deployed its AI operator after hours, gradually expanding its use during business hours. First Student’s Halo platform took approximately two years from inception to enterprise-wide implementation, utilizing iterative design, A/B testing, and user feedback.
OceanFirst Bank acknowledges the extensive foundational work required, likening it to building a layer cake. While initial progress may be gradual, momentum builds as successes are leveraged to create a stronger foundation.
3. Picking the Right Vendor: Partnership is Key
Selecting a vendor aligned with specific organizational needs is critical. Sastry prioritized a vendor understanding the unique challenges of a federally qualified health center, emphasizing transparency and the ability to test and evaluate ROI.
4. Tracking Success and Failure: Data-Driven Insights
Establishing clear metrics to track AI performance is essential. First Student monitors AI safety camera alerts – rolling through stop signs, seatbelt usage, distracted driving – to demonstrate measurable improvements. OceanFirst Bank uses Power BI dashboards to track AI utilization, click-through rates, and answer accuracy.
Lowell Community Health Center focuses on cost reduction and patient experience, tracking call abandonment rates and the number of calls diverted from paid answering services.
5. Fail Fast: Embracing Iteration
Not every AI project will succeed. McCormack advocates for rapid prototyping and testing of multiple ideas, quickly identifying those with the greatest potential. Peer networking is as well valuable. Schaeffer regularly consults with other banks to learn from their experiences.
What Doesn’t Work: Common Pitfalls to Avoid
Enthusiasm without a defined purpose can lead to wasted resources. Scaling AI should be driven by measurable operational improvements, not simply by the allure of fresh technology. Forgetting the complete user is another common mistake. Involving users in testing and addressing change management concerns are crucial for adoption.
Finally, neglecting data and AI governance can derail even the most promising initiatives. Ensuring data quality, security, and regulatory compliance must be prioritized from the outset.
Looking Ahead: Future Trends in AI Scaling
The shift from experimentation to implementation signals a maturing AI landscape. Future trends will likely focus on:
- AI Governance Frameworks: Increased emphasis on establishing robust governance policies to address ethical concerns, data privacy, and regulatory compliance.
- Specialized AI Models: A move away from general-purpose models towards more specialized AI solutions tailored to specific industry needs and use cases.
- Human-in-the-Loop AI: Continued integration of human oversight and intervention to ensure accuracy, fairness, and accountability.
- Low-Code/No-Code AI Platforms: Democratization of AI development through user-friendly platforms requiring minimal coding expertise.
FAQ
- What is the biggest challenge in scaling AI?
- The biggest challenge is aligning AI initiatives with specific business needs and ensuring effective integration into existing workflows.
- Why are so many GenAI pilots failing?
- Most pilots fail due to a lack of clear use cases, inadequate data quality, and insufficient attention to user adoption.
- What role does vendor selection play in AI success?
- Choosing a vendor that understands your organization’s specific needs and is willing to partner for long-term success is crucial.
Ready to dive deeper? Explore our other articles on AI implementation strategies and data governance best practices. Subscribe to our newsletter for the latest insights on AI and digital transformation.
Keep reading