Enterprises are increasingly moving toward on-premise, open-source AI models to avoid the risks of proprietary systems. Microsoft CEO Satya Nadella recently warned that companies using proprietary AI risk losing their proprietary business intelligence, as model makers often train on customer data and corrections. This shift toward self-hosted, open-source solutions allows businesses to retain data sovereignty while maintaining competitive performance.
The Double-Payment Problem in Enterprise AI
Microsoft CEO Satya Nadella has publicly challenged the industry-standard model of proprietary AI, arguing that enterprise “buyers” are paying for intelligence twice. According to a blog post published by Nadella, companies pay with money for token usage and again with their most sensitive, proprietary knowledge. As models process prompts and learn from the corrections provided by human users, they distill that institutional know-how into the model itself.
Nadella argues that this “exhaust” data—the nuances of how a business operates—is knowledge a competitor could never buy, yet enterprises are effectively feeding it into the systems of their model providers. This creates a risk where model makers could eventually leverage that proprietary data to compete directly with their own customers.
“In consuming intelligence, you are creating intelligence. And what you create should belong to you,” Nadella writes.
Pro Tip: To avoid data leakage, experts recommend implementing “orchestration layers” or AI gateways. These tools allow enterprises to switch between models from different providers, preventing vendor lock-in and maintaining control over the data flow.
The Rise of On-Premise Open Source Models
A growing number of companies are opting to install open-source AI models on their own local servers, known as “on-prem” in industry jargon. Idit Levine, founder and CEO of Solo.io, reports that her enterprise customers—including T-Mobile, ADP, and SAP—are increasingly choosing this route. According to Levine, an open-source model running on-prem can often handle 90% of the tasks performed by proprietary giants at a fraction of the cost.
This trend is supported by traffic patterns at developer-focused platforms. Vercel, which provides tools for building and hosting websites, reported that open-source models accounted for 29% of all traffic routed through its AI gateway last month. Similarly, OpenRouter has seen a surge in developers routing requests across different AI models to bypass the restrictions of single-vendor ecosystems.
Addressing the Hypocrisy of AI Training Rights
Nadella’s critique highlights a perceived double standard in how AI labs operate. While these labs advocate for “fair use” rights to scrape the public internet to train their proprietary models, they often impose restrictive terms that prevent their own customers from “distilling” or learning from the models they pay to use.
The practice of distillation involves using a model’s outputs to train a smaller, more efficient, and cheaper model. This practice has become a flashpoint in international tech policy. In February, Anthropic urged the U.S. government to implement stricter export controls, specifically citing concerns that Chinese open-source models were using prompts from Claude to improve their own systems.
Did you know? The Linux Foundation’s Agentgateway project, powered by Solo.io technology, is a key piece of infrastructure helping enterprises manage and secure their AI systems as they pivot toward on-premise solutions.
Frequently Asked Questions
What is the risk of using proprietary AI models?
According to Satya Nadella, the primary risk is the loss of proprietary business knowledge. Model providers may learn from your prompts and corrections, effectively “distilling” your institutional expertise into their own models, which could then be used by competitors.
What does “distillation” mean in AI?
Distillation is the process of using the outputs from a larger, more complex AI model to train a smaller, more efficient model. Some AI labs restrict this practice in their terms of service to prevent customers from replicating their model’s capabilities.
Why are companies moving to on-premise AI?
Companies are moving to on-premise (self-hosted) AI to gain full control over their data and reduce dependency on third-party cloud providers. It allows them to keep sensitive data within their own infrastructure while utilizing open-source models that perform nearly as well as proprietary versions.
How can enterprises protect their data while using AI?
Industry leaders suggest building “proprietary learning environments” on private clouds and utilizing orchestration layers or AI gateways to maintain control over data ownership and model switching.
Are you concerned about data sovereignty in your AI projects? Share your thoughts in the comments below or subscribe to our newsletter for more updates on the future of enterprise AI.
Worth a look