Anthropic, OpenAI, and Google are currently facing a strategic crisis as competitors use “distillation”—the process of training smaller AI models on the outputs of more powerful, expensive ones—to replicate their technology at a fraction of the cost. While these companies have long defended scraping the open web as “fair use” for their own development, they now characterize the harvesting of their proprietary model outputs as a cybersecurity threat. This legal and ethical friction signals a volatile shift in how AI intelligence is protected, shared, and commoditized.
The Distillation Conflict and the “Cat-and-Mouse” Reality
The practice of distillation involves using the responses from a sophisticated AI model to train or refine a smaller, more efficient version. According to Anthropic, rivals are harvesting these outputs at scale, effectively turning billions of dollars in research into a shortcut for competitors. OpenAI and Google have issued similar warnings, framing the unauthorized access as a form of “distillation attack.”
Zilan Qian, a researcher at the Oxford China Policy Lab, notes that this struggle is an inevitable result of making AI models accessible. “It’s always a kind of a cat-and-mouse game,” Qian told Business Insider. As long as model outputs are publicly available, the technical reality is that users and competitors will find ways to access and repurpose them, regardless of terms of service.
Did you know?
The term “distillation” was originally a benign research practice where labs refined their own models. Now, the industry is split over whether using another company’s outputs for training constitutes an ethical breach or simply a standard evolution of software development.
The Symmetry of Web Scraping and Model Harvesting
A significant point of tension is the perceived hypocrisy in the AI industry’s stance. For years, AI developers have scraped the open internet—including content from blogs, news sites, and software repositories—without direct permission, arguing it falls under fair use. Website owners have pushed back, citing increased operating costs due to aggressive bot activity.
Critics point out a direct symmetry between the AI companies’ actions and the actions now being taken against them. Website operators have spent three years complaining that Anthropic and other labs use “data-sucking bots” that crawl pages thousands of times while offering minimal traffic in return. Now that AI labs are experiencing similar unauthorized data extraction from their own products, the industry is struggling to define where the line between “innovation” and “theft” actually lies.
Future Trends: Is “Distillation Panic” Sustainable?
The industry is currently experiencing what some researchers, such as Nathan Lambert, refer to as “distillation panic.” The fear is that if companies like Anthropic succeed in legally or technically restricting how their outputs are used, it could inadvertently stifle open-source research and the development of smaller, more efficient AI tools.
As tech giants tighten access to their models, the most likely outcome is a continuous cycle of workarounds. Because these systems are designed to generate human-readable text and code, they are inherently “leaky.” If information can be viewed by a human, it can be systematically collected by a bot. Legal experts suggest that distillation may ultimately be found to be fair use, mirroring the arguments AI companies have used to justify their own scraping practices.
Many site owners are implementing stricter robots.txt files to manage how AI scrapers interact with their content, though this remains an imperfect solution against determined crawlers.
Frequently Asked Questions
- What is AI distillation?
Distillation is the process of using the outputs or “knowledge” from a large, high-performing AI model to train a smaller, more cost-effective model to perform similar tasks. - Why are AI companies worried about distillation?
Companies like Anthropic, OpenAI, and Google worry that competitors are using their expensive research to build cheaper, rival models, effectively undermining their competitive advantage and return on investment. - Is distillation illegal?
The legality is currently untested. AI companies argue it violates terms of service, but some researchers suggest it may fall under “fair use,” similar to how AI companies defend their own web scraping. - Can companies stop their models from being distilled?
Technically, it is difficult. Because models are designed to provide outputs to users, those outputs can be captured. Labs are currently tightening access, but this often leads to more sophisticated workarounds by competitors.
What is your take on the “cat-and-mouse” game of AI data? Should companies be able to claim ownership over the outputs of their models, or is this simply the new reality of an open internet? Share your thoughts in the comments below or subscribe to our newsletter for more updates on AI policy and ethics.
Worth a look
