Google’s Gemini, its flagship artificial intelligence chatbot, has been the target of numerous “distillation attacks” by actors attempting to replicate the system. These attacks involve repeatedly prompting Gemini – in one instance, more than 100,000 times – to reveal its underlying logic, and patterns.
Model Extraction Attempts
According to a report published Thursday, these attacks, described as “model extraction,” are being carried out to gain insights into how Gemini functions. The goal appears to be using this information to build or improve competing AI models. Google believes the actors are primarily private companies or researchers seeking a competitive advantage.
Whereas Google has measures in place to identify and block these attacks, the company acknowledges that large language models are inherently vulnerable due to their public accessibility. The attacks have focused on extracting the algorithms that enable Gemini to “reason” and process information.
Broader Implications
John Hultquist, chief analyst of Google’s Threat Intelligence Group, stated that Google is likely to experience these attacks before other companies. He also indicated that smaller companies developing custom AI tools are also likely to become targets.
Google considers these distillation attacks to be a form of intellectual property theft, given the substantial investment tech companies have made in developing these large language models.
Potential Future Scenarios
attacks on AI models could become more sophisticated, potentially leading to the compromise of proprietary data used to train these systems. Companies may demand to invest further in security measures to protect their AI assets. It is also likely that the legal landscape surrounding AI intellectual property will evolve as these types of attacks become more common.
Frequently Asked Questions
What is a distillation attack?
A distillation attack involves repeatedly prompting an AI chatbot with questions designed to reveal its inner workings, with the goal of replicating the model.
Who is believed to be behind these attacks?
Google believes the attacks are primarily being carried out by private companies or researchers seeking a competitive advantage.
Is this a recent phenomenon?
No, OpenAI accused DeepSeek of conducting similar attacks last year.
As AI technology continues to advance, how will companies balance the benefits of open access with the need to protect their intellectual property?
