The Rise of the Bot: How AI is Quietly Taking Over the Internet
The internet is changing, and it’s happening faster than many realize. A growing proportion of web traffic now originates from bots, driven by the surge in artificial intelligence. Recent data from TollBit reveals a dramatic shift: in the fourth quarter of 2025, roughly one AI bot visited websites for every 31 human visitors, a stark contrast to the 200:1 ratio observed in the first quarter of the same year.
From Scraping to RAG: The Evolving AI Web Presence
Initially, concerns centered around AI companies scraping websites to train their models. While this practice continues, TollBit’s data indicates a shift. Data collection for training purposes actually decreased by 15% between the second and fourth quarters of 2025. The dominant force now is Retrieval-Augmented Generation (RAG) bots.
RAG bots, utilized by companies like OpenAI and Google, extract real-time information from the internet to answer queries on platforms like ChatGPT and Gemini. Traffic from these bots increased by 33% during the same period. AI search indexers, which build the indexes used by RAG bots, saw an even more substantial increase of 59%.
OpenAI’s ChatGPT-User is identified as the biggest data collector, averaging five times more data requests per page than Meta, the second-largest collector.
The Decline of Human Traffic and the Impact on Publishers
This rise in bot traffic coincides with a decline in human visitors. Web traffic experienced a 5% decrease in human visitors between the third and fourth quarters of 2025. This trend signals a fundamental shift in how people access information.
According to Eight Oh Two, 37% of active AI users now begin their searches on platforms like ChatGPT or Gemini, bypassing traditional search engines. Pew Research indicates that 62% of American adults use AI in some form weekly.
Certain website categories are experiencing a more significant impact. B2B and professional websites, national news sites, and lifestyle content are frequently accessed by AI. However, the largest growth in data collection has been in technology and consumer electronics articles, with a 107% increase.
A Bleak Outlook for Media Companies?
The increasing reliance on RAG bots is exacerbating a critical problem for publishers: declining referral traffic. The rate of clicks from AI-powered applications back to the original source websites has plummeted. The AI-originated click-through rate fell from 0.8% in Q2 2025 to just 0.27% in Q4 2025 – a nearly threefold decrease.
Even websites with AI licensing agreements aren’t immune. Their click-through rates dropped from 6.5x in the same period, falling to 1.33% in Q4 2025.
TollBit’s co-founder and COO, Olivia Joslin, suggests the internet is rapidly becoming a space primarily for AI, not human users. The shift towards bot dominance is forcing media companies to adapt.
AI visitors consume more content, don’t experience fatigue, and can conduct deeper research than human visitors. However, the Washington Post’s recent 30% workforce reduction serves as a stark reminder of the challenges facing the industry.
The Potential Downsides of AI-Driven Information Access
While AI offers convenience, some studies suggest potential negative consequences. Reliance on AI for tasks like essay writing may hinder critical thinking skills and knowledge retention.
AI introduces another layer of algorithmic control over information access, influencing what information users encounter and how it’s presented. This reinforces the idea that the internet, as we know it, is evolving – and perhaps fading.
Navigating the New Web: What Does This Signify for You?
The rise of AI bots presents both challenges and opportunities. For publishers, it’s crucial to explore monetization strategies, such as those offered by TollBit, to ensure fair compensation for content usage. For users, it’s important to be aware of the potential limitations and biases inherent in AI-generated information.
FAQ
Q: What is RAG?
A: Retrieval-Augmented Generation is a technique where AI systems retrieve information from the internet in real-time to enhance their responses.
Q: Is AI scraping illegal?
A: Not necessarily, but it raises copyright concerns and ethical questions about fair use. Publishers are increasingly seeking ways to monetize AI access to their content.
Q: How can publishers protect their content from unauthorized AI use?
A: Implementing access controls, licensing agreements, and bot detection tools can help publishers manage and monetize AI access to their content.
Q: What is TollBit?
A: TollBit is a platform that helps publishers monitor, manage, and monetize AI usage of their content.
Did you know? AI bot activity has now surpassed traffic from Bingbot, the world’s #2 search engine.
Pro Tip: Consider implementing a clear AI usage policy on your website to define how bots can access and utilize your content.
What are your thoughts on the increasing dominance of AI on the web? Share your opinions in the comments below!
