AI bots now account for 13% of all web traffic
AI bot traffic now accounts for 13% of all web sessions, up from 8% a year prior.
That is according to analysis of more than 3bn website visits from the 12 months up to May 2026 conducted by data and technology company 51Degrees.
The data shows a 62.5% increase in bot traffic from AI companies during that timespan. According to 51Degrees, the most prolific bot during the period was AmazonBot, which represented 4.1% of total web sessions.
This was followed by bots for Meta (2.9%), Bing (1.35%), Anthropic (0.19%), OpenAI (0.09%) and Google Gemini (0.0003%).
It’s worth noting, however, that Google also uses its traditional search bot for the purposes of AI scraping. This data was not included in 51Degrees’ analysis, as it would skew the results given it is impossible to tell whether that bot is scraping for the purposes of traditional search or for AI search alone.
The UK Competition and Markets Authority (CMA) moved in June to require Google to allow publishers to opt out of their content being used to power AI features, though Google still has months to implement the changes.
The comparative levels of AI bot traffic from the different AI companies could be related to their phase of AI development. OpenAI, for example, may have already scraped most of the open web by May 2025, hence the sub-1% figure.
Some AI companies have also been known to utilise third-party firms to steal content from publishers. As Press Gazette reported last year, those third parties have emerged to provide AI giants access to data in a “hands off” fashion, a loophole that has muddied attempts by publishers to block AI scraping.
Within 51Degrees’ analysis, these third-party efforts would either be shown as other bots, or could otherwise be disguised as human traffic. AI bot traffic serving Amazon could therefore exceed the 4.1% figure.
AI bots make up a relatively small fraction of total bot traffic online. Cloud company Fastly has previously estimated that nearly half (49%) of total web traffic is actually bot traffic, 99% of which is unwanted by publishers. Those bots include data scrapers, impersonators, automated attackers, and other forms of generic automation. They can inflate web traffic figures and cause publishers to pay for increased bandwidth they otherwise do not need.
However, as 51Degrees CEO James Rosewell pointed out, AI bot traffic is potentially the most harmful to publishers because it is used to scrape copyrighted content and surface it in response to AI chatbot queries, undermining publishers’ ability to receive referral traffic from search and, therefore, monetise the content they produce.
According to the Association of Online Publishers (AOP), digital publishers collectively experienced a 4.6% decline in revenues in Q1 2026, the “first tremors of an earthquake” driven by steep declines in referral traffic from search.
“Whilst AI bots only represent a small percentage of overall bot traffic, they are the most insidious because they not only consume vast quantities of bandwidth but they also use that stolen content to create AI answers that steal traffic from the original creator of that content,” Rosewell said.
In response, 51Degrees is launching a new bot protection product aimed at independent publishers that maintains a constantly updated list of AI bots and automatically updates website Robots.txt files to ensure unwanted bots are directed to the website’s licensing information.
It also integrates Movement for an Open Web’s (a non-profit co-founded by Rosewell) Search Only Contract, a set of usage terms aimed at giving publishers legal recource against unwanted data harvesting through a pathway to claiming payment from AI giants.
For Rosewell, the sharp uptick in AI bot traffic could, in theory, be “a huge licensing revenue opportunity” for publishers. However, publishers currently lack a standardised marketplace by which to license access to their content to AI companies in return for fees. The Standards for Publisher Usage Rights (SPUR) coalition is actively developing a telemetry standard that would be used as the basis for any such future marketplace.
Left unchecked, Rosewell warned, AI bot traffic presents “a growing and unwanted burden, particularly for smaller, independent players who can ill afford the impact of IP theft and increased bandwidth costs”.
How SPUR is building an economic model for publishers – with technical lead Alex Springer