‘End of an era for search as we know it’? Publishers grapple with gen-AI search

Analysis

Generative-AI search, be it through Google’s AI Overviews and its recently announced AI Mode, or other AI chatbots like OpenAI’s ChatGPT, is upending the broader search market and, with it, the potential reliability of search traffic for publishers.

According to one representative from a publishing industry trade body, sentiment among UK publishers is that generative AI has led to “the end of an era for search as we know it”.

The shift takes place as consumers have increasingly adopted generative AI for search purposes. According to a recent report from Adobe, traffic to US retail websites via generative-AI sources increased by 1,200% in February compared with July 2024.

Apart from non-Google AI chatbots, consumers have also in recent years begun exploring other alternatives via social platforms like TikTok and Reddit for product recommendations and answers to basic queries. TikTok has leaned in to search as a way to drive growth for its commerce business, TikTok Shop. Reddit, meanwhile, has said it is investing in an improved search experience that it will later look to monetise via advertising.

In response to the rapidly changing search market, publishers have begun aggressively planning audience strategies with “reduced search dependency”, The Media Leader understands, with one audience development lead remarking that “with [Google’s] AI Mode on the horizon, if your business is search-referral dependent, then goodnight, Vienna”.

Organic search traffic down

According to Google Search vice-president of product, Robby Stein, AI Overviews have quickly become one of the tech giant’s “most popular search features”, with more than 1bn users accessing it.

Stein announced in a recent blog post that Google is meeting that demand with an upgraded version of AI Overviews in the US, tailored to help with “harder questions”, initially on the topics of coding, advanced maths and multimodal queries. In addition, teen accounts are now allowed to use AI Overviews and users no longer need to sign in to access the feature.

But perhaps more notable is the much more substantial overhaul of the search experience through what Google is dubbing AI Mode — an expansion of AI Overviews that allows users to ask multi-step questions and which delivers a full page of AI-generated text in response to queries.

The new feature — which is currently only available in the US — has spooked publishers, many of which have already reported declining search traffic following the introduction of AI Overviews in the UK in August 2024. The Media Leader previously reported that, by November, AI Overviews were negatively impacting search traffic referrals to publishers by 5-10% on average.

Publishers say Google’s AI Overviews have reduced traffic potential

However, this was seen as a “conservative ballpark” and commentators have now indicated that they have seen data suggesting the decline in referral traffic is more likely at least 25%.

According to data from SEO agency Authoritas, the presence of AI Overviews has, on average, reduced click-through rates by 40-52% on desktop. In a Press Gazette op-ed, ex-Financial Times director of platform strategy and public affairs David Buttle commented that such figures struck him as “entirely credible”.

Meanwhile, Similarweb data for February, seen by The Media Leader, shows that organic search traffic was generally down year on year across major publishers, although figures varied significantly. For example, organic search traffic for The Guardian dipped 3.3%, The New York Times 4.8%, USA Today 10%, The Washington Post 14.6% and CNBC 20.1%.

In contrast, the BBC grew organic search traffic by 8.1%, Fox News was practically flat with 0.9% growth and the Associated Press saw a substantial 83.8% increase in traffic — potentially attributable to Google cutting a deal with the publisher in January that allows its Gemini chatbot to deliver up-to-date news from the news service.

The challenge for publishers is that it is difficult to directly attribute any such changes in search traffic directly to consumers’ use of AI Overviews, as opposed to other changes in SEO or shifts in user behaviour driving them away from using search more generally.

In Similarweb’s recent SEO Benchmarks report, for instance, an analysis of the percentage of AI Overviews served in response to queries found, surprisingly, that there was no correlation between a higher likelihood of AI Overviews being served and more zero-click searches.

Accordingly, it’s possible that AI Overviews don’t necessarily discourage users from clicking, or that click behaviour varies by topic regardless of the presence of AI Overviews on search queries.

‘Different for every publisher’

Google does not share data with publishers on how precisely AI Overviews impact their traffic, as the tech giant does not currently separate click-through data via AI Overviews from click-through data via regular search.

Liz Reid, Google’s head of search, has previously written that Google has found “links included in AI Overviews get more clicks than if the page had appeared as a traditional web listing for that query”, although publishers have expressed doubts on the veracity of this statement.

A spokesperson for Google further told The Media Leader that AI Overviews “make it easier than ever for people to find the information they need and discover relevant sites across the web, which opens up more opportunities to connect with publishers, businesses and creators”.

What is clear is that, thus far, the data is largely unclear, leaving publishers’ audience development teams uncertain of the overall impact and predictability of AI search on their referral traffic. One audience development lead at a UK publisher told The Media Leader that the effect is likely “different for every publisher”.

“We’ve seen a big uptick in search referrals in the last four months, but I have no idea how that relates to the actual amount of times AI services have shown our content to users,” this person said.

In response to the lack of transparency, they told The Media Leader that they are onboarding with Tollbit, a startup that helps monitor, manage and monetise AI bots scraping content from publishers.

According to Tollbit, traffic from AI bots is on the verge of surpassing human traffic across the web. In Q4 2024 alone, the company said it detected 298m AI bot scrapes on publisher websites, including 65m that bypassed or ignored robots.txt instructions set by publishers to disallow AI crawlers on their websites.

How The Guardian is expanding its commercial footprint — with Imogen Fox