Cloudflare proposes 'pay per crawl' ploy to help publishers monetise AI searches
A group of publishers want to put up a new kind of paywall, with the help of networking technology, cybersecurity and CDN operator Cloudflare.
Cloudflare, which manages content networks for website operators, has unveiled a new tool that seeks to block AI systems from accessing content without permission.
The idea is that this will empower publishers that are sufficiently authoritative to charge AI companies for the use of their content.
Such a “pay per crawl” model can help content creators monetise the data being used to train large language models, according to Cloudflare.
Publishers, we see you! 🙌 Cloudflare just launched pay per crawl to put control over your content back where it belongs.
— Cloudflare (@Cloudflare) July 2, 2025
Now, crawling is more transparent and controlled, by default, creating a better web ecosystem for creators like you. This is about real content… pic.twitter.com/yatB5LSBIm
It's proposed that this new revenue may help offset the lost monetizable web traffic caused by so-called 'zero click' AI-based search interfaces, which answer directly many user queries, without going to the underlying websites.
Cloudflare’s chief strategy officer Stephanie Cohen described it as "the begining of a new model for the internet".
Prominent media outlets including Condé Nast, Reddit, Pinterest, and the Associated Press are backing the venture.
