eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Matthew Prince, the cofounder and chief executive officer of cybersecurity company Cloudflare, thinks the current internet model is broken. While content used to be free for anyone to access during the early days of the internet, paywalls and pop-ups have become the norm, driving readers away even as publishers struggle to make money.
But another type of reader still accesses content for free: AI companies crawl the web relentlessly, scraping data to train their large language models (LLMs). Prince envisions a new internet utopia where this dynamic is flipped, so human readers can access content for free — but he wants AI companies to pay websites for scraping their data.
How AI models have broken the internet
Prince told Fortune that Google crawls each website for six times for each referred human user, which is already a lopsided ratio. But AI companies are even worse: OpenAI’s crawl-to-referral ratio is 250:1, and Anthropic’s is a whopping 6,000:1, according to Prince.
“For these new AI systems, the value of, ‘I’m going to take your data and then in exchange I’m going to send traffic back to your site’—that’s just going to break,” he explained. “And so we have to invent some other model.”
Prince says this dynamic disincentivizes internet content creators — and if creators stop making content, then AI tools will have less and less new data to train on, causing the models to degrade in quality over time.
Prince’s vision for an internet utopia
Prince has already begun rolling out a three-part plan at Cloudflare. Currently, site owners can opt into AI Audit, a collection of tools that allows them to see and control how often AI models use their site’s content. Prince says that AI Audit will become opt-out in the first half of 2025, so that AI crawlers will automatically be blocked.
After enough original content has been blocked, Prince says the power will return to content creators who decide how much to charge AI companies to use their content for training models. The final phase includes creating a marketplace for original content, but he doesn’t have that step fully figured out yet.
“It would be great if we got to a web that was back to: Humans get content for free, and bots pay a lot for that content,” Prince says.