Cloudflare is rolling out a system that will help millions of websites block AI bots from accessing their content without permission.
The internet infrastructure firm, which hosts around a fifth of the internet, plans to create a system where they will introduce a ‘pay per crawl’ system, forcing AI crawlers to pay for scraping content.
This comes after many prominent artists, writers and musicians accused AI bots of training on their content without permission, causing a slew of copyright lawsuits.
The Cloudflare system is being supported by major publishers, including Condé Nast and the Associated Press, as well as social media companies such as Reddit and Pinterest.
Cloudflare's chief strategy officer, Stephanie Cohen said the tool was designed to give publishers control over their content. She also said it was to ensure a sustainable ecosystem between online content creators and AI companies.
"The change in traffic patterns has been rapid, and something needed to change," Cohen said in an interview.
"This is just the beginning of a new model for the internet."
Cloudflare's co-founder and CEO said that this move was necessary if the Internet is to survive AI age.
“AI crawlers have been scraping content without limits,” he said.
“Our goal is to put the power back in the hands of creators, while still helping AI companies innovate.
“This is about safeguarding the future of a free and vibrant Internet with a new model that works for everyone.”
Related content
