Cloudflare dropped a big change today in a series of blog posts about AI bots. I was about to post a quick take, but I’m taking more time to read all the posts first. There’s a lot there.

@manton Usually I am a huge fan of Cloudflare, but I am deeply unsure if setting up toll gates is the right answer for this problem.

@manton I'm super curious about it, but also didn't have time to read everything. I really hope that they get this right for the users.

@vladcampos I hope so too. This feels like a tipping point of sorts, where Cloudflare is going further than they have before. Mixed feelings until I read more.

@manton I sincerely hope there is more where this came from. I would love nothing more than for every AI scraper to be banished from all corners of the internet; the companies behind them driven to the wall.

@manton I’m weary of Cloudflare further strengthening their position as a gatekeeper.

I think if something in this space is going to work, it needs to be open and decentralized.

@reedandpickup I totally get this view, and I think website owners should have control. I’m concerned about broadly applying this, though. Writing a longer blog post about it.

@dfj That’s my feeling too. Cloudflare does reference an upcoming IETF standard for telling crawlers whether to allow training, but I think the payment-related stuff is (for now) specific to Cloudflare.