Cloudflare updated its crawler rules tagging bots as search AI proxy or model training. From Sept 15 2026 AI proxies and training crawlers will be blocked by default on ad-supported sites. Hybrid crawlers are blocked if any purpose violates site policy. A new dashboard adapts to AEO page change monitoring reduces duplicate requests and usage based billing lets owners charge for AI content crawling.





Home
