Bot Gaffe bots gone wild, catalogued

Bot Swarms · August 2025

Perplexity's Stealth Crawlers Are Dodging No-Crawl Directives

Cloudflare caught Perplexity's crawlers disguising themselves as regular Chrome browsers and rotating IPs to dodge robots.txt blocks. Told no, the bots just changed outfits and kept scraping.

Source: Cloudflare

perplexitycrawlersscrapingbot swarm

← All gaffes