AI Crawler Wars

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives