In a debate likely to get louder as AI agent usage grows, some people say Perplexity crawling blocked websites isn't a simple ...

First, Cloudflare accused the AI company of bypassing no-crawl directives. Now Perplexity says Cloudflare has it all wrong.

The Internet is fighting over whether robots.txt applies to AI agents. It all started when Cloudflare published a blog post, ...

AI search engine Perplexity is using stealth bots and other tactics to evade websites’ no-crawl directives, an allegation ...

Perplexity accuses Cloudflare of not sharing its methodology and misattributing crawling activity to its AI assistant.

To test this, Cloudflare says it created new domains with similar restrictions against Perplexity’s AI scrapers. It found ...

Cloudflare’s recent allegations against Perplexity have sparked a debate about the wave of bots and AI agents flooding the ...

Instead, the AI firm claims Cloudflare confused Perplexity requests with unrelated traffic from BrowserBase, a third-party ...

Cloudflare accuses AI startup Perplexity of secretly scraping websites that explicitly blocked bots, claiming it bypassed restrictions and masked its identity to access protected content.

Cloudflare is accusing Perplexity of using stealth crawlers to bypass site restrictions, triggering fresh concerns over how ...

AI search biz Perplexity claims that Cloudflare has mischaracterized its site crawlers as malicious bots and that the content ...

Cloudflare set a trap for Perplexity, and the AI startup crawled right into it. This has lessons for other AI companies ...