I received an email from Perplexity today with an interesting offer. Clicking through the provided link (not shown) reveals the specifics of the offer. If the unique link provided to me is used by someone I invite, and that person uses the Comet browser to ask Perplexity one question, I get $15. The person gets […]

Artificial intelligence startup Perplexity is using stealthy techniques to get around network blocks against systematic browsing and scraping of web pages, Cloudflare said Monday in a blog post.

The alleged activity prompted Cloudflare, which received complaints from its customers, to take action against Perplexity.

“There are clear preferences that crawlers should be transparent, serve a clear purpose, perform a specific activity, and, most importantly, follow website directives and preferences,” Cloudflare engineers wrote. “Based on Perplexity’s observed behavior, which is incompatible with those preferences, we have de-listed them as a verified bot and added heuristics to our managed rules that block this stealth crawling.”

It’s the latest step from Cloudflare in its approach to crawling from AI systems, following last month’s announcement allowing customers to block or charge fees from web crawlers deployed to scrape their websites and data.

Customers who disallowed Perplexity crawling activity in their robots.txt files — a file that instructs search engine crawlers which parts of a website they can and cannot access — told CloudFlare that Perplexity was still able to access their content. 

“These customers told us that Perplexity was still able to access their content even when they saw its bots successfully blocked,” Cloudflare said. “We confirmed that Perplexity’s crawlers were in fact being blocked on the specific pages in question, and then performed several targeted tests to confirm what exact behavior we could observe.”

Emails to a Perplexity spokesperson and media email address seeking a response were not immediately answered. But spokesperson Jesse Dwyer told TechCrunch that Cloudflare’s blog post was no more than a “sales pitch,” that the screenshots in the post “show that no content was accessed” and the bot Cloudflare named “isn’t even ours.”

Perplexity later responded at length in its own blog post, calling Cloudflare’s work “embarrassing” and “disqualifying.”

Perplexity has encountered allegations of unethical web scraping in the past. Most recently, the BBC has threatened to sue the company over content scraping, one of many suits AI companies are facing, although some organizations have signed deals with AI firms, including with Perplexity.

In its blog post, Cloudflare said OpenAI is an example of a company following recommended practices on crawlers and blocked behavior.

Updated 8/5/25: to include link to fuller Perplexity response.

The post AI company Perplexity is sneaking to get around blocks on crawlers, Cloudflare alleges appeared first on CyberScoop.