Skip to content
AI bots / Perplexity

PerplexityBot: the crawler behind Perplexity's citations

PerplexityBot is the crawler Perplexity uses to surface and link websites in its search results. Perplexity states that it does not use this crawler to collect content for AI foundation models.

Perplexity answers almost always come with numbered sources. So for a publisher, this crawler decides whether your page can be one of those numbers.

The Pagereckon team Updated
Perplexitybot at a glance, from Pagereckon
User-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
Operator
Perplexity
Type
AI search indexer
robots.txt token
PerplexityBot
Obeys robots.txt
Yes, by the operator's documentation. Perplexity says changes can take up to 24 hours
Official docs
docs.perplexity.ai

Why a firewall can hide you from PerplexityBot

Perplexity's own docs spend a whole section on Web Application Firewalls. The reason is simple: bot protection from a CDN or host can challenge or block the crawler even when robots.txt allows it.

Perplexity suggests an allow rule that checks two things together: a user agent that contains the token, and a source IP inside its published ranges. It also recommends refreshing those ranges on a schedule, since the list changes.

  • Cloudflare WAF: create a custom rule with User Agent contains the token AND IP Source Address in the published list, action Allow.
  • AWS WAF: create an IP set from the JSON file, add a string match for the token, and combine them in one allow rule.
  • Any WAF: give the allow rule a higher priority than your blocking rules.

If your posts never appear in Perplexity answers, check this before you rewrite anything.

How to block PerplexityBot

Perplexity treats each of its agents on its own, and its docs say robots.txt changes can take up to 24 hours to apply. A single Disallow handles the crawler:

robots.txt: leave Perplexity's index
User-agent: PerplexityBot
Disallow: /

This rule does not stop Perplexity-User, the agent that fetches a page live for one question. Perplexity says that fetcher generally ignores robots.txt, so it needs a separate server rule.

Pagereckon Pro offers one-click blocking for bots that crawl hard and send nothing back, such as Bytespider. The Perplexity crawler sends visitors through cited links, so it does not get that button.

What blocking PerplexityBot costs

Perplexity recommends allowing the crawler so your site appears in its search results. Blocking it works the other way: your pages lose the chance to show up as cited sources. Because Perplexity does not train foundation models with this crawler, a block buys you no training opt-out either.

As a result, blocking makes sense mainly when the crawl load hurts a small server. Even then, check your WAF and caching first.

Verify a PerplexityBot request

Perplexity publishes the crawler's IP addresses at perplexity.com/perplexitybot.json. To confirm a hit, check that its IP falls inside one of those ranges. Perplexity itself recommends combining the user agent check with the IP check.

If the IP is not on the list, a scraper borrowed the name. That request has nothing to do with Perplexity, and you can block it freely.

PerplexityBot in Pagereckon

Pagereckon files the crawler under AI search, while Perplexity-User goes under live reads. Visits from perplexity.ai count as AI referral traffic, so all three Perplexity signals sit in one Assistants row.

Then you can see the path a page takes: indexed, read live for a question, and finally clicked. Try the AI crawler checker for your site, or read about AI referral traffic to see the visits side.

PerplexityBot questions

Does PerplexityBot train AI models?

Perplexity says no. It states the crawler is not used to crawl content for AI foundation models.

Why does Perplexity not cite my site?

One common cause is a firewall or bot protection rule that blocks the crawler. Perplexity's docs show how to allow its user agent and IP ranges.

How long does a robots.txt change take for PerplexityBot?

Perplexity says up to 24 hours.

Can I track it for free?

Yes. Download the free plugin and the AI crawlers view lists it per page.

Sources and references

Free plugin, optional Pro

See your own AI traffic in ten minutes

Install the free plugin, point it at your access log and read today's humans, bots and AI visits. Then add Pro when you want Search Console and the before and after reports.