PerplexityBot: the crawler behind Perplexity's citations
PerplexityBot is the crawler Perplexity uses to surface and link websites in its search results. Perplexity states that it does not use this crawler to collect content for AI foundation models.
Perplexity answers almost always come with numbered sources. So for a publisher, this crawler decides whether your page can be one of those numbers.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
- Operator
- Perplexity
- Type
- AI search indexer
- robots.txt token
PerplexityBot- Obeys robots.txt
- Yes, by the operator's documentation. Perplexity says changes can take up to 24 hours
- Official docs
- docs.perplexity.ai
Why a firewall can hide you from PerplexityBot
Perplexity's own docs spend a whole section on Web Application Firewalls. The reason is simple: bot protection from a CDN or host can challenge or block the crawler even when robots.txt allows it.
Perplexity suggests an allow rule that checks two things together: a user agent that contains the token, and a source IP inside its published ranges. It also recommends refreshing those ranges on a schedule, since the list changes.
- Cloudflare WAF: create a custom rule with User Agent contains the token AND IP Source Address in the published list, action Allow.
- AWS WAF: create an IP set from the JSON file, add a string match for the token, and combine them in one allow rule.
- Any WAF: give the allow rule a higher priority than your blocking rules.
If your posts never appear in Perplexity answers, check this before you rewrite anything.
How to block PerplexityBot
Perplexity treats each of its agents on its own, and its docs say robots.txt changes can take up to 24 hours to apply. A single Disallow handles the crawler:
User-agent: PerplexityBot
Disallow: /This rule does not stop Perplexity-User, the agent that fetches a page live for one question. Perplexity says that fetcher generally ignores robots.txt, so it needs a separate server rule.
Pagereckon Pro offers one-click blocking for bots that crawl hard and send nothing back, such as Bytespider. The Perplexity crawler sends visitors through cited links, so it does not get that button.
What blocking PerplexityBot costs
Perplexity recommends allowing the crawler so your site appears in its search results. Blocking it works the other way: your pages lose the chance to show up as cited sources. Because Perplexity does not train foundation models with this crawler, a block buys you no training opt-out either.
As a result, blocking makes sense mainly when the crawl load hurts a small server. Even then, check your WAF and caching first.
Verify a PerplexityBot request
Perplexity publishes the crawler's IP addresses at perplexity.com/perplexitybot.json. To confirm a hit, check that its IP falls inside one of those ranges. Perplexity itself recommends combining the user agent check with the IP check.
If the IP is not on the list, a scraper borrowed the name. That request has nothing to do with Perplexity, and you can block it freely.
PerplexityBot in Pagereckon
Pagereckon files the crawler under AI search, while Perplexity-User goes under live reads. Visits from perplexity.ai count as AI referral traffic, so all three Perplexity signals sit in one Assistants row.
Then you can see the path a page takes: indexed, read live for a question, and finally clicked. Try the AI crawler checker for your site, or read about AI referral traffic to see the visits side.
PerplexityBot questions
Does PerplexityBot train AI models?
Perplexity says no. It states the crawler is not used to crawl content for AI foundation models.
Why does Perplexity not cite my site?
One common cause is a firewall or bot protection rule that blocks the crawler. Perplexity's docs show how to allow its user agent and IP ranges.
How long does a robots.txt change take for PerplexityBot?
Perplexity says up to 24 hours.
Can I track it for free?
Yes. Download the free plugin and the AI crawlers view lists it per page.