Applebot Extended: Apple's opt-out for AI training
Applebot Extended is a robots.txt token that Apple reads to decide how it may use pages that Applebot already crawled. Apple states plainly that it does not crawl web pages. Disallow it, and your content stays out of the training data for Apple's generative foundation models.
So the token works like Google-Extended: a usage rule, not a visitor. The actual requests in your log still come from Applebot.
Applebot-Extended
- Operator
- Apple
- Type
- AI training crawler
- robots.txt token
Applebot-Extended- Obeys robots.txt
- Yes, by the operator's documentation. It is a robots.txt control; Apple says it does not crawl pages itself
- How to verify
- Requests come from Applebot: reverse DNS to *.applebot.apple.com, or the IP list in applebot.json
- Official docs
- support.apple.com
Applebot versus Applebot Extended
Apple runs one crawler, Applebot. Its data powers features such as Spotlight, Siri and Safari, and Apple says it may also help train its generative models. The extended token is how you separate those two uses.
| Token | Fetches pages? | Disallow means |
|---|---|---|
| Applebot | Yes | Apple stops crawling, so search features lose your pages |
| Applebot-Extended | No | Crawled pages stay out of Apple model training |
As a result, the safe choice for most publishers is to allow Applebot and decide on the extended token alone.
How to set the Applebot Extended rule
First, add a group for the token to the robots.txt at your site root. In WordPress, your SEO plugin's robots.txt editor works, or you can upload the file yourself.
User-agent: Applebot-Extended
Disallow: /Also, there is nothing to block at the server. No request ever carries this name, so an .htaccess or Nginx rule would match nothing. For the same reason, Pagereckon Pro's one-click blocking does not apply to it.
What the opt-out costs you
According to Apple, very little on the search side. Pages that disallow the token can still appear in search results, and Apple says it does not consider these rules in Search ranking. Your content stays discoverable through Spotlight, Siri and Safari.
Instead, the trade is about training only. In short, your pages will not help train Apple's foundation models. If that is what you want, the rule costs you no reach in Apple's search features.
Verify Apple's crawler
Since the token sends no traffic, you verify Applebot instead. Apple gives two methods. First, a reverse DNS lookup on the IP should resolve to a host under applebot.apple.com. Second, you can match the IP against the ranges in Apple's JSON file.
If neither check passes, the request only borrowed Apple's name.
How Pagereckon shows Apple's crawls
Pagereckon reads your access log, so it shows Applebot under search crawlers with the pages it fetches. Applebot Extended never appears as a row, because it never makes a request. That is expected, not a gap in the data.
To compare Apple with the AI crawlers that do visit, open the AI crawler checker. The AI bots list covers the similar token from Google, Google-Extended.
Applebot Extended questions
Does Applebot Extended crawl my site?
No. Apple says it does not crawl web pages. It only decides how Apple may use what Applebot crawled.
Will blocking Applebot Extended remove me from Siri?
No. Apple says content stays discoverable through Spotlight, Siri and Safari, and the rule is not used in Search ranking.
How do I verify Applebot?
Do a reverse DNS lookup that resolves under applebot.apple.com, or match the IP against Apple's applebot.json list.
Can I see Applebot in WordPress?
Yes. Download the free plugin and Applebot appears with the other search crawlers.