Skip to content
AI bots / Apple

Applebot Extended: Apple's opt-out for AI training

Applebot Extended is a robots.txt token that Apple reads to decide how it may use pages that Applebot already crawled. Apple states plainly that it does not crawl web pages. Disallow it, and your content stays out of the training data for Apple's generative foundation models.

So the token works like Google-Extended: a usage rule, not a visitor. The actual requests in your log still come from Applebot.

The Pagereckon team Updated
Applebot extended at a glance, from Pagereckon
User-agent string
Applebot-Extended
Operator
Apple
Type
AI training crawler
robots.txt token
Applebot-Extended
Obeys robots.txt
Yes, by the operator's documentation. It is a robots.txt control; Apple says it does not crawl pages itself
Official docs
support.apple.com

Applebot versus Applebot Extended

Apple runs one crawler, Applebot. Its data powers features such as Spotlight, Siri and Safari, and Apple says it may also help train its generative models. The extended token is how you separate those two uses.

What each Apple robots.txt token controls
TokenFetches pages?Disallow means
ApplebotYesApple stops crawling, so search features lose your pages
Applebot-ExtendedNoCrawled pages stay out of Apple model training

As a result, the safe choice for most publishers is to allow Applebot and decide on the extended token alone.

How to set the Applebot Extended rule

First, add a group for the token to the robots.txt at your site root. In WordPress, your SEO plugin's robots.txt editor works, or you can upload the file yourself.

robots.txt: opt out of Apple AI training
User-agent: Applebot-Extended
Disallow: /

Also, there is nothing to block at the server. No request ever carries this name, so an .htaccess or Nginx rule would match nothing. For the same reason, Pagereckon Pro's one-click blocking does not apply to it.

What the opt-out costs you

According to Apple, very little on the search side. Pages that disallow the token can still appear in search results, and Apple says it does not consider these rules in Search ranking. Your content stays discoverable through Spotlight, Siri and Safari.

Instead, the trade is about training only. In short, your pages will not help train Apple's foundation models. If that is what you want, the rule costs you no reach in Apple's search features.

Verify Apple's crawler

Since the token sends no traffic, you verify Applebot instead. Apple gives two methods. First, a reverse DNS lookup on the IP should resolve to a host under applebot.apple.com. Second, you can match the IP against the ranges in Apple's JSON file.

If neither check passes, the request only borrowed Apple's name.

How Pagereckon shows Apple's crawls

Pagereckon reads your access log, so it shows Applebot under search crawlers with the pages it fetches. Applebot Extended never appears as a row, because it never makes a request. That is expected, not a gap in the data.

To compare Apple with the AI crawlers that do visit, open the AI crawler checker. The AI bots list covers the similar token from Google, Google-Extended.

Applebot Extended questions

Does Applebot Extended crawl my site?

No. Apple says it does not crawl web pages. It only decides how Apple may use what Applebot crawled.

Will blocking Applebot Extended remove me from Siri?

No. Apple says content stays discoverable through Spotlight, Siri and Safari, and the rule is not used in Search ranking.

How do I verify Applebot?

Do a reverse DNS lookup that resolves under applebot.apple.com, or match the IP against Apple's applebot.json list.

Can I see Applebot in WordPress?

Yes. Download the free plugin and Applebot appears with the other search crawlers.

Sources and references

Free plugin, optional Pro

See your own AI traffic in ten minutes

Install the free plugin, point it at your access log and read today's humans, bots and AI visits. Then add Pro when you want Search Console and the before and after reports.