ClaudeBot

robots.txt

Respects robots.txt, including the non-standard Crawl-delay directive. Each Anthropic bot has its own token: target ClaudeBot to block only training.

Verify authenticity

No official method (user-agent only)

Anthropic does not publish IP ranges and its bots use public cloud IPs: IP filtering is unreliable and can even block access to robots.txt. Control is best done via robots.txt on the ClaudeBot token rather than an IP list.

SEO recommendation

Block by default

Block by default, like other training scrapers, unless you have a deliberate LLM-contribution strategy. Blocking ClaudeBot does not stop Claude-User from fetching a page at a user's explicit request.

Good to know

ClaudeBot replaced the older anthropic-ai and Claude-Web tokens. Anthropic formalized its three bots (ClaudeBot, Claude-User, Claude-SearchBot) in its documentation.