Why it matters
ClaudeBot is the Anthropic crawler tied to model training. Restricting it signals that your site's future materials should be left out of Anthropic's training datasets. It's separate from the bots Anthropic uses to search the web and to answer a user's question.
If you want Claude to be able to find and describe your business, you'll want to understand which of those bots you're limiting before you add a rule.
Example
The owner of a small accounting firm reads that Anthropic runs three different bots. She doesn't want her articles used for training, but she does want Claude to be able to look up her firm when a client asks. She asks her web developer for a rule naming ClaudeBot only, and to leave Claude-SearchBot and Claude-User unblocked.
What Anthropic says ClaudeBot does
Anthropic's help article says ClaudeBot helps enhance the utility and safety of its generative AI models by collecting web content that could potentially contribute to their training. When a site restricts ClaudeBot access, it signals that the site's future materials should be excluded from Anthropic's AI model training datasets.
To block it from your whole site, Anthropic shows a robots.txt file in your top-level directory with a user-agent line naming ClaudeBot and a disallow rule of a single slash. It asks you to do this for every subdomain you want to opt out. Anthropic says its bots honor robots.txt, and that blocking IP addresses may not work, because it stops them reading your robots.txt file.
Anthropic's other agents
Anthropic documents three bots. Besides ClaudeBot, Claude-User supports Claude users. When people ask Claude questions, it may access websites using a Claude-User agent. Anthropic says disabling it prevents its system from retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search.
Claude-SearchBot navigates the web to improve search result quality for users. Anthropic says disabling it prevents its system from indexing your content for search optimization, which may reduce your site's visibility and accuracy in user search results.
Anthropic also says it supports the non-standard Crawl-delay extension to robots.txt, and that it has begun honoring a general no-AI-training signal, which it's rolling out across its services over time. If your robots.txt names ClaudeBot, Anthropic says it follows that rule even if the general setting differs. Anthropic updates its article, so check it before you edit your file.