ClaudeBot checker: test whether Claude AI can read your site
Claude is increasingly used by professionals for research, analysis, and decision-making. When users ask Claude about products, services, or topics in your domain, Claude relies on content that ClaudeBot has successfully crawled. If ClaudeBot receives an empty JavaScript shell from your site, Claude has nothing to cite.
This free ClaudeBot checker fetches any URL using Anthropic's exact ClaudeBot user-agent and shows you the raw HTML response. Like every other AI crawler, ClaudeBot does not execute JavaScript - your client-rendered content is completely invisible to it.
Paste a URL below to see whether Claude can actually read your site.
How to use this tool
- 1
Enter the page you want Claude to be able to read. ClaudeBot is preselected on this page.
- 2
Click 'Simulate crawl' to fetch it with ClaudeBot's user-agent.
- 3
Check the status code. If it is not 200, Anthropic's crawler is being blocked or redirected before it sees your content.
- 4
Open 'Raw HTML response' and look for your headings and body copy. ClaudeBot reads the HTML as delivered and does not run scripts.
- 5
Review robots.txt for ClaudeBot, Claude-User and Claude-SearchBot, and add a Crawl-delay line if you want to slow Anthropic's crawling down instead of blocking it.
How it works
The request carries the header 'Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)'. We return the status code, the response headers and the HTML body without running any of the page's scripts.
Anthropic uses three agents. ClaudeBot gathers public web content that may be used for model training. Claude-SearchBot crawls to improve the search results Claude can draw on. Claude-User fetches a specific page when someone asks Claude a question that needs it. This test uses the ClaudeBot identity; the same raw-HTML limitation applies to the other two.
Anthropic says its crawlers honour robots.txt, including the non-standard Crawl-delay directive, and do not try to get past access controls such as logins or CAPTCHAs. So a login wall or a challenge page in front of your content ends ClaudeBot's visit. If this test returns a 200 with an empty application shell, Claude has a page it is allowed to read but nothing on it to quote.
Understanding Claude (ClaudeBot)
ClaudeBot is Anthropic's web crawler that indexes content for Claude, one of the leading AI assistants. Claude uses crawled web content to provide accurate, well-sourced answers to user questions. If your pages return empty HTML, Claude cannot reference your business, products, or expertise.
Like other AI crawlers, ClaudeBot does not execute JavaScript. It reads the raw HTML response from your server. For modern JavaScript applications, this typically means ClaudeBot sees an empty page with no content, no meta tags, and no structured data.
Claude is increasingly used in professional and enterprise contexts for research, analysis, and decision-making. When professionals ask Claude about products, services, or topics in your domain, having your content accessible to ClaudeBot means Claude can accurately represent your offering.
ClaudeBot respects robots.txt and identifies itself clearly in its User-Agent string. Pre-rendering ensures your content is accessible to ClaudeBot without any code changes to your application.
Frequently asked questions
Is ClaudeBot the same as Claude Code?▾
No. ClaudeBot is Anthropic's web crawler that indexes content for Claude's knowledge. Claude Code is a development tool. ClaudeBot crawls websites to help Claude provide accurate answers about your content.
How do I make my site visible to Claude?▾
Ensure your pages return meaningful HTML content in the initial server response. If you use a JavaScript framework like React or Vue, pre-rendering serves fully-rendered HTML to ClaudeBot while keeping your normal app for human visitors.
Can I control what ClaudeBot crawls?▾
Yes. ClaudeBot respects robots.txt directives. You can allow or block specific paths. However, blocking ClaudeBot means Claude cannot reference your content when users ask about topics you cover.
Related crawler tests
ChatGPT SEO
Check whether ChatGPT can read your content. GPTBot crawls pages to build OpenAI's training data and power ChatGPT's browsing feature. If your site returns blank HTML, ChatGPT cannot reference or recommend your content.
AI crawlerPerplexity SEO
Verify your content is accessible to Perplexity AI's search engine. PerplexityBot powers real-time web search answers - if it cannot parse your JavaScript, your site will not appear in Perplexity results.
AI crawlerGooglebot simulator
See your page exactly as Google's primary web crawler renders it. Googlebot processes JavaScript using a headless Chromium engine, but may still miss dynamically-loaded content that depends on user interaction or delayed API calls.
Search engineRelated free tools
What bots see
Enter any URL and see the raw HTML that search engine bots, AI crawlers, and social platforms receive - before and after JavaScript rendering.
Try it freeOG image checker and Open Graph debugger
Test your Open Graph and Twitter Card meta tags. Preview exactly how your links appear when shared on Facebook, LinkedIn, Twitter/X, and WhatsApp.
Try it freeSEO audit
Run a free SEO audit on any website. Check meta tags, headings, structured data, alt text, canonical URLs, and crawler accessibility.
Try it freeLLMs.txt generator
Build a valid llms.txt for your site - the proposed standard that helps AI assistants navigate your content. Spec-compliant, copy or download.
Try it freeMeta tag checker
Test every SEO and social meta tag on any page. Title, description, canonical, robots, viewport, Open Graph, Twitter - with bot-eye-view validation.
Try it freeLovable SEO audit
Audit any Lovable site for the JavaScript SEO problem. Lovable apps render in the browser - bots see an empty shell. This tool shows you what's missing.
Try it freeMake every crawler see your content
PageGlass automatically serves rendered HTML to all bots. No per-crawler configuration needed - one simple change covers Googlebot, AI crawlers, and social platforms.