agentability

← AI-Readiness Index

claude.ai

claude.ai blocks AI agents at the door. In our latest audit, a plain fetch of its homepage got a bot challenge or a refusal, not the page. Its robots.txt also restricts AI bots.

It is one of the online shops we score for AI shopping agents.

18/100   F #948 of 1186

Audited November 9, 2026 from public surfaces only. Can AI agents discover, read, and act on claude.ai? Check by check:

GPTBot: blocked ClaudeBot: partial PerplexityBot: partial Google-Extended: blocked CCBot: partial Claude-User: partial ChatGPT-User: blocked Perplexity-User: partial

CheckFindingPointsResult
A1 llms.txt for AI readers
No /llms.txt found — AI readers get no curated entry point.
0/15 fail
A2 AI crawler access policy
robots.txt blocks: GPTBot, Google-Extended, ChatGPT-User.
8/15 warn
A3 Content readable without a browser
A bot challenge page blocks plain fetches — agents see a wall, not your content.
0/25 fail
A4 Structured data (schema.org JSON-LD)
No machine-readable entity markup on the homepage.
0/15 fail
A5 Sitemap discoverable
Sitemap advertised in robots.txt or served at /sitemap.xml.
10/10 pass
A6 Everyday agent tasks reachable from the homepage
The homepage itself returned 403 to a plain fetch, so nothing could be reached from it. An agent hits the same wall, which is why this scores zero — but it is not evidence the pages are missing.
0/20 fail
B1 MCP server advertised
No MCP advertisement (bonus check — most sites skip this).
0/5 skip
B2 OpenAPI description published
No OpenAPI document (bonus check — only relevant for APIs).
0/5 skip
B3 UCP commerce profile published
No UCP profile (bonus check — agentic commerce, relevant to shops).
0/5 skip

What claude.ai would have to change

5 checks cost claude.ai points. Each one below is the specific remedy for what we measured, not general advice.

  1. llms.txt for AI readers0/15

    Serve a plain-text file at https://claude.ai/llms.txt listing the pages an agent most often needs — pricing, docs, support, account management — each as a one-line description and an absolute URL. It is read by an agent arriving cold with no idea of your structure. Fifteen points, and most teams ship it in an afternoon.

  2. AI crawler access policy8/15

    Your robots.txt currently refuses GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, Claude-User, ChatGPT-User, Perplexity-User. If that is deliberate, this score is the cost and the trade is yours to make. If it is inherited from an old rule or a security plugin — which is the commonest cause we see — those agents cannot read claude.ai on a user's behalf at all.

  3. Content readable without a browser0/25

    A bot challenge page blocks plain fetches — agents see a wall, not your content. An agent fetches the page and reads what comes back; it does not run your JavaScript. Server-render the main content, or serve the same text to a plain request. This is the heaviest check on the board at 25 points, and the one that most often decides whether an agent can use a site at all.

  4. Structured data (schema.org JSON-LD)0/15

    Add schema.org JSON-LD to the homepage — Organization at minimum, plus Product or Article where they apply. It is how an agent learns what claude.ai is without inferring it from prose.

  5. Everyday agent tasks reachable from the homepage0/20

    An agent could not reach pricing, support/contact, docs/api, legal/privacy from the claude.ai homepage. These are what agents are sent to find most often. Each needs a plain link an agent can follow from the homepage in one hop — not behind a menu that only appears once JavaScript runs, and not solely in a logged-in view.

  6. MCP server advertised0/5

    Optional, and rare — only a handful of the sites we track run one. An MCP server lets an agent act on claude.ai through a defined interface instead of scraping it. Advertise it at /.well-known/mcp/server-card.json.

  7. OpenAPI description published0/5

    Optional. If claude.ai has a public API, publishing an OpenAPI document at /openapi.json lets an agent call it correctly without a human reading your docs first.

How this score has moved

Unchanged at 18/100 since October 7. Every audit is a fresh fetch, so a change here means something on claude.ai changed.

18Oct 7 18Oct 12 18Oct 14 18Oct 19 18Oct 25 18Oct 26 18Oct 30 18Nov 1 18Nov 2 15Nov 3 18Nov 9

Put this score on your site

The badge reads live from this page, so it updates itself when the score changes. Free, no attribution required beyond the link it already carries.

AI-Ready 18 out of 100 for claude.ai

HTML

Markdown

Work on claude.ai? Each failed check maps to a concrete fix in the methodology — most ship in under an hour. Scores refresh weekly; request a re-run after you deploy.