pexels.com blocks AI agents at the door. In our latest audit, a plain fetch of its homepage got a bot challenge or a refusal, not the page. Its robots.txt also restricts AI bots.
18/100 F #965 of 1186
Audited November 9, 2026 from public surfaces only. Can AI agents discover, read, and act on pexels.com? Check by check:
GPTBot: blocked ClaudeBot: blocked PerplexityBot: partial Google-Extended: partial CCBot: blocked Claude-User: partial ChatGPT-User: partial Perplexity-User: partial
| Check | Finding | Points | Result |
|---|---|---|---|
| A1 | llms.txt for AI readers No /llms.txt found — AI readers get no curated entry point. |
0/15 | fail |
| A2 | AI crawler access policy robots.txt blocks: GPTBot, ClaudeBot, CCBot. |
8/15 | warn |
| A3 | Content readable without a browser A bot challenge page blocks plain fetches — agents see a wall, not your content. |
0/25 | fail |
| A4 | Structured data (schema.org JSON-LD) No machine-readable entity markup on the homepage. |
0/15 | fail |
| A5 | Sitemap discoverable Sitemap advertised in robots.txt or served at /sitemap.xml. |
10/10 | pass |
| A6 | Everyday agent tasks reachable from the homepage The homepage itself returned 403 to a plain fetch, so nothing could be reached from it. An agent hits the same wall, which is why this scores zero — but it is not evidence the pages are missing. |
0/20 | fail |
| B1 | MCP server advertised No MCP advertisement (bonus check — most sites skip this). |
0/5 | skip |
| B2 | OpenAPI description published No OpenAPI document (bonus check — only relevant for APIs). |
0/5 | skip |
| B3 | UCP commerce profile published No UCP profile (bonus check — agentic commerce, relevant to shops). |
0/5 | skip |
5 checks cost pexels.com points. Each one below is the specific remedy for what we measured, not general advice.
Serve a plain-text file at https://pexels.com/llms.txt listing the pages an agent most often needs — pricing, docs, support, account management — each as a one-line description and an absolute URL. It is read by an agent arriving cold with no idea of your structure. Fifteen points, and most teams ship it in an afternoon.
Your robots.txt currently refuses GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, Claude-User, ChatGPT-User, Perplexity-User. If that is deliberate, this score is the cost and the trade is yours to make. If it is inherited from an old rule or a security plugin — which is the commonest cause we see — those agents cannot read pexels.com on a user's behalf at all.
A bot challenge page blocks plain fetches — agents see a wall, not your content. An agent fetches the page and reads what comes back; it does not run your JavaScript. Server-render the main content, or serve the same text to a plain request. This is the heaviest check on the board at 25 points, and the one that most often decides whether an agent can use a site at all.
Add schema.org JSON-LD to the homepage — Organization at minimum, plus Product or Article where they apply. It is how an agent learns what pexels.com is without inferring it from prose.
An agent could not reach pricing, support/contact, docs/api, legal/privacy from the pexels.com homepage. These are what agents are sent to find most often. Each needs a plain link an agent can follow from the homepage in one hop — not behind a menu that only appears once JavaScript runs, and not solely in a logged-in view.
Optional, and rare — only a handful of the sites we track run one. An MCP server lets an agent act on pexels.com through a defined interface instead of scraping it. Advertise it at /.well-known/mcp/server-card.json.
Optional. If pexels.com has a public API, publishing an OpenAPI document at /openapi.json lets an agent call it correctly without a human reading your docs first.
Unchanged at 18/100 since October 7. Every audit is a fresh fetch, so a change here means something on pexels.com changed.
The badge reads live from this page, so it updates itself when the score changes. Free, no attribution required beyond the link it already carries.
HTML
Markdown