How scoring works

Eight checks, all against public surfaces, all reproducible with plain HTTP requests. No invented standards: every check is traceable to a convention that real AI systems use in 2026.

Which sites are audited, and why those

The panel is derived, not hand-picked. Every rebuild takes the current Tranco top 5,000 — a daily, citable ranking built from five independent sources — and keeps only the domains that also appear in Google's Chrome UX Report top 10,000, which measures real visits by real Chrome users. Two independent rankings have to agree before a site is in.

Two things are then removed. Infrastructure: CDNs, DNS, ad servers, API hosts and login walls — nobody is ever sent to gtld-servers.net to find a price, so scoring it tells you nothing. Pornography and piracy: neither ranking carries a category, and Google publishes none, so each domain is classified once by a model and the verdict is cached and published alongside the data. You can check every call we made in /data/classification.json, and the exact source list is recorded in /data/panel-source.json with the Tranco list ID, so anyone can rebuild the identical panel.

This replaced a hand-typed list of 113 sites that was mostly AI startups — which could not honestly answer the question this site asks, because it was not the web people actually use.

Check	What & why	Points
A1	llms.txt — the llmstxt.org convention: a curated, plain-text entry point for AI readers. Substantive file with links required.	15
A2	AI-crawler policy — robots.txt rules for GPTBot, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, CCBot. Open access scores full points; blanket blocks score zero and are labeled “closed by policy” (a stance, not a bug — but it is what it is).	15
A3	Content readable without a browser — a plain fetch of the homepage must yield real text, not a JavaScript wall or a bot challenge. Agents don't run your SPA.	25
A4	Structured data — valid schema.org JSON-LD so machines learn what the entity is.	15
A5	Sitemap — advertised in robots.txt or served at /sitemap.xml.	10
A6	Everyday agent tasks — pricing, support, docs, and legal reachable from the homepage; the links an assistant needs for the questions people actually ask.	20
B1	MCP server advertised at /.well-known/mcp.json — bonus; the emerging standard for callable sites.	+5
B2	OpenAPI document published — bonus; only relevant where an API exists.	+5

Grades

A ≥ 85 · B ≥ 70 · C ≥ 55 · D ≥ 40 · F < 40. Sites whose robots.txt blocks essentially all AI crawlers are labeled “Closed by policy” instead of graded — refusing agents is a legitimate choice, and we report it as one rather than pretending it's a defect.

The Agent Field Test

The weekly Field Test is the empirical companion to the Index: instead of checking plumbing, it watches a real agent try real errands. The rules:

Autonomous end to end	An AI producer invents each episode's tasks (grounded in the audited panel, avoiding past topics); a separate agent attempts them. No human writes, selects, or edits an episode.
Read-only, by construction	The agent's only tool is a plain HTTP GET. It cannot run JavaScript, log in, submit forms, create accounts, or buy anything.
Hard limits	At most 14 page visits per task, a fixed token budget per episode, and the model is named on every page. (We pay for the API calls for fun.)
Verbatim or nothing	Transcripts are published exactly as they happened — no retries, no cherry-picking, failures included. The raw JSON is open data.
Honest reporting	The agent is instructed never to invent facts it didn't read on a page, and to report failure plainly. "Gave up honestly" is a first-class outcome.

Reproducibility

The evaluator is ~300 lines of dependency-free TypeScript in the open repo, runs weekly in public GitHub Actions, and commits raw results to the repository. The field-test agent and producer are in the same repo. Disagree with a check or a transcript? Open an issue — everything is versioned in public.