llms.txt and OpenAPI for AI Agents

llms.txt is the front door for AI readers; OpenAPI and MCP are how they get past the lobby. This guide covers what to put in each, what makes the difference between a file that scores and one that doesn't, and the two lines of code it takes to serve it from whatever framework you run. 52% of the 113 sites we audit publish an llms.txt; far fewer publish one worth reading.

What llms.txt is

A Markdown file at your site root, defined at llmstxt.org. The shape is strict enough to parse and loose enough to write in ten minutes:

• An H1 with the name of the site or project — the only required element.
• A blockquote with a one-paragraph summary.
• H2 sections, each a list of links in the form - [Title](URL): one line on what's there.
• An optional ## Optional section for links that can be skipped when context is tight.

Optionally, /llms-full.txt carries the full text of your docs in one file for readers that want everything.

A template that scores

# Acme

> Acme sells widgets by subscription. Plans, cancellation and support are all self-serve; the API is public.

## Product
- [Pricing](https://acme.com/pricing): every plan with the current monthly and annual price
- [Cancel or change a plan](https://acme.com/help/cancel): the exact steps, no login needed to read
- [Contact support](https://acme.com/support): email, hours, and the phone number for billing

## Developers
- [API reference](https://acme.com/docs/api): REST, auth, rate limits
- [OpenAPI document](https://acme.com/openapi.json): machine-readable description of every endpoint

## Legal
- [Terms of service](https://acme.com/terms)
- [Privacy policy](https://acme.com/privacy)

## Optional
- [Changelog](https://acme.com/changelog)

What separates a real one from a token one

• Links, not prose. The Index checks for a substantive file with links. A paragraph about your mission scores like an empty file.
• The task pages. Pricing, cancellation, support, legal. Those are what an agent is sent to find.
• Match your crawler policy. Publishing llms.txt while your robots.txt blocks ClaudeBot, GPTBot and PerplexityBot is the paradox — we flag it in red on your report. Decide which you mean.
• Plain text. Content-Type: text/plain; charset=utf-8. Not HTML, not a redirect to a docs page.

OpenAPI: so agents can call, not just read

If you have an API, publish its OpenAPI description at a stable URL and link it from both your docs and your llms.txt. Common locations are /openapi.json and /.well-known/openapi.json; what matters is that the link is discoverable and the document is valid. Include request and response examples — an agent will copy them. Document authentication in plain words: where to get a key, which header carries it.

MCP: advertise your tools

If you expose tools over the Model Context Protocol, publish /.well-known/mcp.json describing the server endpoint. It's a bonus check on the Index because it is still rare — which is exactly why shipping it stands out.

Serving llms.txt from your framework

The file has to live at the root and return as plain text. Here is the shortest correct way in each stack.

Next.js

# drop it in public/ — served at /llms.txt automatically
public/llms.txt

FastAPI

from fastapi.responses import PlainTextResponse

@app.get("/llms.txt", response_class=PlainTextResponse)
def llms_txt():
    return open("llms.txt").read()

Flask

@app.get("/llms.txt")
def llms_txt():
    return send_from_directory(".", "llms.txt", mimetype="text/plain")

Django

from django.views.generic import TemplateView

urlpatterns += [path("llms.txt", TemplateView.as_view(template_name="llms.txt", content_type="text/plain"))]

Express

app.get("/llms.txt", (req, res) => res.type("text/plain").sendFile(path.join(__dirname, "llms.txt")));

Fastify

fastify.get("/llms.txt", (req, reply) => reply.type("text/plain").send(LLMS_TXT));

Hono

app.get("/llms.txt", (c) => c.text(LLMS_TXT));

Laravel and Rails

# both serve the public directory at the site root
public/llms.txt

Go (net/http)

http.HandleFunc("/llms.txt", func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/plain; charset=utf-8")
    http.ServeFile(w, r, "llms.txt")
})

ASP.NET Core

// wwwroot/llms.txt, with static files enabled
app.UseStaticFiles();

Angular, React and other single-page apps

Put llms.txt in the static assets that deploy to the site root (for Angular, an assets entry in angular.json with output at /). Then read the bigger warning: a single-page app fails the 25-point readability check unless the pages agents need are pre-rendered. The file helps; server-side rendering is the fix.

Check it

curl -sI https://yourdomain.com/llms.txt | grep -i '^content-type'
curl -s https://yourdomain.com/llms.txt | grep -c '](http'

The first line should say text/plain; the second is your link count. Under five, keep writing.