llms.txt is a plain-text Markdown file at the root of a site (/llms.txt) that gives AI agents and large language models a curated, machine-readable map of your most important content. It's an H1 with your site name, a one-line summary, and H2 sections listing links to your key pages — so a model can find and load the content that matters instead of crawling and guessing. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at llmstxt.org; the spec was revised as v2 in August 2026.

Is llms.txt the same as robots.txt or sitemap.xml?

No. robots.txt tells crawlers what they may and may not access; sitemap.xml lists every URL for search-engine indexing. llms.txt does neither — it's a curated, human-written guide to your best content, written for an LLM that needs to load context, not a crawler that needs to enumerate URLs. The three are complementary: ship all of them. robots.txt and sitemap.xml are for discovery and crawl control; llms.txt is for comprehension.

What is llms.txt? A plain-English guide

llms.txt is a plain-text Markdown file served at the root of a site (/llms.txt) that gives AI agents and large language models a curated, machine-readable map of your most important content. Instead of leaving a model to crawl your HTML and guess what matters, you hand it an H1 with your site name, a one-line summary, and a set of H2 sections linking to the pages it should actually read. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at llmstxt.org, which shipped a v2 revision in August 2026 based on two years of real-world adoption.

What goes in an llms.txt file?

The format is deliberately minimal so any model can parse it: one H1 with your project name, a blockquote summary, then H2 sections — each a bulleted list of links with a short description after the colon. Point links at the Markdown version of each page where you have one. An Optional section is the conventional home for secondary links — v2 of the spec dropped the special skip-this semantics it carried in v1.

# Acme

> Acme is an API that turns messy addresses into clean, validated ones.

## Docs

- [Quickstart](https://acme.com/docs/quickstart.md): Make your first call in 5 minutes
- [Authentication](https://acme.com/docs/auth.md): API keys and OAuth
- [API reference](https://acme.com/docs/api.md): Every endpoint and parameter

## Optional

- [Changelog](https://acme.com/changelog.md): Recent releases
- [Pricing](https://acme.com/pricing.md): Plans and limits

Serve it as text/plain so agents (and the L9 content-type check) treat it as a text document, not HTML. For larger docs sets, pair it with an llms-full.txt companion that inlines the full content in a single fetch.

How do agents discover llms.txt and Markdown pages?

The v2 revision (August 2026) answers the commonest question adoption raised: given a page, how does an agent find its Markdown version, or the llms.txt file that covers it, without guessing? The answer is two standard link relations. rel="alternate" type="text/markdown" points from a page to its Markdown twin, and rel="describedby" points to the covering llms.txt. Both can be declared as HTML <link> elements or as an HTTP Link: response header — the header form also works for non-HTML resources (the Markdown files themselves) and can be set in server or CDN configuration without touching pages:

Link: </docs/page.md>; rel="alternate"; type="text/markdown",
      </llms.txt>; rel="describedby"

v2 also blesses both Markdown URL forms — page.html.md (appended) and page.md (extension replaced) — and defines subpath files: an llms.txt covers the pages under its path, and the most specific file applies, so /docs/llms.txt covers everything in /docs/. That lets a site that only controls a path, such as a GitHub Pages project, participate fully. The Agent Ready scanner checks both discovery forms (check P17) and probes both Markdown URL forms when looking for a page’s mirror (check P15).

How is llms.txt different from robots.txt and sitemap.xml?

They solve different problems and you should ship all three. robots.txt and sitemap.xml are for crawl control and indexing; llms.txt is for comprehension.

	llms.txt	robots.txt	sitemap.xml
Purpose	Curated content map	Crawl permissions	URL inventory
Written for	LLMs loading context	Crawlers	Search indexers
Format	Markdown	Plain-text rules	XML
Curated?	Yes — hand-picked	No	No — every URL

For the full breakdown of the two that get confused most often, see llms.txt vs sitemap.xml.

Do AI agents actually read llms.txt?

Honestly: support is growing but not universal. Tools built on retrieval — coding assistants, documentation agents, and some AI search products — increasingly look for /llms.txt to load curated context, while the major chat models don’t all fetch it automatically yet. Treat it as low-cost insurance: a small file that helps the agents that do read it and costs nothing for those that don’t. Skipping it is one of the most common readability mistakes.

How do I add and validate an llms.txt?

Create the Markdown file, serve it as text/plain, and link the pages an agent should read. The step-by-step guide walks through doing it on a real site. Then run it through the llms.txt checker to catch a missing H1, no summary blockquote, the wrong content-type, or a missing llms-full.txt companion — 10 checks against the llmstxt.org spec.

Frequently asked questions

What is llms.txt?: llms.txt is a plain-text Markdown file at the root of a site (/llms.txt) that gives AI agents and large language models a curated, machine-readable map of your most important content. It's an H1 with your site name, a one-line summary, and H2 sections listing links to your key pages — so a model can find and load the content that matters instead of crawling and guessing. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at llmstxt.org; the spec was revised as v2 in August 2026.
What goes in an llms.txt file?: A single H1 with your project or site name, a blockquote summary describing what the site is, then H2 sections (e.g. 'Docs', 'API', 'Optional') each containing a bulleted list of links. Each link points to a page — ideally its Markdown version — with a short description after a colon. An 'Optional' section is the conventional home for secondary links (in v1 it carried special skip-this semantics for context-expansion tools; v2 dropped that machinery). That's the whole format: it's deliberately simple so any model can parse it.
Is llms.txt the same as robots.txt or sitemap.xml?: No. robots.txt tells crawlers what they may and may not access; sitemap.xml lists every URL for search-engine indexing. llms.txt does neither — it's a curated, human-written guide to your best content, written for an LLM that needs to load context, not a crawler that needs to enumerate URLs. The three are complementary: ship all of them. robots.txt and sitemap.xml are for discovery and crawl control; llms.txt is for comprehension.
Do AI agents actually read llms.txt?: Support is growing but not universal. Tools that build on retrieval — coding assistants, documentation agents, and some AI search products — increasingly look for /llms.txt to load curated context. The major chat models don't all fetch it automatically yet. Treat llms.txt as low-cost insurance: it's a small file that helps the agents that do read it and costs nothing for those that don't, and adoption is trending up as the convention spreads.
What changed in llms.txt v2?: The v2 revision (August 10, 2026) added standard link relations for discovery: rel="alternate" type="text/markdown" points from a page to its Markdown version, and rel="describedby" points to the llms.txt file that covers it — declared as HTML <link> elements or an HTTP Link response header. v2 also blesses both Markdown URL forms (page.html.md appended, or page.md with the extension replaced), defines subpath files (/docs/llms.txt covers everything under /docs/, most specific file wins), and drops the context-expansion tooling — the Optional section remains as a convention for secondary links but no longer carries mechanical semantics.
How do I add and validate an llms.txt?: Create a Markdown file at /llms.txt following the format above, serve it as text/plain, and link the key pages an agent should read. Then run it through a validator to catch structural issues — a missing H1, no summary blockquote, the wrong content-type, or a missing llms-full.txt companion. The Agent Ready llms.txt checker runs 10 checks against the llmstxt.org spec, including ones most validators skip.