llms.txt is a plain-text Markdown file at the root of a site (/llms.txt) that gives AI agents and large language models a curated, machine-readable map of your most important content. It's an H1 with your site name, a one-line summary, and H2 sections listing links to your key pages — so a model can find and load the content that matters instead of crawling and guessing. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at llmstxt.org.

Is llms.txt the same as robots.txt or sitemap.xml?

No. robots.txt tells crawlers what they may and may not access; sitemap.xml lists every URL for search-engine indexing. llms.txt does neither — it's a curated, human-written guide to your best content, written for an LLM that needs to load context, not a crawler that needs to enumerate URLs. The three are complementary: ship all of them. robots.txt and sitemap.xml are for discovery and crawl control; llms.txt is for comprehension.

What is llms.txt? A plain-English guide

llms.txt is a plain-text Markdown file served at the root of a site (/llms.txt) that gives AI agents and large language models a curated, machine-readable map of your most important content. Instead of leaving a model to crawl your HTML and guess what matters, you hand it an H1 with your site name, a one-line summary, and a set of H2 sections linking to the pages it should actually read. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at llmstxt.org.

What goes in an llms.txt file?

The format is deliberately minimal so any model can parse it: one H1 with your project name, a blockquote summary, then H2 sections — each a bulleted list of links with a short description after the colon. Point links at the Markdown version of each page where you have one. An Optional section marks links a model can skip when its context window is tight.

# Acme

> Acme is an API that turns messy addresses into clean, validated ones.

## Docs

- [Quickstart](https://acme.com/docs/quickstart.md): Make your first call in 5 minutes
- [Authentication](https://acme.com/docs/auth.md): API keys and OAuth
- [API reference](https://acme.com/docs/api.md): Every endpoint and parameter

## Optional

- [Changelog](https://acme.com/changelog.md): Recent releases
- [Pricing](https://acme.com/pricing.md): Plans and limits

Serve it as text/plain so agents (and the L9 content-type check) treat it as a text document, not HTML. For larger docs sets, pair it with an llms-full.txt companion that inlines the full content in a single fetch.

How is llms.txt different from robots.txt and sitemap.xml?

They solve different problems and you should ship all three. robots.txt and sitemap.xml are for crawl control and indexing; llms.txt is for comprehension.

	llms.txt	robots.txt	sitemap.xml
Purpose	Curated content map	Crawl permissions	URL inventory
Written for	LLMs loading context	Crawlers	Search indexers
Format	Markdown	Plain-text rules	XML
Curated?	Yes — hand-picked	No	No — every URL

For the full breakdown of the two that get confused most often, see llms.txt vs sitemap.xml.

Do AI agents actually read llms.txt?

Honestly: support is growing but not universal. Tools built on retrieval — coding assistants, documentation agents, and some AI search products — increasingly look for /llms.txt to load curated context, while the major chat models don’t all fetch it automatically yet. Treat it as low-cost insurance: a small file that helps the agents that do read it and costs nothing for those that don’t. Skipping it is one of the most common readability mistakes.

How do I add and validate an llms.txt?

Create the Markdown file, serve it as text/plain, and link the pages an agent should read. The step-by-step guide walks through doing it on a real site. Then run it through the llms.txt checker to catch a missing H1, no summary blockquote, the wrong content-type, or a missing llms-full.txt companion — 10 checks against the llmstxt.org spec.

Frequently asked questions

What is llms.txt?: llms.txt is a plain-text Markdown file at the root of a site (/llms.txt) that gives AI agents and large language models a curated, machine-readable map of your most important content. It's an H1 with your site name, a one-line summary, and H2 sections listing links to your key pages — so a model can find and load the content that matters instead of crawling and guessing. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at llmstxt.org.
What goes in an llms.txt file?: A single H1 with your project or site name, a blockquote summary describing what the site is, then H2 sections (e.g. 'Docs', 'API', 'Optional') each containing a bulleted list of links. Each link points to a page — ideally its Markdown version — with a short description after a colon. An 'Optional' section marks links a model can skip when context is tight. That's the whole format: it's deliberately simple so any model can parse it.
Is llms.txt the same as robots.txt or sitemap.xml?: No. robots.txt tells crawlers what they may and may not access; sitemap.xml lists every URL for search-engine indexing. llms.txt does neither — it's a curated, human-written guide to your best content, written for an LLM that needs to load context, not a crawler that needs to enumerate URLs. The three are complementary: ship all of them. robots.txt and sitemap.xml are for discovery and crawl control; llms.txt is for comprehension.
Do AI agents actually read llms.txt?: Support is growing but not universal. Tools that build on retrieval — coding assistants, documentation agents, and some AI search products — increasingly look for /llms.txt to load curated context. The major chat models don't all fetch it automatically yet. Treat llms.txt as low-cost insurance: it's a small file that helps the agents that do read it and costs nothing for those that don't, and adoption is trending up as the convention spreads.
How do I add and validate an llms.txt?: Create a Markdown file at /llms.txt following the format above, serve it as text/plain, and link the key pages an agent should read. Then run it through a validator to catch structural issues — a missing H1, no summary blockquote, the wrong content-type, or a missing llms-full.txt companion. The Agent Ready llms.txt checker runs 10 checks against the llmstxt.org spec, including ones most validators skip.