--- title: "What is llms.txt? A plain-English guide" description: What llms.txt is, what goes in the file, and why AI agents read it — with a minimal working example, how it differs from sitemap.xml and robots.txt, and how to validate yours. last_updated: 2026-06-20 canonical_url: https://agent-ready.dev/what-is-llms-txt --- # What is llms.txt? A plain-English guide > llms.txt is a plain-text Markdown file at the root of a site that gives AI agents a curated, machine-readable map of your most important content. **llms.txt** is a plain-text Markdown file served at the root of a site (`/llms.txt`) that gives AI agents and large language models a curated, machine-readable map of your most important content. Instead of leaving a model to crawl your HTML and guess what matters, you hand it an H1 with your site name, a one-line summary, and H2 sections linking to the pages it should actually read. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at [llmstxt.org](https://llmstxt.org). ## What goes in an llms.txt file? One H1 with your project name, a blockquote summary, then H2 sections — each a bulleted list of links with a short description after the colon. Point links at the Markdown version of each page where you have one. An `Optional` section marks links a model can skip when its context window is tight. ```markdown # Acme > Acme is an API that turns messy addresses into clean, validated ones. ## Docs - [Quickstart](https://acme.com/docs/quickstart.md): Make your first call in 5 minutes - [Authentication](https://acme.com/docs/auth.md): API keys and OAuth - [API reference](https://acme.com/docs/api.md): Every endpoint and parameter ## Optional - [Changelog](https://acme.com/changelog.md): Recent releases - [Pricing](https://acme.com/pricing.md): Plans and limits ``` Serve it as `text/plain` so agents (and the L9 content-type check) treat it as a text document, not HTML. For larger docs sets, pair it with an [llms-full.txt](https://agent-ready.dev/what-is-llms-full-txt) companion that inlines the full content in a single fetch. ## How is llms.txt different from robots.txt and sitemap.xml? They solve different problems and you should ship all three. robots.txt and sitemap.xml are for crawl control and indexing; llms.txt is for comprehension. | | llms.txt | robots.txt | sitemap.xml | | --- | --- | --- | --- | | Purpose | Curated content map | Crawl permissions | URL inventory | | Written for | LLMs loading context | Crawlers | Search indexers | | Format | Markdown | Plain-text rules | XML | | Curated? | Yes — hand-picked | No | No — every URL | For the full breakdown of the two that get confused most often, see [llms.txt vs sitemap.xml](https://agent-ready.dev/llms-txt-vs-sitemap-xml). ## Do AI agents actually read llms.txt? Support is growing but not universal. Tools built on retrieval — coding assistants, documentation agents, and some AI search products — increasingly look for `/llms.txt` to load curated context, while the major chat models don't all fetch it automatically yet. Treat it as low-cost insurance: a small file that helps the agents that do read it and costs nothing for those that don't. Skipping it is one of the most [common readability mistakes](https://agent-ready.dev/common-agent-readability-mistakes). ## How do I add and validate an llms.txt? Create the Markdown file, serve it as `text/plain`, and link the pages an agent should read. The [step-by-step guide](https://agent-ready.dev/how-to-add-llms-txt-to-nextjs) walks through doing it on a real site. Then run it through the [llms.txt checker](https://agent-ready.dev/llms-txt-checker) to catch a missing H1, no summary blockquote, the wrong content-type, or a missing llms-full.txt companion — 10 checks against the llmstxt.org spec. ## Frequently asked questions ### What is llms.txt? llms.txt is a plain-text Markdown file at the root of a site (/llms.txt) that gives AI agents and large language models a curated, machine-readable map of your most important content. It's an H1 with your site name, a one-line summary, and H2 sections listing links to your key pages — so a model can find and load the content that matters instead of crawling and guessing. It was proposed by Jeremy Howard of Answer.AI in 2024 and is specified at llmstxt.org. ### What goes in an llms.txt file? A single H1 with your project or site name, a blockquote summary describing what the site is, then H2 sections (e.g. 'Docs', 'API', 'Optional') each containing a bulleted list of links. Each link points to a page — ideally its Markdown version — with a short description after a colon. An 'Optional' section marks links a model can skip when context is tight. That's the whole format: it's deliberately simple so any model can parse it. ### Is llms.txt the same as robots.txt or sitemap.xml? No. robots.txt tells crawlers what they may and may not access; sitemap.xml lists every URL for search-engine indexing. llms.txt does neither — it's a curated, human-written guide to your best content, written for an LLM that needs to load context, not a crawler that needs to enumerate URLs. The three are complementary: ship all of them. robots.txt and sitemap.xml are for discovery and crawl control; llms.txt is for comprehension. ### Do AI agents actually read llms.txt? Support is growing but not universal. Tools that build on retrieval — coding assistants, documentation agents, and some AI search products — increasingly look for /llms.txt to load curated context. The major chat models don't all fetch it automatically yet. Treat llms.txt as low-cost insurance: it's a small file that helps the agents that do read it and costs nothing for those that don't, and adoption is trending up as the convention spreads. ### How do I add and validate an llms.txt? Create a Markdown file at /llms.txt following the format above, serve it as text/plain, and link the key pages an agent should read. Then run it through a validator to catch structural issues — a missing H1, no summary blockquote, the wrong content-type, or a missing llms-full.txt companion. The Agent Ready llms.txt checker runs 10 checks against the llmstxt.org spec, including ones most validators skip. --- Read the full guide on the web: Scan your site: ## Sitemap See the full [sitemap](https://agent-ready.dev/sitemap.md) for all pages on agent-ready.dev.