--- title: State of Agent Readability 2026 description: "The 2026 edition: a frozen snapshot of agent readability across 523 sites — llms.txt and AGENTS.md adoption, top failures, protocol uptake, and AI-crawler blocking. A curated corpus, not a web-wide census." last_updated: 2026-07-15 canonical_url: https://agent-ready.dev/state-of-agent-readability/2026 --- # State of Agent Readability 2026 > The 2026 edition — a frozen, point-in-time snapshot of 523 distinct sites scanned on Agent Ready. Captured 2026-07-15. Immutable. **This is a curated corpus of 523 sites, not a web-wide census.** Every site here was scanned because someone chose to check it on Agent Ready, which skews toward owners who already care about AI agents. Adoption rates below run ahead of the web at large — read them as a trend signal among sites being actively checked, not as a share of all websites. These figures are frozen and do not change; the live edition at tracks the corpus as it grows. ## How agent-ready is the average site? Across 523 distinct sites, the mean agent-readability score is **48/100** and the median is **52/100**. Almost half of all sites fall into the Needs improvement band. | Rating band | Sites | Share | | --- | --- | --- | | Excellent (90–100) | 1 | 0% | | Good (70–89) | 31 | 6% | | Fair (50–69) | 258 | 49% | | Needs improvement (0–49) | 233 | 45% | ## How many sites publish an llms.txt or AGENTS.md? 46% of sites publish a valid llms.txt, 26% ship the llms-full.txt companion, and 26% publish an AGENTS.md. The mean llms.txt sub-score is 37/100. | Discovery file | Sites | Adoption | | --- | --- | --- | | Publish a valid llms.txt | 241 | 46% | | Publish llms-full.txt | 135 | 26% | | Publish AGENTS.md | 134 | 26% | ## What are the most common agent-readability failures? Site-level and llms.txt checks ranked by fail rate among the sites each check ran on. | Check | Fail rate | Sites checked | | --- | --- | --- | | S13 — AGENTS.md has required sections | 96% | 523 | | S11 — sitemap.md has headings + links | 95% | 523 | | S10 — sitemap.md exists | 80% | 523 | | S12 — AGENTS.md exists | 74% | 523 | | L10 — llms-full.txt available | 74% | 523 | | L4 — Blockquote summary | 73% | 523 | | L5 — H2 file-list sections | 71% | 523 | | L6 — Link format correct | 70% | 523 | | S2 — llms.txt Content-Type | 67% | 523 | | S9 — sitemap.xml has lastmod | 67% | 523 | ## Which agent protocols are sites actually adopting? Agent-protocol manifests are conditional — a site only "has" one if it advertises the matching well-known endpoint. Adoption is early and counts are small, so figures lead with the raw number of sites. | Protocol surface | Sites | Share | | --- | --- | --- | | A2A agent card | 28 | 5% | | MCP server card | 22 | 4% | | API catalog | 17 | 3% | | Agent Skills discovery | 9 | 2% | | agent-permissions.json | 6 | 1% | | Web Bot Auth | 6 | 1% | | NLWeb endpoint | 5 | 1% | | agents.json † | 4 | 1% | | UCP profile † | 1 | 0% | _† Fewer than 5 sites — indicative only, not a robust rate._ ## How many sites block AI crawlers? 11% of sites in this corpus block at least one major AI crawler in robots.txt. | AI crawler | Sites blocking it | Share | | --- | --- | --- | | GPTBot | 42 | 8% | | Google-Extended | 41 | 8% | | ClaudeBot | 38 | 7% | | Applebot-Extended | 36 | 7% | | PerplexityBot | 31 | 6% | | YouBot | 27 | 5% | | cohere-ai | 22 | 4% | | CCBot | 9 | 2% | | Meta-WebIndexer | 7 | 1% | | DuckAssistBot | 7 | 1% | ## How was this measured? Every figure is computed from 523 distinct sites scanned on agent-ready.dev between 2026-04-18 and 2026-07-15, captured as a frozen snapshot on 2026-07-15. To stop a single heavily-scanned site from skewing the corpus, only the most recent completed scan per domain is kept, and sites operated by Agent Ready are excluded so our own pages don't flatter the numbers. No individual site is named — the report is aggregate and anonymous by design. This is a sample of sites whose owners chose to check their agent readiness, so adoption rates run ahead of the web at large; read it as a trend signal, not a census. See the [methodology](https://agent-ready.dev/methodology) and [specs](https://agent-ready.dev/specs) for the full check registry and scoring rules. ## Frequently asked questions ### How big is the sample and when was it taken? This edition is a frozen snapshot of 523 distinct sites, captured on 2026-07-15 from every public scan run on agent-ready.dev between 2026-04-18 and 2026-07-15. One scan per domain is kept — the most recent completed scan — so a single heavily-scanned site cannot skew the numbers. No individual domain is named. ### Is this a representative sample of the whole web? No. It is a curated corpus, not a census. These are sites whose owners chose to scan them with Agent Ready, which skews toward owners who already care about AI agents — so adoption rates here run ahead of the web at large. Read every figure as 'among sites being actively checked for agent readiness', not as a web-wide rate. ### Why is there a frozen 2026 edition separate from the live page? The live State of Agent Readability page recomputes from the corpus every day, so its numbers drift as sites are scanned and improve. That makes it a poor citation target — the figure you quote today is different next week. This edition freezes one point in time so a reference to it stays true. When the corpus has moved enough to be worth re-reporting, we publish a new dated edition rather than editing this one. ### How is the agent-readability score calculated? Each scan runs dozens of checks across four layers — site-level discovery files, per-page extraction signals, the llms.txt standard, and agent-protocol manifests — and the score is the share of applicable checks that pass, expressed 0–100. The full formula, rating bands, and per-check weighting are on the methodology page. --- Read the full 2026 edition on the web: See the live edition (recomputed daily): Scan your site: ## Sitemap See the full [sitemap](https://agent-ready.dev/sitemap.md) for all pages on agent-ready.dev.