This is a curated corpus of 523 sites, not a web-wide census. Every site here was scanned because someone chose to check it on Agent Ready, which skews toward owners who already care about AI agents. Adoption rates below run ahead of the web at large — read them as a trend signal among sites being actively checked, not as a share of all websites. The figures are frozen at 2026-07-15 and do not change; the live edition tracks the corpus as it grows.
How agent-ready is the average site?
Across 523 distinct sites, the mean agent-readability score is 48 out of 100 and the median is 52. Just 1 site scored in the Excellent band; almost half fall into Needs improvement. The score is the share of applicable checks a site passes; see the methodology for the formula and rating bands.
- Excellent0% · 1
- Good6% · 31
- Fair49% · 258
- Needs improvement45% · 233
| Rating band | Sites | Share |
|---|---|---|
| Excellent (90–100) | 1 | 0% |
| Good (70–89) | 31 | 6% |
| Fair (50–69) | 258 | 49% |
| Needs improvement (0–49) | 233 | 45% |
How many sites publish an llms.txt or AGENTS.md?
The discovery files are the cheapest agent-readiness win and the most-skipped. In this corpus, 46% of sites publish a valid llms.txt, 26% ship the llms-full.txt companion, and 26% publish an AGENTS.md. The mean llms.txt sub-score is 37/100.
| Discovery file | Sites | Adoption |
|---|---|---|
| Publish a valid llms.txt | 241 | 46% |
| Publish llms-full.txt | 135 | 26% |
| Publish AGENTS.md | 134 | 26% |
What are the most common agent-readability failures?
These are the site-level and llms.txt checks that fail most often, ranked by fail rate among the sites the check actually ran on. They are the highest-leverage fixes for a typical site.
| Check | Fail rate | Sites checked |
|---|---|---|
| S13AGENTS.md has required sections | 96% | 523 |
| S11sitemap.md has headings + links | 95% | 523 |
| S10sitemap.md exists | 80% | 523 |
| S12AGENTS.md exists | 74% | 523 |
| L10llms-full.txt available | 74% | 523 |
| L4Blockquote summary | 73% | 523 |
| L5H2 file-list sections | 71% | 523 |
| L6Link format correct | 70% | 523 |
| S2llms.txt Content-Type | 67% | 523 |
| S9sitemap.xml has lastmod | 67% | 523 |
Which agent protocols are sites actually adopting?
Agent-protocol manifests are conditional — a site only has one if it advertises the matching well-known endpoint. Adoption is still early, and the counts are small, so these figures lead with the raw number of sites.
| Protocol surface | Sites | Share |
|---|---|---|
| A2A agent card | 28 | 5% |
| MCP server card | 22 | 4% |
| API catalog | 17 | 3% |
| Agent Skills discovery | 9 | 2% |
| agent-permissions.json | 6 | 1% |
| Web Bot Auth | 6 | 1% |
| NLWeb endpoint | 5 | 1% |
| agents.json † | 4 | 1% |
| UCP profile † | 1 | 0% |
† Fewer than 5 sites — indicative only, not a robust rate.
How many sites block AI crawlers?
A robots.txt that disallows AI user-agents keeps a site out of AI search and assistants entirely. 11% of sites in this corpus block at least one major AI crawler. The most-blocked crawlers:
| AI crawler | Sites blocking it | Share |
|---|---|---|
| GPTBot | 42 | 8% |
| Google-Extended | 41 | 8% |
| ClaudeBot | 38 | 7% |
| Applebot-Extended | 36 | 7% |
| PerplexityBot | 31 | 6% |
| YouBot | 27 | 5% |
| cohere-ai | 22 | 4% |
| CCBot | 9 | 2% |
| Meta-WebIndexer | 7 | 1% |
| DuckAssistBot | 7 | 1% |
How was this measured?
Every figure on this page is computed from 523 distinct sites that were scanned on agent-ready.dev between 2026-04-18 and 2026-07-15, captured as a frozen snapshot on 2026-07-15. To stop a single heavily-scanned site from skewing the corpus, we keep only the most recent completed scan per domain, and we exclude sites operated by Agent Ready so our own pages do not flatter the numbers. No individual site is named — the report is aggregate and anonymous by design.
This is a sample of sites whose owners chose to check their agent readiness, so adoption rates here run ahead of the web at large. Read it as a trend signal, not a census. The full check registry and scoring rules live on the methodology and specs pages.
Frequently asked questions
- How big is the sample and when was it taken?
- This edition is a frozen snapshot of 523 distinct sites, captured on 2026-07-15 from every public scan run on agent-ready.dev between 2026-04-18 and 2026-07-15. One scan per domain is kept — the most recent completed scan — so a single heavily-scanned site cannot skew the numbers. No individual domain is named.
- Is this a representative sample of the whole web?
- No. It is a curated corpus, not a census. These are sites whose owners chose to scan them with Agent Ready, which skews toward owners who already care about AI agents — so adoption rates here run ahead of the web at large. Read every figure as 'among sites being actively checked for agent readiness', not as a web-wide rate.
- Why is there a frozen 2026 edition separate from the live page?
- The live State of Agent Readability page recomputes from the corpus every day, so its numbers drift as sites are scanned and improve. That makes it a poor citation target — the figure you quote today is different next week. This edition freezes one point in time so a reference to it stays true. When the corpus has moved enough to be worth re-reporting, we publish a new dated edition rather than editing this one.
- How is the agent-readability score calculated?
- Each scan runs dozens of checks across four layers — site-level discovery files, per-page extraction signals, the llms.txt standard, and agent-protocol manifests — and the score is the share of applicable checks that pass, expressed 0–100. The full formula, rating bands, and per-check weighting are on the methodology page.