--- title: "Can an AI Agent Use Your Website? 6 Questions to Ask" description: "Six questions that decide whether AI agents can use your website — discovery, documentation, authentication, APIs, standards, and what to fix first. Each maps to concrete, checkable artifacts: llms.txt, Markdown mirrors, structured data, AGENTS.md, OpenAPI, agents.json, API catalogs, MCP server cards, and A2A agent cards." last_updated: 2026-08-04 canonical_url: https://agent-ready.dev/can-an-ai-agent-use-your-website --- # Can an AI agent use your website? > AI visibility tools ask whether AI *mentions* your brand. Agent readiness asks whether an autonomous agent can *use* your site — and that decomposes into six testable questions, each with a concrete artifact behind it. ## Why these six questions? An agent that can't discover you, can't parse your docs, can't work out how to authenticate, and can't call your API will route around you — no matter how often the underlying model mentions your name. Agent readiness is an engineering property, not a vibe: each question below names the files and endpoints involved, and the [scanner methodology](https://agent-ready.dev/methodology) turns them into deterministic pass/fail checks. This page is the diagnostic pass; for the depth-first treatment, read the [complete guide to agent readability](https://agent-ready.dev/complete-guide-to-agent-readability), and for the implementation sequence, the [8-step checklist](https://agent-ready.dev/how-to-make-your-site-ai-agent-friendly). ## 1. Can an AI agent discover my site? Only if you publish the artifacts agents actually look for. Agents and AI crawlers probe a handful of well-known locations before reading any HTML: robots.txt for crawl policy, llms.txt at the site root for a curated content map, sitemap.xml (and its Markdown twin sitemap.md), and protocol manifests under /.well-known/ such as an MCP server card or an A2A agent card. If none of those exist, an agent falls back to scraping your rendered HTML — the slowest and least reliable path. Agent Ready's site-family checks probe each location and report exactly which discovery artifacts are missing. Generators: [llms.txt generator](https://agent-ready.dev/llms-txt-generator), [AI robots.txt generator](https://agent-ready.dev/ai-robots-txt-generator). ## 2. Can an AI agent understand my documentation? Agents parse Markdown far more reliably than JavaScript-rendered HTML. The load-bearing artifacts are a curated llms.txt (with llms-full.txt for full content), Markdown mirrors of key pages served at .md URLs and via Accept: text/markdown content negotiation, JSON-LD structured data naming the things on each page, and an AGENTS.md file giving coding agents context on how to use your product. Semantic HTML matters too: agents that drive a browser act through the accessibility tree, so unlabeled controls and div-soup lock them out. See: [What is llms.txt?](https://agent-ready.dev/what-is-llms-txt), [What is AGENTS.md?](https://agent-ready.dev/what-is-agents-md). ## 3. Can an AI agent authenticate correctly? Answer it in both directions. For agents calling your APIs, publish machine-readable auth: your OpenAPI spec's securitySchemes should state exactly which header or flow a caller needs, and an agent-permissions.json can declare what agents may and may not do on your site. For agents visiting your pages, Web Bot Auth (HTTP Message Signatures, RFC 9421) lets an agent prove its identity with signed requests that you — or your CDN — verify. An ordinary website publishes nothing for Web Bot Auth; it simply gains the ability to tell real agents from spoofed user-agent strings. See: [What is agent-permissions.json?](https://agent-ready.dev/what-is-agent-permissions-json), [What is Web Bot Auth?](https://agent-ready.dev/what-is-web-bot-auth). ## 4. Can an AI agent invoke my APIs? Only if the API surface is published somewhere an agent can find and parse. The layers, from simplest to richest: an OpenAPI spec linked from a discoverable location; an RFC 9727 API catalog — a linkset at /.well-known/api-catalog pointing at every API you offer; an agents.json manifest describing multi-step flows over your OpenAPI operations; an MCP server exposing your capabilities as tools; and an A2A endpoint for agent-to-agent calls. Each layer is a concrete artifact Agent Ready validates, and generators exist for most of them, so they can be valid by construction. Tools: [agents.json generator](https://agent-ready.dev/agents-json-generator), [API catalog generator](https://agent-ready.dev/api-catalog-generator), [A2A agent card generator](https://agent-ready.dev/agent-card-generator), [MCP server card validator](https://agent-ready.dev/mcp-card-validator). ## 5. Am I implementing emerging agent standards? Most sites aren't — State of Agent Readability data shows the large majority of scanned sites publish no agent-protocol manifest at all, which makes the standards a first-mover opportunity rather than table stakes. Agent Ready tracks the moving landscape (MCP, A2A, agents.json, agent-permissions.json, UCP, x402, NLWeb, API Catalog, Web Bot Auth, A2UI, MPP, AP2, ACP) as a registry of 69 deterministic checks, each mapped to the specification it validates. Grading is discover-then-validate: a manifest you don't publish never counts against you; one you do publish is graded on correctness. See: [the spec registry](https://agent-ready.dev/specs), [State of Agent Readability](https://agent-ready.dev/state-of-agent-readability). ## 6. What technical changes will improve agent interoperability? Run a scan and work the fix list — every failing check comes with a concrete remediation. The highest-leverage sequence for most sites: publish and curate llms.txt, ship Markdown mirrors with content negotiation, add JSON-LD structured data, and write an AGENTS.md. If you expose an API, add the protocol layer: OpenAPI first, then an API catalog, agents.json, or an MCP server card. Generators can produce spec-valid artifacts from your existing sitemap and manifests, so most fixes take minutes, not sprints. ## Frequently asked questions ### Is agent readiness the same as AEO or AI SEO? No. Answer-engine optimisation (AEO/GEO) asks whether AI assistants mention or cite your brand — a marketing measurement. Agent readiness asks whether an autonomous agent can technically discover, parse, authenticate against, and transact with your site — an engineering property you can test deterministically. They're complementary: agent-readable content is easier for answer engines to retrieve and quote, but the metrics, tools, and fixes are different. ### Do I need to implement every agent protocol to be agent-ready? No. Agent Ready grades discover-then-validate: optional manifests you haven't published are skipped, not penalised — only what you publish is graded on correctness. A content site can score well with the fundamentals alone: discovery files, Markdown mirrors, structured data, and clean semantics. Protocol manifests earn their keep when you have an API to call or transactions an agent could complete; add them then, ideally via a generator so they're valid from day one. --- Read the full guide on the web: Answer all six questions in one scan: ## Sitemap See the full [sitemap](https://agent-ready.dev/sitemap.md) for all pages on agent-ready.dev.