llms.txt is a proposed web convention: a markdown file, usually at your site’s root, that gives LLM agents a concise machine-readable index of your content — a brief background, guidance, and links to the detailed markdown versions of your pages. It is not a ranking lever and the spec makes no claim that it affects search rankings or citations in any AI product. Its stated purpose is narrower: to serve the agents that fetch it on demand, mainly at inference time, by giving them a cheaper path into your content than crawling and guessing.

What the file actually is

The specification lives at https://llmstxt.org/ — v2, modified August 10, 2026 (v1 was published September 3, 2024, authored by Jeremy Howard). The core format, which v2 did not change:

  • A file named llms.txt at your root or any subpath, written as markdown.
  • An H1 with your site name — the only required element in the entire spec.
  • Optionally, a blockquote with a short summary, then free-form markdown sections.
  • H2-delimited “file lists” where each entry is [name](url) with an optional : note — the actual index: what the page is, and where its machine-readable version lives.

v2 (August 2026) added four things, all about discoverability and consumption: standard link relations so an agent can find a page’s markdown version (rel="alternate" type="text/markdown") and the llms.txt covering it (rel="describedby", settable as an HTTP Link: header without touching any page); both URL forms for markdown alternates (.html.md appended or .md replacing the extension); subpath llms.txt files, where the most specific file wins; and an explicit consumption model — agents view or search the file, then follow the relevant links, so the file should stay small enough to fit in context.

Two things people get wrong, from the spec itself: llms-full.txt does not appear in the v2 spec (it is a v1-era convention many 2026 guides still list as a requirement — treat it as optional legacy practice), and the v1 context-expansion tool is gone in favor of the “view, then follow links” model. The full, check-by-check version of all of this — including the spec’s own recommended test of your file — is in our llms.txt v2 audit.

Who reads it

The readers are LLM agents that fetch the file on demand — coding agents, browsing agents, and the infrastructure behind AI products that look for it. The spec is explicit about the use case: “was that llms.txt would mainly be useful for inference rather than training, and that is how it has been used.” So the honest framing is:

  • It is a courtesy index. Agents that honor the file get a shorter, cheaper path into your content; agents that don’t read it are completely unaffected. Nothing breaks if you skip it — and nothing magically improves if you add it.
  • It is not a crawler. It does nothing for robots.txt, and it changes nothing about which bots can reach your site. The fetch path — who crawls you and what they see — is a separate layer, covered in how AI engines crawl your site.
  • Its stated use is inference, not training. The file exists to help agents find your content when they are answering a question or doing work, not to feed datasets.

What the spec page claims — and what it doesn’t

Marketing around llms.txt has run ahead of the spec. The adoption evidence below is what the live spec page (modified August 10, 2026) itself states; we verified it in our audit and cite it as the spec’s own claims rather than re-auditing each named site:

  • Scale: “thousands of sites publish an llms.txt file.”
  • Platform automation: “documentation platforms generate one automatically” — the spec’s integrations list names Mintlify, GitBook and Wix for hosted sites, plus Yoast SEO and AIOSEO for WordPress, and plugins for VitePress, Docusaurus and Drupal.
  • Tooling: “Chrome’s Lighthouse audits sites for one as part of its agentic browsing checks.”
  • Model providers: “The AI labs themselves publish llms.txt files for their own developer docs: OpenAI, Anthropic, and Gemini.”

What the spec claims nothing about: rankings, click-through, or citation in any specific AI product. The practical read for a B2B SaaS site: the file has moved from novelty to baseline infrastructure — and if your docs already run on one of the platforms above, you may already have one. Check before writing it by hand.

What it cannot do

Four limits, stated plainly:

  1. No citation guarantee. Nothing in the public data we work with says an llms.txt gets you cited in an answer to a money query. It makes your site cheap and reliable for the agents that fetch it — a lower bound, not a lever.
  2. It cannot create content. If the page a link points to is a JavaScript shell, paraphrased marketing, or behind a form, the index just points an agent at it faster. The content layer (extractability, entity clarity, corroboration) is where citations are actually won or lost.
  3. It is optional, and silence is free. A 404 on /llms.txt is not a fault. The failure mode is the inverse: a file that is a prose essay with no lists, or a stale monolith on a big site, which defeats the “small enough to fit in context” design.
  4. It is not robots.txt. It controls nothing about access or training; it only describes.

Whether you need one

Our take, as a judgment call rather than a spec requirement:

  • Add it if you have documentation, product or comparison content that agents plausibly fetch, and especially if your platform already generates it — in which case the real work is making sure the file lists point at clean markdown alternates. That is the v2 feature: the detail lives behind the links, fetched only when an agent needs it.
  • Defer it if you are a small marketing site with no docs and no machine-readable pages to point at. A file listing six HTML pages is ceremony. Get the fetch path right first (crawl mechanics) and the content layer before the index file.
  • Fix before adding: if you already have one, run the v2 audit — it is twenty minutes, and the spec’s own test (ask an agent a question with only your llms.txt as a starting point) is the pass/fail.

Where it sits in the AI-visibility stack

llms.txt is one check in a stack, not the strategy: crawlers must be able to reach your pages, the index file (if you have one) must make them cheap to navigate, the content must be extractable and bound to a clear entity, and third-party corroboration must exist for the claims to land. Every term in that paragraph is defined in the AEO and GEO glossary, and the whole series — definitions, infrastructure, content and corroboration — is indexed on the AEO/GEO basics hub page.