What is llms.txt, and why does it matter for GEO?
llms.txt is a markdown file at your site's root that tells AI crawlers what your site is about and which pages matter most. As more people discover businesses through ChatGPT, Claude, and Perplexity instead of Google, having a correctly formatted llms.txt improves the odds those tools describe and cite your site accurately — the same goal as classic SEO, aimed at a different reader.
Unlike robots.txt, which restricts crawler access, llms.txt is an invitation: it points AI systems to your best content instead of leaving them to guess. Getting the format wrong — missing the H1 title, skipping the summary blockquote, or malformed links — means crawlers may skip the file entirely, which is what the AI Readiness Score above is built to catch.
The standard was proposed by Jeremy Howard of Answer.AI in September 2024. It's still young, still evolving, and not every crawler honors it yet — but the sites adopting it early are the ones showing up when someone asks an AI assistant "who does X" instead of googling it.
llms.txt vs. robots.txt vs. sitemap.xml
Three files, three jobs. Mixing them up is the fastest way to publish an llms.txt that does nothing.
| File | Reader | What it does |
|---|
| robots.txt | Search + AI crawlers | Blocks or allows access to specific paths. Purely a gatekeeper — it says nothing about content. |
| sitemap.xml | Search engines | Lists every indexable URL for crawl discovery. Machine-readable, no context or prioritization. |
| llms.txt | LLMs and AI assistants | Summarizes the site in plain language and curates the pages worth reading first. Context, not just links. |
Anatomy of a spec-compliant file
Four parts, in this order. Skip or reorder them and your Readiness Score drops.
# Project Name
> One-sentence summary — this becomes the description most parsers surface first.
Optional paragraph with extra context: what the project does, who it's for, anything a one-line summary can't cover.
## Docs
- [Getting started](https://example.com/docs/start): setup guide
- [API reference](https://example.com/docs/api)
## Optional
- [Changelog](https://example.com/changelog): lower-priority, safe to skip under tight context budgets
6 mistakes that tank your AI Readiness Score
- 1No H1 title. The file has to open with "# Project Name" as the very first line. Anything before it — blank lines aside — breaks parsing.
- 2Skipping the summary blockquote. Most parsers treat the "> text" line right after the H1 as the canonical description. No blockquote, no description.
- 3Dumping every link in one section. Flat lists with no "## SectionName" headers give crawlers no signal about what's core vs. secondary.
- 4Broken markdown link syntax. Bare URLs or "[title] (url)" with a stray space aren't valid links — they read as plain text and get ignored.
- 5No Optional section. Changelogs, legal pages, and archives belong under "## Optional" so crawlers can skip them when context budgets are tight.
- 6Publishing llms-full.txt-sized content as llms.txt. The index file should be lean — links and one-line notes, not your entire site pasted in.
llms.txt by platform
The spec is platform-agnostic, but where the file lives and how it gets generated differs.
llms.txt vs. llms-full.txt — when to use which
llms.txt is an index: a title, a summary, and curated links. llms-full.txt is the extended version — the actual markdown content of your key pages, concatenated into one file. Use llms.txt as your default; only add llms-full.txt if you have documentation or reference content dense enough that an AI assistant would benefit from reading it in full, and check its token count against a model's context window before publishing it.
How do I generate an llms.txt file?
Use the Generate tab in the tool above: enter your project name, a one-line summary, and group your key links into sections like Docs or Optional. It builds a spec-compliant file instantly and lets you copy or download it — no signup or API required.
Is this llms.txt generator free?
Yes. It runs entirely in your browser with no backend, no API calls, and no usage limits. Nothing you type is uploaded or stored anywhere.
What's the difference between llms.txt and robots.txt?
robots.txt restricts which pages crawlers can access. llms.txt does the opposite — it points AI crawlers toward your most important content and summarizes the site, improving how accurately an LLM can describe or cite it.
What is llms-full.txt?
An extended version of llms.txt that includes full markdown content instead of just links. It's much larger, so check its token size against a model's context window before publishing it.
How is the AI Readiness Score calculated?
Six checks: a valid H1 title, a summary blockquote, correctly formatted H2 sections, valid markdown links, use of an Optional section for lower-priority links, and a total file size that fits comfortably in an LLM's context window.
Who created the llms.txt standard?
Jeremy Howard of Answer.AI proposed it in September 2024. It's an open, informal standard — there's no single governing body, and adoption varies by crawler.
Does llms.txt affect Google rankings?
No — it's not a confirmed Google ranking signal. It's built for AI assistants and LLM-based answer engines, not traditional search crawlers. Treat it as a GEO/AEO tactic, separate from your SEO work.