Few SEO topics have spread faster than llms.txt. Proposed in September 2024 by Jeremy Howard of Answer.AI and documented at llmstxt.org, it is a plain Markdown file placed at your site's root — yoursite.com/llms.txt — that gives large language models a clean, curated map of your content. Within months, thousands of sites added one, tool vendors shipped generators, and "do you have an llms.txt?" became a checklist item. But does it actually do anything? This guide explains what the file is, how to create one properly, and what the crawler data honestly shows — including the number most llms.txt advocates never mention.

What llms.txt Actually Is

llms.txt is a voluntary proposal, not a web standard enforced by anyone. The idea: HTML pages are messy for machines — navigation, ads, scripts, popups. A Markdown file is clean. So the proposal suggests maintaining a file that lists your most important pages in a structured way, written for an AI reader rather than a browser. It has two relatives in the same proposal:

• /llms.txt: A concise index — your site name, a short summary, and categorized links to key pages with brief descriptions.
• /llms-full.txt: An optional expanded version containing the full text content of your important pages in one file.
• Markdown versions of pages: The proposal also suggests serving .md versions of individual pages for models that prefer them.

What llms.txt is not: it is not robots.txt (it blocks nothing and grants no permissions), it is not a sitemap (it is curated and annotated, not exhaustive), and — critically — no major AI company has committed to using it as a ranking or citation input. Google has publicly stated it does not use llms.txt for AI Overviews or Search. That does not make the file useless, but it frames what it realistically is today: an inexpensive experiment, not a lever.

Adoption was fastest where you would expect: developer documentation platforms and AI companies themselves were early implementers, because API references and product docs are exactly the kind of structured, factual content the format was designed to summarize. From there it spread to SaaS marketing sites and then, inevitably, to SEO checklists — which is how a niche documentation convention became something every site owner suddenly felt behind for not having. That adoption curve matters for interpreting the data in the next sections: the file is common enough now that crawler behavior toward it is measurable, not theoretical.

What an llms.txt File Looks Like

The format is deliberately simple Markdown:

• An H1 with your site or project name — the only strictly required element.
• A blockquote with a one or two sentence summary of what the site is.
• Optional paragraphs with extra context a model should know.
• H2 sections grouping links by category, each link in the format: - [Page title](URL): short description of what the page contains.
• An optional "Optional" section — links in this section are explicitly marked as skippable if the model needs a shorter context.

A minimal real-world example for a small business site would have an H1 like "Acme Plumbing", a summary line, then sections such as "Services", "Guides", and "Company" with 5-20 curated links. That is the whole format. If you can write Markdown, you can write this file in fifteen minutes.

How to Create One (Step by Step)

Step 1: Pick your pages

Choose the 10-30 pages that best represent what you do: your homepage, core service or product pages, your best guides, pricing, and about page. Curation is the point — dumping all 400 URLs defeats the purpose.

Step 2: Write it in Markdown

Follow the structure above. Write descriptions that state what the page delivers ("Pricing for all plans, from $9/mo, with a feature comparison table") rather than marketing copy.

Step 3: Publish at the root

Upload the file so it resolves at yourdomain.com/llms.txt and returns a 200 status as plain text. On most platforms this is a static file upload; on WordPress, several SEO plugins now generate one automatically.

Step 4: Keep it current

An llms.txt pointing to deleted pages or last year's pricing is worse than none. Review it whenever your site structure changes.

The Data Nobody Shows You

Here is the part of the conversation that gets skipped. When researchers analyzed server logs to see whether AI crawlers actually request llms.txt files, the answer was sobering: an Ahrefs study examining sites that had implemented the file found that roughly 97% of llms.txt files received zero requests from AI crawlers. The models, overwhelmingly, are not reading the file. They crawl the same HTML web everyone else does, they train on their own schedules, and retrieval systems like AI Overviews and ChatGPT search pull from their own indexes — not from a Markdown file you volunteered.

Read that correctly. The study does not prove llms.txt can never matter. Adoption by AI companies could change, and a few smaller AI tools do read it. The honest verdict for 2026:

• Cost: About 15 minutes, once. Nearly zero.
• Downside: None, if the file is accurate.
• Proven upside: None for rankings or AI citations today.
• Possible upside: Future-proofing if AI platforms adopt the standard, and a small chance some current AI tools use it.

Add the file if it takes you fifteen minutes. Just do not let anyone sell it to you as an AI-visibility strategy, and do not prioritize it over the work that demonstrably drives citations.

What Actually Gets You Cited in AI Answers

AI answers are assembled from sources the model already trusts and can retrieve. The inputs that matter are unglamorous and familiar:

• Pages that rank: Retrieval systems heavily reuse top organic results. If your page ranks on page one for a question, it is a candidate for citation; if it is on page five, it is not.
• Clear, extractable facts: Direct answers near the top of the page, specific numbers, definitions, and structured formatting that a model can lift cleanly.
• Brand mentions across the web: Models learn which brands are associated with a topic from the wider corpus — reviews, comparisons, forums, and third-party articles.
• Accuracy and consistency: Your prices, features, and claims should match everywhere they appear. Contradictions make you a risky citation.

In other words: classic SEO, done well, is currently the AI-visibility strategy. The measurement side is where things genuinely are new — you need to know whether AI systems mention and cite you at all. AutoRank Enterprise tracks exactly that: GEO Tracking monitors your brand's presence across AI search experiences, and the AI Overview Optimizer scores your pages against the sources Google's AI Overviews currently cite, so you optimize against real citation data instead of guesses about crawler behavior.

Frequently Asked Questions

Q: Does Google use llms.txt?

Google has stated publicly that it does not use llms.txt for AI Overviews or Search ranking. Treat any claim that the file boosts Google AI visibility as unproven.

Q: Should I create an llms-full.txt too?

Only if it stays accurate automatically. A stale full-text dump with old pricing can actively mislead any system that reads it. For most small sites, the index file alone is enough.

Q: Will adding llms.txt hurt my SEO?

No. It is an inert text file. It cannot hurt rankings — it simply, on current evidence, does not help them either.

---

Skip the hype and measure what matters: see how your brand shows up in AI search with GEO Tracking →