GEO guides

llms.txt explained: what it is, how to write one and what it will not do

llms.txt is one of the most talked about files in AI search right now. It is useful in the right situations, but it is often oversold. Here is what you actually need to know.

By Geolytic team · 2 November 2026 · 5 min read

What is llms.txt?

llms.txt is a plain text file written in Markdown that sits at the root of a website, for example at yoursite.com/llms.txt. Its job is to give large language models a short, tidy map of the most useful content on your site. It was proposed in 2024 by Jeremy Howard of Answer.AI as a simple convention, not as an official web standard.

The idea is easy to understand. Normal web pages are full of menus, scripts and design elements. A model with limited context has to dig through all of that to find the content. A clean summary with links to the best pages saves it the trouble.

How it differs from robots.txt and sitemap.xml

robots.txt tells crawlers where they may and may not go. It is about permission.
sitemap.xml lists the URLs you want search engines to discover. It is about coverage.
llms.txt describes your site and points to the most helpful pages in readable form. It is about context.

These files do different jobs. llms.txt does not replace either of the others and it does not grant or remove crawler access. If you want to control AI crawlers, that still happens in robots.txt. Our AI crawler guide explains how.

Does any AI company actually use it?

This is the honest part that many articles skip. As of today no major AI provider has publicly committed to reading llms.txt when it builds answers. Some developer tools and documentation platforms support it and some AI coding assistants can use it when a person points them at a site. That is real value, but it is not the same as a guaranteed boost in ChatGPT or Google AI Overviews.

So treat llms.txt as low cost housekeeping. It takes an hour to write, it does no harm and it may help in tools that choose to read it. Do not treat it as a replacement for crawlable pages, clear content and good structured data.

What goes inside a good llms.txt

The proposed format is short and predictable. It starts with the name of your site as a heading, follows with a one paragraph summary and then lists sections of links with a short note for each link.

A top heading with your brand or site name.
A short summary in a quote block describing what the site or business does.
Optional plain paragraphs with important context, such as who you serve or where you operate.
Sections with headings like Docs, Products or Guides, each containing a list of links with one line descriptions.
An optional section for secondary links that a model can skip when it is short on space.

Writing tips that make it useful

Keep it factual

Describe what you do in the same words a customer would use. Avoid slogans. If a sentence would sound odd read aloud by a neutral narrator, rewrite it.

Link to your best pages, not every page

This is a curated list. Pick the pages that answer the most common questions about your business: what you offer, pricing, how it works, support and key guides. Your sitemap already covers everything else.

Match what is on the site

Every claim in llms.txt should also appear on a public page. If the file says you offer something your website never mentions, a model has no way to confirm it and a human reviewer will lose trust.

Keep it current

Outdated prices or retired products in llms.txt are worse than having no file. Add it to the same checklist you use when you update your pricing or services pages.

Common mistakes

Stuffing the file with keywords. Models are not impressed by repetition.
Copying the whole website into one giant file. That defeats the purpose of a summary.
Blocking AI crawlers in robots.txt while publishing llms.txt. If crawlers cannot fetch your pages, the links inside the file lead nowhere for them.
Expecting instant results. There is no confirmed ranking effect to wait for.

Should agencies sell llms.txt?

Some agencies now sell llms.txt files as a standalone product. We would not recommend building an offer around it. The file is quick to write and the benefit is unconfirmed for major AI search tools, so charging much for it on its own is hard to justify. It fits better as one small task inside a wider GEO sprint.

Where it does belong in your process

Finish the proven basics first: crawler access, technical health, structured data and clear copy.
Write llms.txt at the end, when the client's key pages are stable and well written.
Link only to pages that already answer customer questions clearly.
Add a review of the file to your monthly checklist so it never goes out of date.

How to talk about it with clients

Clients may arrive asking for llms.txt because they read about it online. Take the question seriously. Explain what the file does, who proposed it and that no major AI provider has publicly committed to reading it. Then offer to include it as part of a broader readiness plan. You keep credibility and you redirect the budget toward work with clearer value.

Spotting it in a prospect audit

A missing llms.txt is a minor finding, not a headline. When you review a prospect, lead with issues that clearly affect visibility, such as blocked AI search crawlers or pages without titles. Mention llms.txt as a later improvement. The free Pitch Audit flags it alongside the more important checks so you can choose what to emphasise.

A short template to reuse

Keep a simple internal template: site name as the heading, a two sentence summary, then sections for services, pricing, about and contact, each with three to six links. Filling it in for a new client should take under an hour once their pages are clean.

Should your site have one?

If you run documentation, a software product or a content rich site, yes, it is a reasonable small step. If you run a local business with five pages, your time is better spent making those five pages clear and making sure crawlers can read them. Either way, llms.txt is the last step of GEO, not the first.

Start with the basics that are proven to matter. Read our introduction to generative engine optimization, then run a free GEO audit to check crawler access, structured data and content clarity on your real pages.