Dattva Blog · October 16, 2026
What Is llms.txt and Does Your Site Actually Need One
llms.txt is a plain-text file placed at a website's root that gives AI models a curated list of a site's most important pages with a short description of each. It is a guide for models that already have access, not an access control file like robots.txt, and most sites should add it only after the more foundational fixes are done.
Why This File Exists Separately From robots.txt
robots.txt tells a crawler what it is allowed to access. llms.txt tells a model, once it has access, which pages matter most and what each one is about. One is an access layer, the other is a curation layer.
The distinction matters because a large site can have hundreds of pages a crawler is technically allowed to read, with no signal about which handful represent the brand's most authoritative content. llms.txt exists to fill that gap. It gives a model a shortcut to the highest-value pages instead of asking it to crawl and weigh every page equally. The access side is covered in robots.txt for AI crawlers explained, and it has to be right before this file means anything.
What llms.txt Actually Contains
A typical file opens with a short plain-language description of the company, followed by a curated list of links under a few headings. Each link carries a one-line description of what that page covers.
# Dattva AI
> B2B AI visibility and Generative Engine Optimisation agency.
## Services
- [Generative Engine Optimisation](https://dattva.com/services/generative-engine-optimisation): Technical fixes, citation-native content, and authority building
- [AI Visibility Monitoring](https://dattva.com/services/ai-visibility-monitoring): Weekly tracking across ChatGPT, Perplexity, Gemini, and Claude
## Key guides
- [What is GEO](https://dattva.com/blog/what-is-generative-engine-optimization-geo): Definition and scope of GEOUnlike a sitemap, which lists every page mechanically, llms.txt is deliberately selective. The value comes from what you leave out. The format is simple Markdown-style text so a model can parse it quickly.
What the Evidence Actually Shows
llms.txt is a proposed standard, not a confirmed requirement of any major AI platform. I am not aware of any platform that has publicly confirmed it uses the file when deciding what to cite, so any claim that llms.txt "boosts citations" should be treated with suspicion, including from agencies, ours included.
The honest position from our side is narrower. When I implement it, it is almost always alongside other fixes, such as crawler access, schema, and restructured pages. That makes it hard to attribute any citation change to the file alone. I would rather say that plainly than publish a before-and-after number that the setup cannot support.
Compare that with the foundations. The same OtterlyAI analysis found that 73% of sites carry technical barriers blocking AI crawler access (OtterlyAI, February 2026). That is a far larger and better-evidenced gap than whether a site has an llms.txt file.
How to Build One Correctly
Place a plain-text file named llms.txt at the root domain, in the same location as robots.txt. Open with a concise factual description of the company, in plain language, with no marketing claims. List the most important pages under a small number of clear headings, and be genuinely selective, since a file that lists every page defeats its own purpose.
Pair each link with one line that states specifically what that page covers. Then maintain it. A stale llms.txt that points to renamed or removed pages is worse than having none, because it sends a model to dead ends. Put it on the same review schedule as robots.txt, after any site restructure.
Is This Worth Prioritising Right Now
For most B2B brands it is worth adding, because it takes under an hour and carries no real downside. It should not come before the better-evidenced fixes: confirming AI crawlers are not blocked, putting a direct answer in sentence one, and sourcing statistics.
A sensible order is: check crawler access first using a developer's checklist for AI crawler accessibility, fix structure and sourcing on your key pages, then add llms.txt as a low-effort extra. If you want to know where a site actually stands before deciding what to prioritise, Dattva's free diagnostic checks crawler access and citation performance across all four major platforms.
Conclusion
llms.txt gives AI models a curated shortcut to a site's most important content, which is a different job from robots.txt controlling access. It is a reasonable, low-effort addition, but the evidence for its effect on citations is thin, and it should come after the foundations. Treat it as housekeeping, not a growth lever.
Frequently Asked Questions
Is llms.txt the same thing as robots.txt?
No. robots.txt controls which crawlers can access a site at all, while llms.txt provides a curated guide to a site's most important content for models that already have access.
Do all AI platforms currently use llms.txt?
No platform has publicly confirmed that it uses the file when deciding what to cite, so its effect on citations is not established the way crawler access and content structure are.
How many pages should be listed in an llms.txt file?
A short, curated list of the most important pages, rather than every page on the site, since the value comes from prioritisation and not completeness.
Where should llms.txt be placed on a website?
At the root domain, in the same location as robots.txt, as a plain-text file named llms.txt.
Should a brand prioritise llms.txt over fixing crawler access issues?
No. Confirming AI crawlers are not blocked and structuring content properly should come first, since those have much clearer evidence behind their effect on AI visibility.
See where your brand stands in AI answers today
Run a free AI Visibility diagnostic across ChatGPT, Perplexity, Gemini, and Claude — with prioritised, copy-paste fixes at no cost.