# llms.txt

llms.txt is a proposed standard for a Markdown file at the root of a website, at /llms.txt, that gives large language models a short summary of the site and links to the pages that matter most, in a form they can read easily.

Also called: llms-full.txt, LLMs.txt, llms.txt file, Markdown for LLMs.

## In short

- llms.txt is a **plain Markdown guide to your site**, written for language models instead of browsers.
- It has a title, a one-paragraph summary, and lists of links to the most useful pages.
- It is a community proposal from 2024, not a formal standard, and support varies by tool.
- It takes little effort to add, so treat it as a useful extra alongside good pages, not a substitute for them.

## Where llms.txt came from

Jeremy Howard of Answer.AI proposed llms.txt in September 2024. The problem it addresses is simple. Web pages are built for people in browsers, full of navigation, scripts, and layout. A language model reading a site has a limited context window and does better with clean text and a clear map of what is where.

The proposal suggests two things: a file at /llms.txt that summarizes the site and links to key content, and Markdown versions of important pages, available at the same address with .md added. Some sites also publish an llms-full.txt with all of their content in one file.

## What goes in the file

**Diagram: The structure of an llms.txt file.**

- Title (An H1 with the site or project name)
- Summary (A short blockquote saying what it is)
- Details (Optional paragraphs of key context)
- Link sections (H2 headings, each with a list of links)

_Only the title is required. The summary does the most work, because it is what a model reads first._

Each link can carry a short note saying what the page contains. A section titled Optional marks links that a tool can skip when it needs to save space. The format is ordinary Markdown, so the file is also readable by people.

Choie's own file follows this shape: a one-paragraph summary that includes its prices in US dollars, then link sections for services, answers, work, guides, and the glossary, with the sitemap and legal pages under Optional.

| File | Written for | What it does |
| --- | --- | --- |
| robots.txt | Crawlers | Says which parts of a site may be crawled |
| sitemap.xml | Search engines | Lists every URL with update dates |
| llms.txt | Language models and agents | Summarizes the site and points to the best content |

## Who uses it

Adoption has been strongest among documentation sites and developer tools, where programmers point AI coding assistants at a library's docs. In that setting, a clean Markdown summary saves tokens and improves answers.

Choie's builds use AI in this setting too. It drafts implementation from a decided design and proposes tests, and a senior person reads every line before it ships. See [AI-accelerated development](https://www.choie.co/guides/ai-accelerated-development).

For general AI search, the picture is less clear. It is a proposal rather than an adopted standard, and AI search products have not made clear commitments to read it. The main route into AI answers is still the same as in search: pages that are crawlable, clear, and useful. That is the work of [answer engine optimization](https://www.choie.co/glossary/answer-engine-optimization).

> **Cheap to add, easy to keep current.** If your site already builds its pages from structured content, an llms.txt can be generated from the same source, so it never drifts out of date.

## Should you add one?

For most product and company websites, yes, as long as it stays small. A useful file takes an hour or two to write. It should say in plain words what the company does, who it is for, and what it costs if prices are public, then link to the pages a model would need to answer questions about you.

- **Generate it from your content.** On a [custom website](https://www.choie.co/glossary/custom-website) or a site run from a [headless CMS](https://www.choie.co/glossary/headless-cms), build the file from the same data as the pages.
- **Keep facts identical.** The prices and claims in the file should match the site word for word.
- **Link to Markdown where you can.** Clean .md versions of key pages are easier for models to read than HTML.
- **Do not hide anything in it.** Content meant only for machines, or that differs from what people see, erodes trust.

Choie publishes both files. The llms.txt at choie.co links Markdown versions of the site's pages, and llms-full.txt holds the whole site, glossary included, in one document. Both are built from the same data as the pages, using [Next.js](https://www.choie.co/glossary/nextjs) routes.

## Common mistakes

- **Treating it as a ranking trick.** It will not make up for thin or unclear pages.
- **Listing everything.** A file with hundreds of links loses the point. Pick the pages that answer real questions.
- **Letting it go stale.** Old prices in llms.txt can end up repeated in AI answers.
- **Confusing it with robots.txt.** llms.txt does not grant or block access. Crawling rules stay in robots.txt.

## Common questions

### Where does the llms.txt file go?

At the root of the domain, so it can be found at yoursite.com/llms.txt, the same way robots.txt and sitemap.xml are found at fixed addresses.

### What is llms-full.txt?

A companion file some sites publish that contains the full text of the site or its documentation as one Markdown document, so a tool can load everything with a single request.

### Does Google use llms.txt?

Google has not said that its search or AI features rely on llms.txt. Its guidance for AI Overviews is the same as for regular search: crawlable, helpful, well-structured pages.

### Does llms.txt replace robots.txt?

No. robots.txt controls which parts of a site crawlers may access. llms.txt is a guide to the content and has no effect on access.

Related: [Answer engine optimization (AEO)](https://www.choie.co/glossary/answer-engine-optimization), [Custom website](https://www.choie.co/glossary/custom-website), [Headless CMS](https://www.choie.co/glossary/headless-cms), [Next.js](https://www.choie.co/glossary/nextjs), [AI-assisted development](https://www.choie.co/glossary/ai-assisted-development).

HTML: https://www.choie.co/glossary/llms-txt