llms.txt: what it is, who reads it and how to write one

llms.txt is a plain Markdown file at /llms.txt that gives AI assistants and agents a short, curated map of your site: what it is, and which pages to read for what. This guide is for site owners and developers who keep hearing about it and want a straight answer. By the end you will know what the file does and does not do, who reads it today, and you will have a valid llms.txt in place. It takes about 15 minutes.

Quick answer

• llms.txt is a proposal from llmstxt.org, first published in September 2024 and revised in August 2026. It is not a web standard and no search engine has committed to using it.
• Coding agents and documentation tools read it; Google has said Search does not use it. It does not change rankings, and it does not get you "indexed by an LLM".
• The format: an # H1 with the site name, a > blockquote summary, then ## Section headings with lists of - [Title](URL): what the page answers.
• Put it at the site root, https://example.com/llms.txt, served as text/plain or text/markdown, under 100 KB.
• It controls nothing. Which AI crawlers may fetch your pages is decided in robots.txt.
• Build a draft from your sitemap with the llms.txt generator, then check it with the AI crawler check.

What is llms.txt?

The llms.txt file is a Markdown index of a website written for language models rather than for people or search crawlers. Jeremy Howard of Answer.AI proposed it in September 2024 because web pages are hard for a model to read: navigation, cookie banners and scripts wrap the actual content, and a whole site does not fit into a model's context. A short file at a known address lets an agent find the few pages it needs and fetch only those.

Two things make it different from the files you already have:

• It is curated. Your sitemap lists every indexable URL, often thousands. llms.txt lists the 10 to 100 pages that best explain what you do, each with a sentence on what it answers.
• It is written for reading, not for rules. It holds no allow or deny rules, unlike robots.txt, and search engines do not read it for crawling, unlike a sitemap.

People search for both "llms.txt" and "llm.txt". The proposal's name is llms.txt, with an s; a file at /llm.txt is not what any tool looks for.

Does llms.txt help SEO or AI visibility?

No, not for search rankings. Google's Search team has said publicly that Google Search does not use llms.txt, and that AI Overviews and AI Mode are built from the normal search index, crawled by Googlebot. No other search engine has announced support. A file that no ranking system reads cannot move rankings.

For AI assistants the picture is "sometimes, when someone asks". No major assistant (ChatGPT, Claude, Gemini, Perplexity, Copilot) has said its crawlers look for llms.txt or treat it as a signal for what to cite. Most AI answers still come from pages fetched or indexed the ordinary way, which is why server-rendered text, clear titles and structured data matter far more.

So why write one? Because it costs 15 minutes and helps the tools that do use it: an agent that is pointed at your site finds the right pages faster, and for documentation it is already routine. Treat it as a cheap bet, not a ranking factor.

Which AI tools actually read llms.txt?

Honest status in September 2026:

You can check your own case: look for requests to /llms.txt in your server access log. On most small business sites they are rare, and many come from SEO tools checking whether the file exists.

What goes in an llms.txt file?

The llmstxt.org format is strict about order. In a valid file:

1. An H1 with the name of the site or project. This is the only required part.
2. A blockquote (> …) with a one- or two-sentence summary.
3. Optional paragraphs or lists with more context, but no headings.
4. Zero or more H2 sections, each holding a list of links: - [Name](URL), optionally followed by : notes.
5. By convention, a section called ## Optional for secondary links an agent can skip when it is short of space.

A complete file for a small online shop:

# Example Shop

> Example Shop sells handmade ceramic tableware from Zagreb and ships across the EU. Orders over €60 ship free; returns are accepted within 30 days.

Prices are in euros and include VAT. Stock levels change daily; the product pages are the source of truth.

## Products

- [Dinner plates](https://example-shop.hr/en/plates/): All plate sizes, glazes and set prices
- [Mugs and cups](https://example-shop.hr/en/mugs/): Capacities, dishwasher and microwave use
- [Gift sets](https://example-shop.hr/en/gift-sets/): Ready-made sets with gift wrapping

## Buying

- [Shipping](https://example-shop.hr/en/shipping/): Countries, delivery times and costs
- [Returns](https://example-shop.hr/en/returns/): 30-day returns and how refunds are paid
- [Care instructions](https://example-shop.hr/en/care/): Washing, chips and glaze questions

## Optional

- [About the studio](https://example-shop.hr/en/about/): Who makes the pieces and where
- [Privacy policy](https://example-shop.hr/en/privacy/)

And one for a service business, with the context paragraph doing more work:

# Harbour Physio

> Physiotherapy clinic in Split for sports injuries, back pain and post-surgery rehabilitation. Appointments on weekdays 8:00–20:00.

We do not need a doctor's referral. Private health insurance covers most treatments; check with your insurer first.

## Treatments

- [Sports injuries](https://harbour-physio.example/sports-injuries/): Conditions treated and a typical course of sessions
- [Back and neck pain](https://harbour-physio.example/back-pain/): Assessment, treatment and exercises
- [Prices](https://harbour-physio.example/prices/): Session prices and packages

## Visiting

- [Book an appointment](https://harbour-physio.example/book/): Online booking and cancellation rules
- [Location and parking](https://harbour-physio.example/contact/): Address, parking and public transport

Write the notes after each link as the question the page answers ("Countries, delivery times and costs"), not as marketing copy. Link to pages that work without JavaScript, because most AI fetchers do not run scripts.

Markdown versions of pages

The 2026 revision of the proposal also suggests offering a clean Markdown copy of important pages at the same URL with .md added (/shipping/index.md or /docs/page.html.md), and pointing to it with <link rel="alternate" type="text/markdown" href="…">. It also defines rel="describedby" for pointing a page at the llms.txt that covers it. Both are optional. For documentation sites they are worth it; for a shop or a clinic the HTML pages are fine.

What is llms-full.txt?

llms-full.txt is a companion convention: one Markdown file with the full text of the pages your llms.txt links to, so an agent can load everything in a single request. It grew up around documentation platforms rather than the llmstxt.org proposal itself, and the proposal does not define it.

It makes sense for documentation, API references and knowledge bases, where a developer wants a whole manual in one go. For a marketing site or a shop it is usually not worth maintaining, because it goes stale every time a page changes. If you publish one, generate it at build time from the same source as the pages, and keep llms.txt itself short: long content belongs in llms-full.txt, not in the index.

Where do I put llms.txt on my website?

At the root of the host, next to robots.txt:

https://example.com/llms.txt

The proposal also allows a file on a subpath that covers only the pages below it, such as https://example.com/docs/llms.txt for a documentation section; when more than one applies, the most specific wins. Most sites need just the root file.

The file must:

• answer 200 at exactly that URL, without redirecting to a different host;
• be served as text/plain or text/markdown, UTF-8; a route that returns your HTML 404 page with status 200 fails;
• stay under 100 KB; move long content to llms-full.txt;
• be reachable by the crawlers you want to read it. It is fetched like any other URL, so a Disallow: / for a crawler in robots.txt covers it too.

Each host is separate: shop.example.com needs its own file.

How do I create an llms.txt file (by hand, plugin or generator)?

With the generator

The generator reads your robots.txt and sitemaps (up to 500 pages), takes the titles and meta descriptions of the 40 most important ones, and groups them into sections by where they live: products, blog, docs, about. Carts, account pages, tag archives, feeds and pagination are dropped, and privacy and terms go to ## Optional. You then edit the draft in your browser: rename sections, reorder links, rewrite the notes. It is validated as you type with the same rules as the report check, and nothing you type is sent to us.

By hand

1. Write the H1 and a summary a stranger would understand: what you do, for whom, where.
2. Add one or two context lines an assistant would otherwise get wrong: prices include VAT, no referral needed, which regions you serve.
3. Pick 10–40 pages that answer the questions customers actually ask. Group them under 2–5 H2 sections.
4. Write a short note after each link: what the page answers.
5. Save as llms.txt (UTF-8, no BOM needed) and upload it to the web root.

With your CMS

• WordPress: Yoast SEO, AIOSEO and Rank Math can generate and update an llms.txt; check the file they produce, because automatic ones tend to list recent posts rather than your key pages. Or upload your own file to the folder that holds wp-config.php. A physical file wins over a plugin's virtual one on most servers, so do not use both.
• Wix generates one for every site; review it in the SEO settings.
• Shopify does not let you upload arbitrary files to the root. The usual workaround is a URL redirect from /llms.txt to a file uploaded under Content → Files, which lands on Shopify's CDN host. Tools that follow redirects can read it, but it is not a file at your root; whatever route you use, check what /llms.txt actually answers.
• Static sites and frameworks: put llms.txt in the folder copied to the web root (public/ in Next.js, Astro and Vite, static/ in Hugo and SvelteKit).
• Documentation platforms (Mintlify, GitBook, Docusaurus and VitePress plugins) generate llms.txt, and often llms-full.txt, from the docs.

llms.txt vs robots.txt: what is the difference?

They work together but never override each other. If robots.txt blocks GPTBot, GPTBot will not fetch your pages, whatever llms.txt lists. If you want AI assistants to be able to cite you, allow their crawlers in robots.txt first; our guide on AI crawlers and robots.txt goes through them one by one, and robots.txt before it hides your site covers the mistakes that block everyone.

"robots.txt for LLMs" is a common description of llms.txt, and a misleading one. There is no llms.txt syntax for "do not train on this"; that is robots.txt (User-agent: GPTBot / Disallow: /) or a noai directive.

How do I check a site's llms.txt?

The quickest way is to open https://thatsite.com/llms.txt in a browser. For a full check, including format and crawler access:

The format check applies the llmstxt.org rules: an H1 first, a summary, links grouped under H2 sections, a text content type and a sensible size. It also notes whether /llms-full.txt answers.

From a terminal:

curl -sI https://example.com/llms.txt | grep -iE "^HTTP|content-type"
curl -s https://example.com/llms.txt | head -20

Can AI crawlers read JavaScript pages?

Mostly not. A study by Vercel and MERJ of AI crawler traffic, published in December 2024, found that GPTBot, ClaudeBot and PerplexityBot download JavaScript files but do not run them; of the large AI crawlers, only Google's (which shares Googlebot's renderer) and Apple's render pages. If your prices, product descriptions or opening hours only appear after scripts run, most AI assistants never see them, and an llms.txt link to that page does not help.

The fix is server-side or static rendering of the content that matters; JavaScript-rendered content and Google explains the options.

Verify

• https://yoursite/llms.txt opens as plain text in the browser, not as a download and not as an HTML page.
• The AI crawler check reads "llms.txt follows the llmstxt.org format".
• Every link in the file answers 200; paste them into the bulk URL checker if there are many.
• Give an assistant only the llms.txt URL and ask it three questions a customer would ask. If it cannot find the answers, improve the notes or add the missing page.

Common mistakes

• Naming it llm.txt or putting it under /.well-known/. Tools look for /llms.txt at the root (or on a subpath it covers).
• Pasting the sitemap. A list of 2,000 URLs without notes is exactly what the file is meant to replace.
• Headings before the links start. Anything between the blockquote and the first H2 must be paragraphs or lists; an H3 there breaks the format.
• Serving HTML. A single-page app that answers every path with index.html returns a 200 HTML page for /llms.txt. Add a real file or route.
• Expecting rankings or traffic. It is an aid for agents, not a search signal. Spend the bigger effort on crawlable, server-rendered pages.
• Listing pages robots.txt blocks. The crawler still obeys robots.txt; the link is useless to it.
• Letting it go stale. Review it when you change prices, services or the site structure, the same way you would a sitemap.

Questions people ask

How do I add llms.txt to WordPress?

Upload the file to the root of your site, the folder that holds wp-config.php, with your host's file manager or SFTP, so it answers at https://yourdomain.com/llms.txt. Recent versions of Yoast SEO and Rank Math can also generate one for you. Then open the address in a browser and confirm it loads as plain text and is not redirected to a WordPress page.

Is it llm.txt or llms.txt?

It is llms.txt, with an s, at https://yoursite/llms.txt. That is the name in the llmstxt.org proposal and the only one tools and plugins look for. "llm.txt" is a common misspelling in searches. If you created a file called llm.txt, rename it; there is no need to keep both.

Where do I place the llms.txt file on my website?

Put it in the web root, next to robots.txt, so it answers at https://yoursite/llms.txt with status 200 and a text/plain or text/markdown content type. On WordPress that is the folder with wp-config.php; on static sites and frameworks, the public or static folder. Each subdomain needs its own file.

Can llms.txt get my site indexed by an LLM?

No. Language models are not indexes you submit to. Assistants find pages through their own crawlers, search indexes they license, or live fetches when a user asks. llms.txt only helps an agent that already reads your site to find the right pages. To be found, allow AI crawlers in robots.txt and make sure your content is in the HTML without JavaScript.

How do I see the llms.txt file of a website?

Open https://thatsite.com/llms.txt in your browser; if the site has one, it shows as plain text. A 404 means there is none. To check that it is also well-formed and that the site's robots.txt lets AI crawlers in, run the free AI crawler check, which validates the file against the llmstxt.org format.

Does ChatGPT read llms.txt?

There is no public confirmation that it does. None of the major AI companies has said its crawlers or assistants use llms.txt when answering questions, and server logs show only occasional requests for the file. Some coding tools and documentation assistants do read it, and it is cheap to publish, so treat it as a low-cost bet rather than a way to rank in AI answers.

Will an llms.txt file help my SEO?

No. Google has said Search does not use llms.txt, and no other search engine has announced support, so it does not affect rankings or AI Overviews. It can help coding agents and some AI tools find your key pages faster. At 15 minutes of work it is a reasonable extra, but it is not an SEO task.

What should I do with an llms.txt file?

Write it once, upload it to your web root and review it when your key pages change. Keep it to the pages that answer real customer questions, with a short note on each. Then leave it alone: there is nothing to submit and no console to register it in. Check it occasionally with a validator so a redesign does not break it.