# llms.txt: what it is, who reads it and how to write one > What the llms.txt proposal is, which AI tools actually read it and which do not, how to write one with real examples, where it goes, llms-full.txt, and how it differs from robots.txt. Updated 2026-09-26 · Technical SEO · HTML version: https://getreport.app/guides/llms-txt-what-it-is-and-how-to-write-one llms.txt is a plain Markdown file at `/llms.txt` that gives AI assistants and agents a short, curated map of your site: what it is, and which pages to read for what. This guide is for site owners and developers who keep hearing about it and want a straight answer. By the end you will know what the file does and does not do, who reads it today, and you will have a valid llms.txt in place. It takes about 15 minutes. ## Quick answer - **llms.txt** is a proposal from [llmstxt.org](https://llmstxt.org/), first published in September 2024 and revised in August 2026. It is not a web standard and no search engine has committed to using it. - Coding agents and documentation tools read it; Google has said Search does not use it. It does not change rankings, and it does not get you "indexed by an LLM". - The format: an `# H1` with the site name, a `> blockquote` summary, then `## Section` headings with lists of `- [Title](URL): what the page answers`. - Put it at the site root, `https://example.com/llms.txt`, served as `text/plain` or `text/markdown`, under 100 KB. - It controls nothing. Which AI crawlers may fetch your pages is decided in robots.txt. - Build a draft from your sitemap with the [llms.txt generator](https://getreport.app/tools/llms-txt-generator), then check it with the [AI crawler check](https://getreport.app/tools/ai-crawler-check). ## What is llms.txt? The llms.txt file is a Markdown index of a website written for language models rather than for people or search crawlers. Jeremy Howard of Answer.AI proposed it in September 2024 because web pages are hard for a model to read: navigation, cookie banners and scripts wrap the actual content, and a whole site does not fit into a model's context. A short file at a known address lets an agent find the few pages it needs and fetch only those. Two things make it different from the files you already have: - **It is curated.** Your sitemap lists every indexable URL, often thousands. llms.txt lists the 10 to 100 pages that best explain what you do, each with a sentence on what it answers. - **It is written for reading, not for rules.** It holds no allow or deny rules, unlike robots.txt, and search engines do not read it for crawling, unlike a sitemap. People search for both "llms.txt" and "llm.txt". The proposal's name is **llms.txt**, with an s; a file at `/llm.txt` is not what any tool looks for. ## Does llms.txt help SEO or AI visibility? No, not for search rankings. Google's Search team has said publicly that Google Search does not use llms.txt, and that AI Overviews and AI Mode are built from the normal search index, crawled by Googlebot. No other search engine has announced support. A file that no ranking system reads cannot move rankings. For AI assistants the picture is "sometimes, when someone asks". No major assistant (ChatGPT, Claude, Gemini, Perplexity, Copilot) has said its crawlers look for llms.txt or treat it as a signal for what to cite. Most AI answers still come from pages fetched or indexed the ordinary way, which is why server-rendered text, clear titles and structured data matter far more. So why write one? Because it costs 15 minutes and helps the tools that do use it: an agent that is pointed at your site finds the right pages faster, and for documentation it is already routine. Treat it as a cheap bet, not a ranking factor. ## Which AI tools actually read llms.txt? Honest status in September 2026: | Who | Uses llms.txt? | | --- | --- | | Coding agents and IDE assistants | Yes, routinely for documentation: when a developer points them at a library's docs, they read its llms.txt and follow the links | | Documentation platforms (Mintlify, GitBook) | Generate one automatically for every docs site they host | | AI labs' own developer docs | OpenAI, Anthropic and Google's Gemini API docs each publish one | | Google Search, AI Overviews, AI Mode | No; Google has said Search does not use it | | ChatGPT, Claude, Perplexity crawlers | No public commitment to fetch or use it | You can check your own case: look for requests to `/llms.txt` in your server access log. On most small business sites they are rare, and many come from SEO tools checking whether the file exists. ## What goes in an llms.txt file? The llmstxt.org format is strict about order. In a valid file: 1. An **H1** with the name of the site or project. This is the only required part. 2. A **blockquote** (`> …`) with a one- or two-sentence summary. 3. Optional paragraphs or lists with more context, but **no headings**. 4. Zero or more **H2 sections**, each holding a list of links: `- [Name](URL)`, optionally followed by `: notes`. 5. By convention, a section called `## Optional` for secondary links an agent can skip when it is short of space. A complete file for a small online shop: ```text # Example Shop > Example Shop sells handmade ceramic tableware from Zagreb and ships across the EU. Orders over €60 ship free; returns are accepted within 30 days. Prices are in euros and include VAT. Stock levels change daily; the product pages are the source of truth. ## Products - [Dinner plates](https://example-shop.hr/en/plates/): All plate sizes, glazes and set prices - [Mugs and cups](https://example-shop.hr/en/mugs/): Capacities, dishwasher and microwave use - [Gift sets](https://example-shop.hr/en/gift-sets/): Ready-made sets with gift wrapping ## Buying - [Shipping](https://example-shop.hr/en/shipping/): Countries, delivery times and costs - [Returns](https://example-shop.hr/en/returns/): 30-day returns and how refunds are paid - [Care instructions](https://example-shop.hr/en/care/): Washing, chips and glaze questions ## Optional - [About the studio](https://example-shop.hr/en/about/): Who makes the pieces and where - [Privacy policy](https://example-shop.hr/en/privacy/) ``` And one for a service business, with the context paragraph doing more work: ```text # Harbour Physio > Physiotherapy clinic in Split for sports injuries, back pain and post-surgery rehabilitation. Appointments on weekdays 8:00–20:00. We do not need a doctor's referral. Private health insurance covers most treatments; check with your insurer first. ## Treatments - [Sports injuries](https://harbour-physio.example/sports-injuries/): Conditions treated and a typical course of sessions - [Back and neck pain](https://harbour-physio.example/back-pain/): Assessment, treatment and exercises - [Prices](https://harbour-physio.example/prices/): Session prices and packages ## Visiting - [Book an appointment](https://harbour-physio.example/book/): Online booking and cancellation rules - [Location and parking](https://harbour-physio.example/contact/): Address, parking and public transport ``` Write the notes after each link as the question the page answers ("Countries, delivery times and costs"), not as marketing copy. Link to pages that work without JavaScript, because most AI fetchers do not run scripts. ### Markdown versions of pages The 2026 revision of the proposal also suggests offering a clean Markdown copy of important pages at the same URL with `.md` added (`/shipping/index.md` or `/docs/page.html.md`), and pointing to it with ``. It also defines `rel="describedby"` for pointing a page at the llms.txt that covers it. Both are optional. For documentation sites they are worth it; for a shop or a clinic the HTML pages are fine. ## What is llms-full.txt? `llms-full.txt` is a companion convention: one Markdown file with the **full text** of the pages your llms.txt links to, so an agent can load everything in a single request. It grew up around documentation platforms rather than the llmstxt.org proposal itself, and the proposal does not define it. It makes sense for documentation, API references and knowledge bases, where a developer wants a whole manual in one go. For a marketing site or a shop it is usually not worth maintaining, because it goes stale every time a page changes. If you publish one, generate it at build time from the same source as the pages, and keep llms.txt itself short: long content belongs in llms-full.txt, not in the index. ## Where do I put llms.txt on my website? At the root of the host, next to robots.txt: ```text https://example.com/llms.txt ``` The proposal also allows a file on a subpath that covers only the pages below it, such as `https://example.com/docs/llms.txt` for a documentation section; when more than one applies, the most specific wins. Most sites need just the root file. The file must: - answer **200** at exactly that URL, without redirecting to a different host; - be served as `text/plain` or `text/markdown`, UTF-8; a route that returns your HTML 404 page with status 200 fails; - stay under **100 KB**; move long content to llms-full.txt; - be reachable by the crawlers you want to read it. It is fetched like any other URL, so a `Disallow: /` for a crawler in robots.txt covers it too. Each host is separate: `shop.example.com` needs its own file. ## How do I create an llms.txt file (by hand, plugin or generator)? ### With the generator > **Free tool:** [llms.txt generator: built from your sitemap](https://getreport.app/tools/llms-txt-generator): Free llms.txt generator: we read your sitemap, draft the file with real page titles in sections, and you edit it in your browser with live checks. The generator reads your robots.txt and sitemaps (up to 500 pages), takes the titles and meta descriptions of the 40 most important ones, and groups them into sections by where they live: products, blog, docs, about. Carts, account pages, tag archives, feeds and pagination are dropped, and privacy and terms go to `## Optional`. You then edit the draft in your browser: rename sections, reorder links, rewrite the notes. It is validated as you type with the same rules as the report check, and nothing you type is sent to us. ### By hand 1. Write the H1 and a summary a stranger would understand: what you do, for whom, where. 2. Add one or two context lines an assistant would otherwise get wrong: prices include VAT, no referral needed, which regions you serve. 3. Pick 10–40 pages that answer the questions customers actually ask. Group them under 2–5 H2 sections. 4. Write a short note after each link: what the page answers. 5. Save as `llms.txt` (UTF-8, no BOM needed) and upload it to the web root. ### With your CMS - **WordPress:** Yoast SEO, AIOSEO and Rank Math can generate and update an llms.txt; check the file they produce, because automatic ones tend to list recent posts rather than your key pages. Or upload your own file to the folder that holds `wp-config.php`. A physical file wins over a plugin's virtual one on most servers, so do not use both. - **Wix** generates one for every site; review it in the SEO settings. - **Shopify** does not let you upload arbitrary files to the root. The usual workaround is a URL redirect from `/llms.txt` to a file uploaded under Content → Files, which lands on Shopify's CDN host. Tools that follow redirects can read it, but it is not a file at your root; whatever route you use, check what `/llms.txt` actually answers. - **Static sites and frameworks:** put `llms.txt` in the folder copied to the web root (`public/` in Next.js, Astro and Vite, `static/` in Hugo and SvelteKit). - **Documentation platforms** (Mintlify, GitBook, Docusaurus and VitePress plugins) generate llms.txt, and often llms-full.txt, from the docs. ## llms.txt vs robots.txt: what is the difference? | | robots.txt | llms.txt | | --- | --- | --- | | Purpose | Access rules: which crawler may fetch which paths | A reading list: which pages explain the site | | Standard | RFC 9309, honoured by all major crawlers | A proposal from llmstxt.org | | Who reads it | Every well-behaved crawler, before crawling | Some agents and tools, on demand | | Can it block AI? | Yes, per user agent (GPTBot, ClaudeBot, Google-Extended…) | No | They work together but never override each other. If robots.txt blocks GPTBot, GPTBot will not fetch your pages, whatever llms.txt lists. If you want AI assistants to be able to cite you, allow their crawlers in robots.txt first; our guide on [AI crawlers and robots.txt](https://getreport.app/guides/ai-crawlers-and-robots-txt) goes through them one by one, and [robots.txt before it hides your site](https://getreport.app/guides/robots-txt-before-it-hides-your-site) covers the mistakes that block everyone. "robots.txt for LLMs" is a common description of llms.txt, and a misleading one. There is no llms.txt syntax for "do not train on this"; that is robots.txt (`User-agent: GPTBot` / `Disallow: /`) or a `noai` directive. ## How do I check a site's llms.txt? The quickest way is to open `https://thatsite.com/llms.txt` in a browser. For a full check, including format and crawler access: > **Free tool:** [Check which AI crawlers can read your site](https://getreport.app/tools/ai-crawler-check): See what your robots.txt tells 18 AI crawlers, such as GPTBot, ClaudeBot and PerplexityBot, whether your llms.txt is valid and what needs JavaScript. Free. > **Check: llms.txt found.** /llms.txt is a plain-text index of your most useful pages written for AI assistants, in the format proposed at llmstxt.org. It is optional and new: a few tools read it, but no major assistant has confirmed using it and Google has said Search does not. It costs about ten minutes, so treat it as a cheap bet, not a ranking factor. > > 1. Create /llms.txt: an H1 with the site name, a one-line blockquote summary, then "## Section" headings with Markdown links to your key pages and a short description each. > 2. Serve it as text/plain or text/markdown; optionally add /llms-full.txt with the full text of those pages. > **Check: llms.txt follows the llmstxt.org format.** An llms.txt that is not in the expected shape (an H1 first, links grouped under sections, served as text) is skipped by the tools that read it, so it does nothing. > > 1. Start the file with "# Site name", follow with an optional "> summary" line, then "## Section" headings each holding "- [Page title](https://…): one-line description" entries. > 2. Serve it with a text/plain or text/markdown content type and keep it under 100 KB; put long content in /llms-full.txt instead. The format check applies the llmstxt.org rules: an H1 first, a summary, links grouped under H2 sections, a text content type and a sensible size. It also notes whether `/llms-full.txt` answers. > **Check: AI crawler access.** AI assistants (ChatGPT, Claude, Perplexity) fetch pages with their own user agents. Google's AI Overviews use normal Googlebot, so only blocking Googlebot removes you from them, and that removes you from Search too. Blocking them keeps your content out of AI answers; allowing them can bring citations and visitors. Either is a valid choice, as long as it is the one you meant. > > 1. Decide per crawler: training bots (GPTBot, ClaudeBot, Google-Extended, CCBot) feed models; search bots (OAI-SearchBot, PerplexityBot) feed answers that link back; user bots (ChatGPT-User, Claude-User) fetch a page because someone asked. Google-Extended is a robots.txt token, not a crawler: it controls use in Gemini training, not AI Overviews. > 2. Edit robots.txt: a "User-agent: GPTBot" group with "Disallow: /" blocks it; leave it out of robots.txt (or "Allow: /") to permit it. Rules for "*" apply to any crawler without its own group. From a terminal: ```bash curl -sI https://example.com/llms.txt | grep -iE "^HTTP|content-type" curl -s https://example.com/llms.txt | head -20 ``` ## Can AI crawlers read JavaScript pages? Mostly not. A study by Vercel and MERJ of AI crawler traffic, published in December 2024, found that GPTBot, ClaudeBot and PerplexityBot download JavaScript files but do not run them; of the large AI crawlers, only Google's (which shares Googlebot's renderer) and Apple's render pages. If your prices, product descriptions or opening hours only appear after scripts run, most AI assistants never see them, and an llms.txt link to that page does not help. > **Check: The visible text is present without JavaScript.** Google renders JavaScript later and with a budget, so text that only appears after scripts run can be indexed late or not at all. Other search engines and link previews may never see it. > > 1. Serve the main content in the HTML (server-side rendering or static generation) and use JavaScript only to enhance it. > 2. Check the difference in the technical detail; menus and widgets are fine, headlines and body copy are not. The fix is server-side or static rendering of the content that matters; [JavaScript-rendered content and Google](https://getreport.app/guides/javascript-rendered-content-and-google) explains the options. ## Verify - `https://yoursite/llms.txt` opens as plain text in the browser, not as a download and not as an HTML page. - The [AI crawler check](https://getreport.app/tools/ai-crawler-check) reads "llms.txt follows the llmstxt.org format". - Every link in the file answers 200; paste them into the [bulk URL checker](https://getreport.app/tools/bulk-url-checker) if there are many. - Give an assistant only the llms.txt URL and ask it three questions a customer would ask. If it cannot find the answers, improve the notes or add the missing page. ## Common mistakes - **Naming it `llm.txt` or putting it under `/.well-known/`.** Tools look for `/llms.txt` at the root (or on a subpath it covers). - **Pasting the sitemap.** A list of 2,000 URLs without notes is exactly what the file is meant to replace. - **Headings before the links start.** Anything between the blockquote and the first H2 must be paragraphs or lists; an H3 there breaks the format. - **Serving HTML.** A single-page app that answers every path with `index.html` returns a 200 HTML page for `/llms.txt`. Add a real file or route. - **Expecting rankings or traffic.** It is an aid for agents, not a search signal. Spend the bigger effort on crawlable, server-rendered pages. - **Listing pages robots.txt blocks.** The crawler still obeys robots.txt; the link is useless to it. - **Letting it go stale.** Review it when you change prices, services or the site structure, the same way you would a sitemap. ## Questions people ask ### How do I add llms.txt to WordPress? Upload the file to the root of your site, the folder that holds wp-config.php, with your host's file manager or SFTP, so it answers at https://yourdomain.com/llms.txt. Recent versions of Yoast SEO and Rank Math can also generate one for you. Then open the address in a browser and confirm it loads as plain text and is not redirected to a WordPress page. ### Is it llm.txt or llms.txt? It is llms.txt, with an s, at `https://yoursite/llms.txt`. That is the name in the llmstxt.org proposal and the only one tools and plugins look for. "llm.txt" is a common misspelling in searches. If you created a file called llm.txt, rename it; there is no need to keep both. ### Where do I place the llms.txt file on my website? Put it in the web root, next to robots.txt, so it answers at `https://yoursite/llms.txt` with status 200 and a `text/plain` or `text/markdown` content type. On WordPress that is the folder with `wp-config.php`; on static sites and frameworks, the `public` or `static` folder. Each subdomain needs its own file. ### Can llms.txt get my site indexed by an LLM? No. Language models are not indexes you submit to. Assistants find pages through their own crawlers, search indexes they license, or live fetches when a user asks. llms.txt only helps an agent that already reads your site to find the right pages. To be found, allow AI crawlers in robots.txt and make sure your content is in the HTML without JavaScript. ### How do I see the llms.txt file of a website? Open `https://thatsite.com/llms.txt` in your browser; if the site has one, it shows as plain text. A 404 means there is none. To check that it is also well-formed and that the site's robots.txt lets AI crawlers in, run the free AI crawler check, which validates the file against the llmstxt.org format. ### Does ChatGPT read llms.txt? There is no public confirmation that it does. None of the major AI companies has said its crawlers or assistants use llms.txt when answering questions, and server logs show only occasional requests for the file. Some coding tools and documentation assistants do read it, and it is cheap to publish, so treat it as a low-cost bet rather than a way to rank in AI answers. ### Will an llms.txt file help my SEO? No. Google has said Search does not use llms.txt, and no other search engine has announced support, so it does not affect rankings or AI Overviews. It can help coding agents and some AI tools find your key pages faster. At 15 minutes of work it is a reasonable extra, but it is not an SEO task. ### What should I do with an llms.txt file? Write it once, upload it to your web root and review it when your key pages change. Keep it to the pages that answer real customer questions, with a short note on each. Then leave it alone: there is nothing to submit and no console to register it in. Check it occasionally with a validator so a redesign does not break it.