What llms.txt is
Web pages are built for people, with navigation, scripts and other page furniture around the text. The llms.txt proposal gives agents a shortcut: one markdown file that says what a site is, with links to the pages worth reading. In the proposal's words, agents view or search the llms.txt to find what they need, then follow the relevant links.
Jeremy Howard published the proposal on September 3, 2024. Version 2, from August 2026, adds standard ways for agents to find the file and the markdown versions of pages.
The format
An llms.txt file is markdown with these parts, in this order:
- An H1 with the name of the site or project. It's the only required part.
- A blockquote with a short summary, holding the key information needed to understand the rest of the file.
- Any number of paragraphs or lists, without headings, with more detail about the site and how to read the linked files.
- Any number of sections under H2 headings, each a list of links. Every item is a markdown link, optionally followed by a colon and a note about the page.
A section called Optional holds secondary links an agent can skip when it needs a shorter context.
To draft a file in this format from your own pages, use the free llms.txt generator: it reads your home page, sitemaps and main pages, and you edit the result before you download it. To validate a live file against this format, enter your domain in the free robots.txt and llms.txt checker. It flags a missing H1, sections without a link list and malformed links.
An example
This is the start of onelence.com/llms.txt, shortened:
# OneLence
> OneLence is the growth operating system for founders and marketing teams. It watches the whole growth engine (website journeys and conversions, ad platforms, search, AI assistants and affiliate partners) and says what deserves attention, what to do about it and how sure it is, even when attribution is incomplete.
## Connect OneLence to AI assistants
- [OneLence MCP server](https://onelence.com/integrations/mcp): https://mcp.onelence.com (Streamable HTTP, OAuth), available as a Claude connector
- [Use OneLence in ChatGPT](https://onelence.com/integrations/chatgpt): turn on developer mode, then create an app with the server URL and OAuth.
## Optional
- [Blog](https://onelence.com/blog)
Markdown versions of pages
The proposal also suggests that pages agents might need offer a clean markdown version at the same URL, with .md appended (page.html.md) or with the extension replaced (page.md). Version 2 accepts both forms.
To help agents find these files, version 2 recommends two standard link relations, as HTML <link> elements or an HTTP Link header:
rel="alternate" type="text/markdown"points to the markdown version of a page.rel="describedby"points to the llms.txt file that covers the page.
An llms.txt file covers every page under its path, so /docs/llms.txt covers everything in /docs/, and a more specific file takes precedence.
Who reads llms.txt
- Google Search doesn't. Google says you don't need machine-readable files, AI text files or markdown to appear in Google Search, including its generative AI features, because Google Search doesn't use them. It also says creating llms.txt files for other services is fine and will neither harm nor help your visibility or rankings in Google Search.
- The AI companies publish one, but don't say they read yours. OpenAI, Anthropic and Perplexity each publish an llms.txt for their developer docs. Their crawler documentation doesn't say GPTBot, ChatGPT-User, ClaudeBot or PerplexityBot read llms.txt on other sites.
- Lighthouse checks it. Chrome's Lighthouse has an llms.txt audit among its agentic browsing audits. It fails only when your server returns an error for
/llms.txt. A missing file is marked not applicable, and the audit's documentation calls the file optional for now.
The only way to know whether agents read yours is to look at who requests it.
llms.txt vs robots.txt vs sitemap.xml
| File | What it's for | Who it's written for |
|---|---|---|
robots.txt | Says which parts of a site crawlers may access | Every crawler, before it fetches pages |
sitemap.xml | Lists every indexable page | Search engines building an index |
llms.txt | Summarizes the site and links to the pages worth reading | An agent that needs information while helping someone |
llms.txt doesn't replace either file, and it doesn't grant or block access.
How to write a useful llms.txt
- Put the facts you want repeated in the summary. What you do, for whom, and anything an assistant tends to get wrong, such as pricing or which platforms you support.
- Link the pages that answer real questions: product, pricing, integrations, docs and comparisons, each with a one-line note on what the page answers.
- Move the rest under
Optional. Blog posts and secondary pages go there. - Keep it current. A summary with last year's prices does more harm than no file.
- Check that it's fetchable at
/llms.txtwith a 200 response, and that robots.txt doesn't block it.
Seeing who fetches your llms.txt
Your server and CDN logs record each request for /llms.txt with its user agent and IP address, which is enough to tell a browser from a crawler. Google Analytics won't show these requests, because it excludes known bots and only measures pages that load its tag.
OneLence AI visibility shows the pages AI crawlers request when your server reports those requests to OneLence. Report /llms.txt along with your pages and you can see which AI crawlers fetch it, and whether the pages it links to get read and visited.
Frequently asked questions
What is llms.txt?
llms.txt is a markdown file, usually at /llms.txt, that gives AI agents a short summary of a site and links to the pages with more detail. Jeremy Howard proposed it at llmstxt.org on September 3, 2024, and published version 2 of the proposal in August 2026.
Does Google use llms.txt?
No. Google says Google Search, including its generative AI features, doesn't use llms.txt files, and that having one will neither harm nor help your site's visibility or rankings in Google Search.
Do ChatGPT, Claude and Perplexity read llms.txt?
Their crawler documentation doesn't say so. OpenAI, Anthropic and Perplexity publish llms.txt files for their own developer docs, but none of them says GPTBot, ClaudeBot, PerplexityBot or their other agents read llms.txt on other sites. Your server logs show whether any agent fetches yours.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which parts of a site they may access. llms.txt suggests what's worth reading, for an agent that needs information about a topic while helping someone. It doesn't allow or block anything.
Where do I put llms.txt?
At the root of your site, as /llms.txt. The proposal also allows files at a subpath, such as /docs/llms.txt, which then cover the pages under that path.
Should I create an llms.txt file?
It takes little time, and Google says it won't hurt your rankings. Write one if you want agents that look for it to find an accurate summary and your most useful pages, but don't expect it to change your visibility in Google, ChatGPT or Claude by itself.