As of 30 October 2026, an llms.txt file is not described as a way to get cited in anything we read from Google, OpenAI, Anthropic, Perplexity or Bing. Google says its Search ignores llms.txt files, so keeping one neither helps nor harms your visibility there. We found no statement that the other four read one.
- Google says its Search ignores llms.txt files, so creating one will neither harm nor help your site's visibility or rankings in Google Search (Google, Search Central documentation, read 30 October 2026).
- Google's documentation updates page records a 15 June 2026 entry, Clarifying guidance on llms.txt files, saying Google added a note on llms.txt to its AI optimization guide (Google, Search Central documentation updates, read 30 October 2026).
- Google's AI features page, last updated 10 December 2025, says a page must be indexed and snippet-eligible to be a supporting link in AI Overviews or AI Mode, with no additional technical requirements (Google, Search Central documentation, read 30 October 2026).
- OpenAI and Perplexity each recommend allowing their search crawlers, OAI-SearchBot and PerplexityBot, in your robots.txt file and allowing requests from their published IP ranges (OpenAI and Perplexity crawler documentation, read 30 October 2026).
- OpenAI's and Perplexity's crawler pages mention llms.txt only in notices that point readers to each company's own documentation index (OpenAI and Perplexity crawler documentation, read 30 October 2026).
What is llms.txt?
llms.txt is a text file that a site serves from the root of its address, as /llms.txt, to give AI tools information about the site, such as an index of its pages. Perplexity's crawler page points AI agents to /llms.txt with the line Use this file to discover all available pages before exploring further.
OpenAI's crawler page carries the line For the complete documentation index, see llms.txt.
Both are notices for readers of those companies' own documentation, not statements about how ChatGPT search or Perplexity treat an llms.txt file on someone else's site. Google puts llms.txt under what it calls AI text files
. The idea has a public proposal at llmstxt.org, which is not one of the primary sources this guide cites, so we do not describe its format.
What does Google say about llms.txt?
Google says its Search does not use llms.txt files and that keeping one does not change your visibility or rankings in Google Search. Its guide to optimizing for generative AI features, last updated on 10 July 2026, says You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them.
The same guide names the file: It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files.
Doing so, it adds, will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.
A separate line lists creating unnecessary AI text files (like llms.txt)
among tactics you can ignore for Google Search.
Google also dated the clarification: its documentation updates page records a 15 June 2026 entry, Clarifying guidance on llms.txt files, saying Google added a note to the guide. Its AI features page, last updated on 10 December 2025, says a page must be indexed and eligible to be shown with a snippet to be a supporting link in AI Overviews or AI Mode, and adds There are no additional technical requirements.
What do OpenAI, Anthropic, Perplexity and Bing say about llms.txt?
In the crawler and help pages we read, OpenAI, Anthropic, Perplexity and Bing say nothing about using llms.txt to decide what to crawl or cite, and their crawler pages point to robots.txt instead. OpenAI says we recommend allowing OAI-SearchBot in your site's robots.txt file
, and Perplexity says we recommend allowing PerplexityBot in your site's robots.txt file
. Anthropic says its bots honor industry standard directives in robots.txt
, and Bing says Robots.txt files can be configured to tell Bing crawlers how to interact with your website.
| Engine | What its pages name as the control | llms.txt in the pages we read |
|---|---|---|
| Google Search (AI Overviews, AI Mode) | An indexed page eligible to be shown with a snippet, and inclusion in Search generative AI features in Search Console | Yes: Google says Search ignores it |
| OpenAI (ChatGPT search) | OAI-SearchBot in robots.txt, and requests from OpenAI's published IP ranges | Crawler overview: only a notice for OpenAI's own documentation. Search help article: none |
| Perplexity | PerplexityBot in robots.txt, and requests from published IP ranges | Only a notice for Perplexity's own documentation |
| Anthropic (Claude) | robots.txt directives, with separate bots named ClaudeBot, Claude-User and Claude-SearchBot | None |
| Microsoft Bing | robots.txt, for Bing's crawlers | None |
What would show that llms.txt helps a site get cited?
Our reasoning is that showing llms.txt helps a site get cited would take either a statement from an engine that it reads the file or a measured rise in citations larger than sampling noise. The engine pages we read contain no such statement, and we did not review third-party experiments, so we make no claim about them. Penlid's methodology says answers from language models are stochastic, so a single observation is not a measurement, and it reports a citation rate as k citations out of n samples with a 95% Wilson score interval. It calls two rates significantly different only when their intervals do not overlap, and it names model drift as a real confound: an engine silently upgrading its model can move your citation rate without anything on your site changing. Our guides to measuring AI visibility and why AI visibility changes when you changed nothing cover both, and the methodology page lists the rules. A before-and-after count around an llms.txt launch is timing evidence at best, as our guide to attributing AI citations to a content change explains.
What should you do before writing an llms.txt file?
Before writing an llms.txt file, start with what the engines document: crawl access, indexing and snippet eligibility, plus, for Google's AI features, one Search Console requirement. Google's guide says a site must be included in Search generative AI features in Search Console to be eligible for display in generative AI features on Google Search.
It also says The best practices for SEO continue to be relevant because our generative AI features on Google Search are rooted in our core Search ranking and quality systems.
Our guide to the difference between SEO, AEO and GEO sorts out what each can measure.
- To measure performance in Google's AI features, Google's guide points to the Generative AI performance report in Search Console, and our guide to reading that report covers it.
- For ChatGPT, Perplexity, Claude and Bing, check that robots.txt and any bot-blocking rule let their search crawlers in; which crawlers to allow for AI search lists what each company documents.
- If you still want an llms.txt file, Google calls that fine; our reasoning is to weigh its cost against the lack of any stated citation benefit and keep it only if a tool you use reads it.
Example (illustrative numbers)
These numbers are invented and are not data from any site. A site adds an llms.txt file on a Monday. Its owner asks one AI search engine 9 unbranded buying questions, 10 times each, in the week before and again in the week after, so each week has 9 × 10 = 90 answers. The site is cited in 27 answers before the file and 36 after.
The rates. 27 ÷ 90 = 30.0% before and 36 ÷ 90 = 40.0% after, a gap of 40.0 − 30.0 = 10.0 percentage points.
The intervals. Penlid's methodology reports a rate with a 95% Wilson score interval. With z = 1.96, z² = 3.8416 and n = 90, the interval is the center plus or minus the half-width, where center = (p + z² ÷ (2n)) ÷ (1 + z² ÷ n) and half-width = z × √(p(1 − p) ÷ n + z² ÷ (4n²)) ÷ (1 + z² ÷ n). For the before week, p = 27 ÷ 90 = 0.3 and z² ÷ n = 3.8416 ÷ 90 = 0.04268, so the center is (0.3 + 0.02134) ÷ 1.04268 = 0.3082. The half-width is 1.96 × √(0.002333 + 0.000119) ÷ 1.04268 = 1.96 × 0.04952 ÷ 1.04268 = 0.0931. The interval is 0.3082 ± 0.0931, which is 21.5% to 40.1%. The same steps for the after week give 30.5% to 50.3%.
| Week | Cited | Answers | Rate | 95% interval |
|---|---|---|---|---|
| Before the file | 27 | 90 | 30.0% | 21.5% to 40.1% |
| After the file | 36 | 90 | 40.0% | 30.5% to 50.3% |
The comparison. The intervals overlap between 30.5% and 40.1%, so Penlid's rule calls the 10-point gap no significant change. As our own cross-check, not part of Penlid's rule, a two-proportion z-test pools the weeks: (27 + 36) ÷ (90 + 90) = 0.35, the standard error is √(0.35 × 0.65 × (1 ÷ 90 + 1 ÷ 90)) = 0.0711, and z = 0.1 ÷ 0.0711 = 1.41, a two-sided p of about 0.16, well above the usual 5% line.
Our reasoning. A gap that cleared both tests would still be timing evidence, not proof that the file did anything. The two weeks could also differ by an engine model update, a change in the engine's retrieval or a competitor's PR win, which Penlid's methodology lists beside your own content as possible causes of a citation-rate change.
What this cannot tell you
This guide cannot tell you what any engine does inside its systems, only what its pages said when we read them on 30 October 2026. A page that does not mention llms.txt does not prove that no system reads the file, and Google's own wording allows that other services or systems may.
- It cannot speak for engines, assistants and tools we did not read, and we did not review third-party experiments with the file.
- It cannot say that an llms.txt file never helps anyone, only what Google says about its own Search and that we found no stated citation benefit from the other four.
- The example's intervals cover sampling noise for the questions asked. Penlid's methodology lists prompt choice as a separate limit, because which questions you track changes your measured visibility.
- We did not use the proposal's own text as a source, so we cannot say what a correctly formed file contains.
Questions people also ask
What is llms.txt in SEO?
In SEO, llms.txt is an optional file that, by Google's account, changes nothing about how a site ranks or appears in Google Search. Google says its Search does not use such files, and that the usual SEO best practices still apply to its generative AI features. For the other engines, the pages we read describe robots.txt and crawler access as the levers and do not describe llms.txt as one.
What is llms.txt used for?
Two companies we read, OpenAI and Perplexity, use llms.txt to point readers at an index of their own documentation, and Perplexity's notice speaks to AI agents. Google notes that site owners may keep a file for other services or systems that use it. The pages we read do not name those systems, and none says its crawler uses the file to pick which sites to cite.
How do you create an llms.txt file?
The pages this guide cites do not define a format, so we give no template; the proposal's own text is at llmstxt.org, which we did not use as a source. Whatever tool or plugin writes the file, Google says you do not need one to appear in Google Search, and the other engines' pages we read do not ask for it. Our reasoning is that crawl access and indexing deserve your time first.
What does an llms.txt checker or validator tell you?
Our reasoning is that a checker or validator inspects the file itself, not what any AI engine does with it, so it cannot show that an engine fetched the file or used it in an answer. A pass says only that the file looks well formed to that tool, and nothing about citations. Bing's AI Performance help page, which covers grounding and citation activity, does not mention llms.txt.
Sources
- Optimizing your website for generative AI features on Google Search, Google Search Central documentation, Google, read 30 October 2026.
You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them.
It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files.
will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.
creating unnecessary AI text files (like llms.txt)
a site must be included in Search generative AI features in Search Console to be eligible for display in generative AI features on Google Search.
The best practices for SEO continue to be relevant because our generative AI features on Google Search are rooted in our core Search ranking and quality systems.
- Latest Google Search Documentation Updates, Google Search Central, Google, read 30 October 2026.
- AI features and your website, Google Search Central documentation, Google, read 30 October 2026.
There are no additional technical requirements.
- Overview of OpenAI Crawlers, OpenAI API documentation, OpenAI, read 30 October 2026.
we recommend allowing OAI-SearchBot in your site's robots.txt file
For the complete documentation index, see llms.txt.
- Searching the web with ChatGPT, OpenAI Help Center, OpenAI, read 30 October 2026.
- Perplexity Crawlers, Perplexity documentation, Perplexity, read 30 October 2026.
we recommend allowing PerplexityBot in your site's robots.txt file
Use this file to discover all available pages before exploring further.
- Does Anthropic crawl data from the web, and how can site owners block the crawler?, Claude Help Center, Anthropic, read 30 October 2026.
industry standard directives in robots.txt
- Which Crawlers Does Bing Use?, Bing Webmaster Tools Help, Microsoft, read 30 October 2026.
Robots.txt files can be configured to tell Bing crawlers how to interact with your website.
- AI Performance, Bing Webmaster Tools Help, Microsoft, read 30 October 2026.
- Methodology: how we measure, and what we can't, Penlid, read 30 October 2026.