Is llms.txt Necessary?

Tomohiro Iida · Published June 25, 2026 · Updated August 6, 2026

This article sorts out the proposed spec for llms.txt, the range in which it is actually used, and why Google does not use it, so that you can judge whether it applies to your own site.

llms.txt is not a standardized search requirement

llms.txt is a proposal, not a spec that search engines ask for. That premise comes first.

Published on September 3, 2024, the proposal describes a format for writing a site overview, its important links, and supporting information in Markdown. The author has in mind serving a website as HTML for people while collecting concise, LLM-friendly information in one place. It fits situations such as developer documentation and API references, where an LLM needs an accurate entry point.

The following points, however, need to be kept separate.

• It is not an IETF or W3C standard
• It is not a required spec for search engines
• Not every LLM or AI search reads it
• How it is processed differs by application
• It is not at a stage where an installation effect can be proven with a shared metric

A file format being proposed and major services adopting it are not the same thing. The proposal itself says it includes no specific recommendation on how llms.txt should be processed, because that depends on the application.

Source: The /llms.txt file (llmstxt.org), checked August 6, 2026.

Google Search does not use it

Google states that there is no need to create AI-facing files such as llms.txt for generative AI search.

Google’s documentation on generative AI features states that there is no need to create a new machine-readable file, an AI text file, markup, or Markdown in order to appear in Google Search (including its generative AI features), and that Google Search itself does not use them. It also states plainly that creating and maintaining llms.txt for other services or systems is fine in itself, but neither raises nor lowers visibility or ranking in Google Search.

Source: Google Search Central, "Optimizing for generative AI features," checked August 6, 2026.

The technical premise for being eligible for Google’s AI Overviews or AI Mode is that a page is indexed in ordinary Google Search and can be eligible for snippet display. No additional AI-specific file and no special structured data are asked for. This is also covered in our article on generative AI search being SEO.

What to prioritize for Google

If the goal is visibility in Google, there are items to check before llms.txt.

• Important pages can be indexed
• The body text can be retrieved correctly in the initial HTML or after rendering
• canonical, noindex, and robots.txt are correct
• There is original, useful content
• Duplicate articles and stale information have been cleaned up
• Internal links and the sitemap work
• Impressions and clicks are measured in Search Console

Where effort is limited, we recommend fixing these first. Installing llms.txt is an option for after they are in order.

Where llms.txt may be useful

It is worth considering where the reader, or the target tool, is clearly identified.

An entry point to developer documentation

Where there are many reference pages — APIs, SDKs, libraries, product manuals — it can serve as an index of the main documentation. This is the use the proposal originally had in mind.

A source you hand to your own AI agent

Where an agent or RAG system you operate is implemented to read llms.txt explicitly. In that case the effect is evaluated on source selection and maintainability, not on search exposure.

A test with a clearly defined subject

Where you can confirm from official documentation or access logs that the target AI tool reads llms.txt, and can compare retrieval behavior before and after installing it.

In every case, "AI in general reads it widely, so we should install it" is weak as a reason. Whether you can name the reader is where the decision turns.

Five questions to answer before installing it

The following are the items we check when judging whether it applies. They are our own operating guideline, not a universal standard.

1. Who reads it?

Identify the target service, tool, or agent by name. With a vague subject such as "AI in general," neither the decision to install nor the way to confirm an effect can be settled.

2. Is there a basis for saying it is read?

Confirm it in official documentation, a product spec, or actual access logs. Until the reading side’s implementation can be confirmed, installing the file tells you nothing about whether it is being retrieved.

3. What do you want to improve?

Search ranking, citation in AI answers, documentation discovery by developers, and reference accuracy for your own agent are separate goals. Set them separately rather than mixing them.

4. Can it stay in sync with the canonical source?

Typing pricing, service names, and company information into llms.txt by hand creates mismatches with the web pages. Check first whether it can be generated automatically from the CMS or the code.

5. Can you measure it after installing?

Decide on verifiable indicators: access from the target tool, the pages referenced, changes in the answers. If no indicator can be settled, whether installing it was right cannot be judged afterwards either.

If these five cannot be answered, the priority of installing it can be judged as low.

A minimum structure, if you do write one

If you are building one to experiment, design it as an index pointing to the canonical source rather than packing information into it.

In practice, a workable order is a short description of the organization or product, links to the main documentation, links to APIs and specifications, important constraints, the update date, and the canonical web pages.

Restating the inquiry path, the company’s track record, or a price list separately increases the number of places that have to be updated. Linking to the current canonical source is safer than making llms.txt self-contained.

On derivative files such as llms-full.txt: llms-full.txt and llms-ctx.txt are not defined across the proposal as a single required standard. What the proposal text mentions is llms-ctx.txt and llms-ctx-full.txt, generated by expanding llms.txt in one specific project. Take care not to describe a file generated by a particular implementation or tool as a general standard.

Deciding not to install it is also reasonable

In the following states, spending the effort on ordinary site improvement is likely to be more effective than creating an llms.txt.

• Main pages are not indexed
• Company information and pricing contradict each other from page to page
• There are no articles answering customers’ questions
• The body text cannot be retrieved because of JavaScript
• Old articles remain as duplicates
• The path to an inquiry is hard to follow
• What to measure has not been decided

llms.txt is not a substitute for solving these basic problems. Adding an index changes nothing if the pages it points to cannot be retrieved, or if their content contradicts itself.

Where this sits at Netsujo

Netsujo does not propose llms.txt as a required measure for AI search. We treat it as an experimental option only where the target tool, the purpose, the update method, and the measurement method are clear.

On our own site we place a /llms.txt as an experiment. The file itself is maintained by hand, which makes it easy to miss an update. So information that changes easily, such as pricing, is not written into the file; it is replaced with a link to the canonical pricing page. Alongside that, a weekly automated check confirms that the URLs listed are reachable and that no stale dates or stale counts remain. This is not a measure aimed at search ranking or citation in AI answers; it is placed there so we can verify the maintenance and synchronization work ourselves. Isolating the effect of llms.txt on its own is difficult to measure, and this article presents no outcome figures for it.

What we prioritize for being found in AI search is a technical foundation that can be searched, original and accurate content, consistency of information, the disclosure needed for comparison, and measurement after implementation. What we actually did on our own site is recorded in the practice log, and the wider picture of AI search optimization is collected in our article on GEO.

Put the canonical source on the web in order before adding a new file. That is our current judgment.

FAQ

Is llms.txt required?

No. llms.txt is a proposal published in 2024. It is neither an IETF or W3C standard nor a required spec for search engines. Nor is it the case that a site without one stops being read by AI. It is most accurate to position it as one option where the target tool and the purpose can be identified.

Does installing llms.txt help in Google Search or AI Overviews?

No. Google’s official documentation states that there is no need to create a new AI-facing text file or markup in order to appear in Google Search (including its generative AI features), and that Google Search does not use them. Creating and maintaining llms.txt is fine in itself, but it is also stated plainly that it neither raises nor lowers visibility or ranking in Google Search.

How is it different from robots.txt?

robots.txt is an access-control file that tells crawlers which pages they may retrieve. Each vendor’s AI crawlers are also officially controlled through robots.txt. llms.txt, by contrast, is a proposal whose purpose is to present an index of content; it does not control whether access is permitted. The purposes are different.

Where do you put llms.txt?

The proposal shows placing it at the site root so that it can be retrieved at the URL /llms.txt (placing it under a subpath is also listed as an option). Following the spec on placement, however, is a separate matter from whether the target tool actually reads it. Confirm the reading side’s implementation before deciding to install.

Should we also prepare llms-full.txt?

llms-full.txt is not a file the proposal defines as a required standard. What the proposal text mentions is llms-ctx.txt and llms-ctx-full.txt, generated by expanding llms.txt in one specific project. Because the names and formats of derivative files differ by tool, do not treat them as a general standard; judge them against the spec of the tool you are using.

Summary

llms.txt is a proposal that collects a site’s key points and main pages in Markdown so that an LLM can use them more easily at inference time. It is neither an IETF or W3C standard nor a required spec for search engines. Google states plainly that Google Search, including its generative AI features, does not use this kind of file.

The axes for judging whether it applies are whether you can name the reader, whether there is a basis for saying it is read, whether it can stay in sync with the canonical source, and whether it can be measured after installing. The more of these you cannot answer, the lower the priority. Get indexing, retrievability of the body text, consistency of information, and measurement in order first. llms.txt is an experimental option to place after that.