Draft:Llms.txt

llms.txt

Filename extension	.txt
Developed by	Jeremy Howard (Answer.AI)
Initial release	2023
Type of format	Artificial intelligence, SEO
Standard	Proposed practice

llms.txt is a plain text file written in Markdown format, designed to facilitate access to and understanding of a website's content by large language models (LLMs). It acts as an explicit guide that website owners can provide to AI systems to help them identify the most relevant sections of their pages.

The file serves as a bridge between human-generated content and machine interpretation, allowing LLMs—such as ChatGPT, Claude, or Google Gemini—to better recognize which content deserves to be featured in AI-generated responses.

Origin and context

The concept was introduced in 2023 by Jeremy Howard, co-founder of Answer.AI. It addresses the need to improve how AI systems access and understand online content, especially by overcoming technical barriers such as JavaScript, dynamic menus, or non-semantic structures that hinder indexing.

As language models gain prominence in search and content generation, mechanisms like llms.txt become increasingly relevant to control and optimize how websites are interpreted.

Purpose and function

Diagram showing the information flow from a website to an LLM using llms.txt.

Unlike the traditional robots.txt, which regulates crawler access to specific parts of a website, the llms.txt file provides a structured, curated, and machine-readable summary of the site's most relevant content. Its primary goal is to improve semantic comprehension by AI models and increase a site's visibility in AI-generated results.

The file is especially useful for:

• Corporate or institutional websites that want to highlight services and goals.
• E-commerce sites that wish to prioritize product categories or featured listings.
• Educational platforms or blogs that aim to showcase authors and key articles.

File structure

The file is written in Markdown and organized in a clear hierarchy. A detailed explanation is also available in the original article by Francesc Sánchez at La Teva Web. According to the official documentation, it should include:

• A site title (`#`): main heading identifying the project or domain.
• A summary (`>`): blockquote explaining the site's purpose, target audience, and nature of the content.
• Main sections (`##`): secondary headings listing key links (services, articles, product categories, etc.).
• Optional content: additional links or resources (social media, documentation, etc.).

Sample

# Open Data Portal

> Repository of open datasets for researchers, journalists, and citizens.

## Climate data
- https://opendata.gov/climate

## API documentation
- https://opendata.gov/docs/api

## Contact
- https://opendata.gov/contact

A live example of llms.txt can be found at www.latevaweb.com/llms.txt, implemented by a digital marketing agency.

Implementation and hosting

The file must be named llms.txt and placed in the website's root directory (e.g., https://example.com/llms.txt). Best practices include:

• Verifying public access.
• Avoiding more than one H1 heading.
• Including only relevant links and sections.
• Keeping the file up to date.
• Using proper UTF-8 encoding and clean Markdown syntax.

Plugins for platforms like WordPress and tools like Wordlift or Firecrawl can generate the file automatically.

Related files

Some websites also include a supplementary file called llms-full.txt, offering a more detailed technical description. While optional, it is useful for large or complex projects.

Use cases

The file has been adopted by educational, corporate, and institutional websites as a strategy to improve their presence in AI responses. It is also used to prevent misinterpretation and ensure that key content is correctly represented.

Relevance and adoption

While llms.txt is not yet an official standard supported by all AI platforms, it is emerging as a recommended practice in technical SEO and AI-aligned web development.

See also

• robots.txt
• Large language model
• Markdown
• Search engine optimization
• Artificial intelligence

References

Category:Plain text file formats Category:Artificial intelligence Category:Large language models Category:Metadata standards Category:Web development Category:Search engine optimization