What Is llms.txt? A 2026 Guide for AI Visibility

llms.txt is a Markdown-formatted text file placed at the root of a website, designed to help large language models (LLMs) understand the site more efficiently. Its purpose is simple: when tools like ChatGPT, Claude, or Perplexity process a site, they can find its most important pages in a clean, ordered list instead of wading through HTML full of navigation menus, ads, and JavaScript. The file was proposed in September 2024 by Answer.AI co-founder Jeremy Howard, and it was quickly described as "robots.txt for AI." As of mid-2026, though, the picture is more complicated than the initial excitement suggested: the standard is still a proposal, adoption is low, and the major search engines disagree on it. In this guide I cover what llms.txt is, how to create one, and whether it actually works, using verified data. For the broader framework, see my what is GEO guide; for the full picture of AI search, the what is AI search engine guide is a good starting point.

What Is llms.txt?

llms.txt is a Markdown-formatted guidance file, published at the root of your site (at /llms.txt), that tells AI models which content to prioritize. Its structure is standardized: a single H1 heading with the site name at the top, a blockquote summarizing the site in one sentence just below it, and then lists of links grouped under short descriptions. The idea originates from a real constraint: language models have limited context windows. HTML pages are noisy for a language model, since menus, ads, tracking code, and JavaScript burn through the context budget before the model reaches the actual content. Plain text served as Markdown reduces that load significantly; some documentation platforms reported a clear drop in token usage after serving Markdown instead of HTML. llms.txt follows the same logic: it offers a human-curated content map that tells the model "look at these pages first, the essence of the site is here." The standard also defines an optional llms-full.txt file, which collects the full text of the selected pages into a single Markdown document.

How Does llms.txt Differ From robots.txt and sitemap.xml?

llms.txt belongs to the same family as robots.txt and sitemap.xml, but its function is entirely different: robots.txt manages access permission, sitemap.xml declares crawl coverage, and llms.txt presents selected content with context. The three do not replace one another; they complement one another. robots.txt tells a crawler which paths it may or may not crawl; it allows or blocks, but carries no context about the content. sitemap.xml provides an XML list so machines can discover every URL on your site completely; its goal is indexing coverage. llms.txt, by contrast, is not an index but a reading guide curated with human-written descriptions; its aim is not crawling but easing the language model's inference. The table below places the roles of the three files side by side.

Feature	llms.txt	robots.txt	sitemap.xml
Format	Markdown	Plain text	XML
Location	/llms.txt	/robots.txt	/sitemap.xml
Core purpose	Present selected content with context	Manage crawl access	Expose all URLs for discovery
Audience	Large language models	Search and AI crawlers	Search engine crawlers
Content selection	Human-curated, described	Path-based permission rules	All pages of the site
Status	Proposal, low adoption	Established standard	Established standard

The practical takeaway is this: llms.txt does not replace robots.txt or sitemap.xml. You keep managing access rules with robots.txt and discovery coverage with sitemap.xml; llms.txt adds an optional context layer for AI tools on top of them.

How to Create an llms.txt File: Step by Step

Creating an llms.txt file is technically simple: you add a text file that follows a specific Markdown structure to your root directory. The hard part is not writing the file but deciding correctly which pages truly deserve to be highlighted. The four steps below make the process concrete.

Step 1: Select the Pages to Highlight

Identify not your entire site but the pages a model most needs to understand you correctly. These are usually your main service pages, core guides, about page, and contact details. The value of llms.txt lies in selectivity: if you list every page, the file turns into a copy of sitemap.xml and loses its purpose. The goal is a tidy map that answers "what is this site about, and where are the best sources" in a few seconds.

Step 2: Set Up the Standard Structure

The file follows a specific order: a single H1 with the site name at the top, a blockquote summarizing the site below it, then lists of links grouped under headings. Next to each link you add a short description of what that page contains. A simple skeleton looks like this:

# Abdullah Çalış

> Digital marketing strategist and AI automation architect. Content on GEO, SEO, and automation.

## Guides
- [What is GEO](https://abdullahcalis.com/en/blog/what-is-geo): Generative engine optimization guide
- [What is LLM SEO](https://abdullahcalis.com/en/blog/what-is-llm-seo): Platform-by-platform AI visibility tactics

## Services
- [SEO and GEO services](https://abdullahcalis.com/en/hizmetler/seo): AI visibility audit and strategy

Step 3: Add an Optional llms-full.txt

If you want a model to reach your content in a single request, add an llms-full.txt file. This file merges the full text of your selected pages into one Markdown document, so the model can take in the context without making extra page calls. It is especially useful for products with technical documentation. The downside is size: as content grows, the file can reach tens of thousands of words, so you need to be deliberate about what you include.

Step 4: Validate It and Keep It Current

After publishing, open /llms.txt in your browser to confirm you can reach it as plain text. The most common mistake is creating the file once and forgetting it; whenever you add a new service or an important guide, llms.txt should be updated too. Tracking this maintenance by hand is tedious. In my GEO work I handle llms.txt setup and upkeep within a single system, together with your other AI visibility signals.

Does llms.txt Actually Work? The 2026 Data

As of mid-2026 there is no clear evidence that llms.txt improves AI visibility, though the file does have real value in specific use cases. In a study of nearly 300,000 domains, SE Ranking found that only 10.13% had an llms.txt file, and detected no meaningful relationship between having the file and how often a domain is cited by language models; the researchers reported that removing the llms.txt factor from their machine learning model actually improved its predictions. The search engine side is cautious too: in mid-2025, Gary Illyes of Google's Search Relations team said Google does not support llms.txt and is not planning to, while John Mueller compared the file to the old "keywords meta tag," which was abandoned because sites describing themselves are hard to trust. Multiple independent studies show that AI crawlers such as GPTBot, ClaudeBot, and PerplexityBot mostly skip the file and crawl HTML directly. In short, llms.txt is not a ranking or citation trick; at best it is a small helper.

Who Is llms.txt Valuable For?

The clearest value of llms.txt is not in search visibility but in AI developer tools and documentation sites. When Mintlify enabled llms.txt support across the documentation sites it hosts in late 2024, thousands of sites, including Anthropic and Cursor, gained the file overnight. Coding assistants like Cursor, Continue, Cline, and Aider increasingly look for llms.txt when pointed at a documentation site; MCP servers built for documentation can consume the file directly too. So if your product has technical documentation and you want developers to reach it through AI tools, llms.txt delivers concrete benefit. What makes a brand appear as a source in ChatGPT or Perplexity answers, by contrast, is not llms.txt but the content itself: clear definition paragraphs, independently readable sections, cited data, and a solid technical foundation. That is the real work, and it is called GEO. Setting up llms.txt is a low-cost, low-risk step; the right frame is to see it as a small part of a holistic AI visibility strategy.

Frequently Asked Questions

Does llms.txt affect my SEO rankings?

No, llms.txt does not directly affect your classic Google rankings. Google has stated clearly that it does not use llms.txt as a ranking signal, so adding the file will not lift your position in search results. The purpose of llms.txt is not ranking but making it easier for language models to understand your site. The factors that determine your rankings have not changed: content quality, technical SEO soundness, page speed, and authority. llms.txt does not replace these; at most it stands alongside them.

Can llms.txt and robots.txt be used at the same time?

Yes, the two serve different jobs and are used together. robots.txt manages which crawler may crawl which paths; it grants or blocks access. llms.txt contains no access rules; it only presents highlighted content with context. You keep regulating AI crawler access through robots.txt and add llms.txt on top as a context layer. It is important not to conflate the two: writing access permissions inside llms.txt goes against the purpose of the standard.

Is creating an llms.txt file mandatory?

No, llms.txt is not mandatory and is still a proposed standard. The absence of the file does not stop your site from appearing in AI tools; in fact, the major AI crawlers often skip the file and crawl HTML directly. Still, it is low-cost and low-risk to set up. It is a sensible extra step for sites with technical documentation or those that care about their content being read cleanly by AI tools, but it is not a requirement to place at the top of your priority list.

Does Google use llms.txt?

No. In mid-2025, Gary Illyes of Google's Search Relations team stated that Google does not support llms.txt and is not planning to. Google's John Mueller compared the file to the "keywords meta tag" that was abandoned because it was abused, stressing that a site's self-declared claims cannot be treated as trustworthy without verification. On Google's side, the priority signals remain the classic ones: robots.txt, sitemap.xml, and structured data (schema). llms.txt is not a signal for Google.

What should I focus on first to appear in AI search?

Before llms.txt, focus on content citability and technical SEO. For a language model to cite you as a source, your content needs clear definitions, independently readable sections, and cited data. The whole of this discipline is called GEO, and I covered its platform-level tactics in my LLM SEO guide, the measurement and strategy side in my AI SEO guide, and the Google-specific situation in my Google AI Overview guide. llms.txt is a small complement to this work, not the starting point.

---

Your Next Step

llms.txt is a topic to view with the right frame amid the noise of AI search: not a magic ranking tool, but a helper file with a narrow yet real use case. Setting it up on its own and declaring "I've handled my AI visibility" would be misleading. What genuinely makes a difference is making your content citable by language models, which is GEO work. llms.txt is a small but properly built part of that whole.

If you want to set up a correctly structured llms.txt on your site, review your AI crawler access, and evaluate your content's real visibility potential together, you can browse my SEO and GEO services or set up a strategy call directly.