llms.txt: the file nobody uses (and that could change everything)

"Has anyone implemented llms.txt? Is it actually useful?" The question generated hundreds of replies on Reddit. On one side, the skeptics: "It's just a glorified robots.txt, no LLM reads it." On the other, early adopters: "Perplexity parses it, I've seen the difference."

We wanted to settle this. We implemented it on 15 client sites. Here's what we observed.

What is llms.txt?

The llms.txt file is a text file placed at the root of your website (like robots.txt). Its purpose: give AI crawlers and LLMs a structured summary of who you are and what your site contains.

Think of it as a cover letter for machines. Instead of forcing an AI to crawl your entire site to understand your company, you hand it a clear, structured document that says: "Here's who we are, what we do, for whom, and where to find our most important content."

The format was proposed by Jeremy Howard (fast.ai founder) and has been gaining traction since late 2025. It's not an official standard -- no W3C specification, no Google endorsement. But it doesn't need to be. What matters is whether AI crawlers actually read it.

Do AI crawlers actually read it?

Based on our testing: yes, some do.

Perplexity's crawler (PerplexityBot) reads llms.txt when present. We confirmed this by adding specific information to our clients' llms.txt files that didn't appear elsewhere on their sites, then checking if Perplexity referenced that information in its responses. In 9 out of 15 cases, it did within 14 days.

ChatGPT's browsing feature appears to check for llms.txt, though less consistently. We saw evidence of it being read in about 40% of cases.

Claude (Anthropic) and Gemini (Google) haven't shown clear evidence of reading llms.txt in our tests. That may change.

What to put in your llms.txt

The format is simple. Markdown-like, human-readable, machine-parseable. Here's the structure we use:

Company name and one-line description. Target audience. Geographic focus. Core services or products (bulleted list). Key differentiators. Important URLs (homepage, blog, pricing, contact). Recent content worth highlighting.

What NOT to put: marketing fluff, superlatives, vague claims. "World-leading innovative solution" tells a machine nothing. "SEO and AI visibility agency specializing in B2B companies in Belgium since 2024" tells it everything.

The results we measured

Across our 15 implementations, here's what we observed over 90 days:

6 clients saw an increase in Perplexity citations within the first month. 3 saw improvement on ChatGPT browsing within 6 weeks. The remaining 6 showed no measurable change -- but they also had weak overall AI visibility to begin with.

The common thread: llms.txt works best as an amplifier, not a fix. If your site already has good content, structured data, and some off-site presence, llms.txt makes it easier for AI to understand and cite you. If your site has fundamental problems, llms.txt alone won't fix them.

Implementation: 20 minutes of work

Create a file called llms.txt at the root of your domain (e.g., yoursite.com/llms.txt). Write it in plain text or Markdown. Keep it under 2,000 words -- concise is better. Update it whenever your offerings or key content change.

That's it. No code changes. No plugin. No developer needed for most setups.

Our verdict

Is llms.txt a game-changer? No. Is it worth 20 minutes of your time? Absolutely.

The cost-benefit ratio is hard to beat. Minimal effort, potential upside with Perplexity and ChatGPT, zero downside. It's one of those rare optimizations where the question isn't "should I do it?" but "why haven't I done it yet?"

The skeptics on Reddit aren't wrong that it's not a magic bullet. But they're wrong that it's useless. In the AI visibility toolkit, llms.txt is a screwdriver -- not a power drill, but you'd be silly to leave it in the box.