LLMs.txt Checker

What this llms.txt checker actually tests

There are already a dozen or so tools that will parse your file and tell you whether the markdown is shaped correctly. They all check the same handful of things, because the spec is short and there is not much to check. Format validation is table stakes at this point.

The gap is on the delivery side. A file can be flawless markdown and still be unreachable, blocked, or served with a header that makes the consumer choke. None of that shows up in a parser. So this tool runs the format rules from the spec, and then separately asks whether the file survives the trip: what status code comes back, what Content-Type the server attaches, whether robots.txt disallows the path, and whether a 200 response is secretly an HTML error page.

Every finding is tagged with where the rule came from. Rules marked spec are quoted from llmstxt.org. Rules marked best practice are our opinion, and we say so rather than dressing them up as violations. If you disagree with one, ignore it.

Why a valid file still does nothing

This is the part most tools leave out, and it is the part that decides whether the file was worth your afternoon.

John Mueller at Google has said that none of the AI services have claimed to use llms.txt, and that you can tell from server logs they do not even request it. Gary Illyes has said Google does not support it and is not planning to. Google's own guidance on showing up in AI features, updated in June 2026, says outright that you do not need to create machine readable files or markdown for Google Search, because Search does not use them.

That does not make the file harmful. It costs almost nothing to leave a valid one at your root, and if adoption ever arrives you are already done. It does mean that if you are choosing between fixing your llms.txt and fixing anything a crawler demonstrably reads, the llms.txt loses.

The Content-Type trap that breaks ChatGPT and Gemini

Here is the one failure that has an actual reproduction behind it rather than a theory. A developer writing up why their llms.txt was not working found that the difference came down to a header. Deployments serving the file as text/plain with a utf-8 charset worked in both ChatGPT and Gemini. Deployments serving the identical bytes as text/markdown with a utf-8 charset failed in both.

Nothing about the file content changed. Only the header did. This is easy to miss because your browser renders both cases the same way, and every markdown parser accepts both, so the file looks correct everywhere you would normally look.

The checker above prints the exact header your server sends. If you see text/markdown, that is the first thing to change. For other values we say the behaviour is unverified rather than guessing, because that one write-up is the only reproduction we found and it only covered those two cases.

The llmstxt.org format in plain terms

The spec is smaller than people expect. In order: an optional byte order mark, then an H1 with the project name, then a blockquote summarising the project, then any number of markdown blocks that are not headings, then any number of H2 sections whose contents are link lists. Each link is a markdown hyperlink, optionally followed by a colon and a note.

The H1 is the only element the spec describes as required. Everything else is convention, which is why this checker marks a missing blockquote as a warning instead of a failure. An H2 named Optional is special: it means the links under it can be skipped when a shorter context is needed. Stripe uses it exactly this way, parking support and changelog links at the end.

There is no JSON schema and no formal grammar published alongside the spec. The prose is the spec. That is worth knowing when a tool tells you something is invalid, because a good deal of what gets called invalid is somebody's interpretation.

We ran the two files everyone cites as the good examples through the checker on 17 July 2026. Cloudflare came back at 92 out of 100 across 9 sections and 103 links, with one deviation: a plain sentence sits between the H1 and the summary blockquote, where the spec puts the blockquote first. Stripe came back at 67 across 26 sections and 472 links, with five list items that are sentences containing links rather than the link items the spec asks for, and one bullet that is just a sentence with no link in it at all.

Neither file is broken and neither company is doing anything wrong. Both are perfectly readable. It is a reminder that the reference implementations of this format do not follow it exactly either, so treat a warning as information rather than a verdict.

What crawler logs actually show

Statements from search engineers are one thing. Logs are better. An audit across roughly a thousand domains looked for AI crawler requests to llms.txt and found none from GPTBot, ClaudeBot or PerplexityBot. The overwhelming majority of hits on those files came from ordinary Google crawling, not from anything AI-specific.

The follow-up a year later counted 258 requests that could be attributed to an LLM across thirty days and thousands of domains. About 60 percent of all traffic to llms.txt files came from bots that announce themselves as auditors, monitors or readiness checkers.

Read that last number again, because it is funnier than it is useful. The tools checking llms.txt are the main thing reading llms.txt. This one included.

So what is worth your afternoon instead? Structured data, which search engines document and act on. The Schema Markup Generator builds the JSON-LD, and the Robots.txt Tester shows whether GPTBot and ClaudeBot are allowed to fetch the pages you actually care about. Those two are read by the crawlers that show up in your logs.

Questions people actually ask

Is llms.txt a real thing now?

It is a real format with a real spec, published by Jeremy Howard and AnswerDotAI in September 2024. It is not a standard in the W3C or IETF sense, and no search engine has adopted it. Cloudflare, Stripe and Mintlify ship one, so the format is real in the sense that people use it. Whether anything reads it is a separate question, and the honest answer today is mostly no.

Why is my llms.txt not working in ChatGPT or Gemini?

The most common cause anyone has actually reproduced is the Content-Type header. One developer found that serving the file as text/plain worked in both ChatGPT and Gemini, while serving the identical file as text/markdown failed in both. If your file is valid markdown and still fails, check the header before you rewrite the content. This checker reports the exact header your server sends.

Do LLMs actually read llms.txt?

Almost never, based on the evidence available. Google says it does not use the file and has no plans to. A log audit across about a thousand domains recorded no GPTBot, ClaudeBot or PerplexityBot requests for it. Around 60 percent of the requests these files receive come from readiness checkers rather than from the assistants themselves.

What does an llms.txt checker actually validate?

Most of them parse the markdown and check three things: that an H1 exists, that a summary blockquote follows it, and that the links are well formed. That covers the format and stops there. This one adds the delivery side, so the HTTP status, the Content-Type header, whether robots.txt blocks the path, and whether the server is quietly returning an HTML error page instead of a text file.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they may not fetch, and every major crawler honours it. llms.txt tries to tell assistants what is worth reading, and no major assistant has committed to using it. One is an enforced convention with decades of adoption behind it. The other is a proposal that is still waiting for its audience.

How do I check if my llms.txt returns HTTP 200?

Paste your domain above and the checker reports the status code, the final URL after redirects, the response time and the byte size. Watch for a 200 that carries HTML, which means a catch-all route is answering instead of the file. That case looks fine in a browser and fails for anything parsing the response.

Should I add llms-full.txt as well?

It is optional, and it is not in the official spec. The convention comes from generators like Mintlify, where llms.txt lists links and llms-full.txt inlines the whole documentation body into one file. If you already have llms.txt, adding the full variant is a small job. Given how little either file is read right now, it is hard to argue it deserves priority.