What is llms.txt and does your site need one?

Anyone who looks into visibility through AI soon comes across the term llms.txt. Some providers present it as a mandatory part of every site that wants to appear in ChatGPT. That is overstating it. Below you can read what the file is, what belongs in it and how we assess it for a Flemish SME.

Where llms.txt comes from

llms.txt is a proposal by Jeremy Howard, published on llmstxt.org in September 2024. It starts from a simple observation: web pages are built for people, with menus, banners, cookie notices and scripts. A language model that wants to understand quickly what a site is about is better served by a concise overview in one fixed place.

The proposal is that you publish a markdown file at yourdomain.be/llms.txt, alongside the familiar robots.txt. Where robots.txt says what a crawler may read, llms.txt aims to say what is worth reading.

What goes in it

The proposal describes a fixed structure in markdown:

• a heading with the name of your business or project, the only required part
• a short summary in a blockquote, with the key information needed to understand the rest
• optionally a few paragraphs with further explanation
• lists of links to your most important pages, grouped under subheadings, each with a short description

The proposal also recommends offering a clean markdown version of important pages, at the same address with .md appended. For an SME that is usually a step too far. A good llms.txt file with the right links is enough as a starting point.

For an installation company it could look like this:

```

Voorbeeld Installaties

Installer of heat pumps and solar panels for households and SMEs in East Flanders, founded in 2009.

Services

• Heat pumps: installation and maintenance
• Solar panels: design and installation

About us

• Contact: address, phone, opening hours

```

What it has been proven to deliver

Here we have to be honest: it has not been shown that any major AI service uses llms.txt to decide which sources make it into an answer. In the Google and OpenAI documentation we checked, llms.txt is nowhere listed as a factor for their search features.

Google is the clearest. In its explanation of AI features, Google writes that you do not need to create new machine-readable files, AI text files or special markup to appear in AI Overviews or AI Mode. What counts is that your page is indexed and eligible to appear with a snippet in search results.

OpenAI does not mention llms.txt in its documentation on crawlers either. What it does say: OAI-SearchBot must have access to your site, otherwise you will not appear in ChatGPT's search answers.

llmstxt.org states that major AI companies, among others, publish an llms.txt for their own documentation. That says something about its usefulness for developers searching technical documentation with an AI assistant. It says nothing about how a search feature chooses sources for a question like "which accountant in Aalst would you recommend?".

Why we often add it anyway

Despite that uncertainty, we usually recommend the file. The reasons are down to earth:

• It costs little. You can write a good file for an SME site in an hour. It requires no technical intervention on your site.
• It does no harm. An AI service that ignores the file notices nothing.
• It forces clarity. Anyone who has to write a three-line summary of their business often discovers that those sentences are missing from the homepage itself. That insight is worth more than the file.
• It is ready if the standard takes off. If a major service later decides to use the file, you are already there.

So see llms.txt as a small, cheap signal without a guarantee. It comes at the very end of the list, after the things the providers themselves document or that demonstrably help: giving crawlers access, writing clear content, gathering mentions on other sites and adding structured data.

When you are better off without it

If you have no time to keep the file up to date, leave it out. An llms.txt that points to pages that no longer exist or to old services gives an AI service exactly the wrong information. That risk outweighs the uncertain benefit. The file therefore belongs in the same maintenance routine as your site itself: if a service or an address changes, you update it in llms.txt too.

Working from Ghent for SMEs across Flanders, we always write the file in Dutch, in the same wording as the site, so that an AI service does not come across two versions of the same facts.

What you can do now

• Check whether your site already has an llms.txt by opening yourdomain.be/llms.txt.
• Write three sentences that say what your business does, for whom and in which region. If those sentences are not on your homepage, put them there first.
• Make a list of your five most important pages with one line of explanation per page. With that, you already have the content of an llms.txt.

In our one-off optimisation we write and add the file together with the measures that carry more weight. If you want to rework your website itself, our sister agency hetdesignbureau.be can help you further.