llms.txt in WordPress: how to add it and whether it’s worth it

Below, we explain what the file looks like, how to add it, and whether it’s worth the effort at all.

What is llms.txt?

This is a proposal by Jeremy Howard from September 2024, described on the llmstxt.org website. The idea is simple: AI models have a limited context window and struggle to navigate pages full of menus, scripts and adverts. The file llms.txt He gives them a brief, clear summary of the key information, with links to pages that are worth reading.

llms.txt is often confused with robots.txt, although they work in opposite ways. robots.txt tells web crawlers what not to visit. llms.txt suggests what is worth reading. If robots.txt blocks AI crawlers, llms.txt won’t make any difference, as the crawler won’t be able to access it.

What should the llms.txt file look like?

The only required element is an H1 heading containing the page title. The rest is optional, but it is this that gives the template its value. The specification sets out the following sections in a fixed order:

This is how the start of our file on aeloapp.io/llms.txt looks: the „# Aelo” header, a single sentence description in quotation marks, a list of key facts (company, measurement method, modules, plan prices, terms and conditions) and sections with links to the product page, knowledge base and blog. The facts in the file are the same as on the website and in structured data. Consistency is more important here than the file itself, because a model that receives conflicting information is less likely to mention the brand, as we explain in the text ChatGPT doesn’t know my company.

How do I add llms.txt in WordPress?

There are three ways to add llms.txt in WordPress, ranging from the simplest to the one offering the most control.

Yoast SEO. Go to Yoast SEO → Settings → Site Features and toggle the llms.txt switch on, then save your changes. This is how you enable llms.txt in WordPress without any code, and the feature is included in the free version of the plugin. By default, Yoast adds the latest posts and pages from the last 12 months to the file, but you can select pages manually in the customisation options. Manual selection is better, as the file should include pages featuring your services, price lists and answers to customer queries, rather than simply the most recent ones.

Manual file. Create the file in a text editor, save it as llms.txt in UTF-8 encoding, and upload it to the WordPress root directory, where the wp-config.php file is located. If you’re using a caching plugin, clear the cache after uploading the file. However, a manually added llms.txt file in WordPress does not update automatically, so you’ll need to remember to update it whenever prices or offers change.

Is the llms.txt file in WordPress useful?

As an affordable add-on, yes; as a way to boost visibility in AI, no. Google states in its documentation on AI features in the search engine that for AI Overviews and AI mode No special files or tags are required. John Mueller from Google compared llms.txt to the ‘keywords’ meta tag, which search engines stopped taking into account a long time ago. OpenAI, Anthropic and Perplexity have not publicly confirmed that their bots use the file when generating responses.

So why do we have it? Because it takes just a few minutes, and it organises what should be consistent anyway: the company description, facts and a list of the most important pages. It can also be useful when someone pastes a web address into a chat or uses a bot that reads the page out loud. But if you have an hour’s visibility in AI, you’re better off spending it on content that answers customers’ questions. That’s what it’s all about AEO, and more broadly GEO.

What works better than llms.txt?

• Access for the bots that the chatbots use for searching: OAI-SearchBot (ChatGPT), Claude-SearchBot and Claude-User (Claude), PerplexityBot (Perplexity) and Googlebot (Gemini, AI Overviews). Check that robots.txt is not blocking them.
• The answer in the first sentence of each section and specific figures: prices, dates, terms and conditions. These are the parts the model quotes most often, as we describe in the text Citations in AI Overviews.
• Structural data for ‘Organisation’ and ‘FAQPage’, consistent with the page’s content.
• Mentions in independent sources, without which a model might be familiar with the brand but would not recommend it. Read more in the article Knows the brand, but doesn’t recommend it.

We describe a simple test that will show whether these changes have made a difference on the page How to find out what AI is saying about your business. You’ll have to wait for the results; we explain how long on our website Why do changes take time?.

How can you check whether models are using your website?

Looking at the citations rather than the file itself. If ChatGPT, Gemini, Claude or Perplexity list your domain as the source of the answer, the bots are reading the page and the content checks out. This figure shows citation index. In Aelo, you can see it in the module Visibility in AI, separately for each model, whilst the website audit within the same module suggests technical improvements, such as structured data and FAQs. Tasks from the audit are sent to Tickets. We carry out the measurement with web search enabled, as described in the text Measurement with web search.

Try it for free

FAQ

Sources

• Jeremy Howard, The /llms.txt file (September 2024): https://llmstxt.org/
• Yoast, How to enable llms.txt with Yoast SEO: https://yoast.com/help/enable-llmstxt/
• Google Search Central, AI features and your website: https://developers.google.com/search/docs/appearance/ai-features
• OpenAI, Overview of OpenAI Crawlers: https://developers.openai.com/api/docs/bots
• Anthropic, the ClaudeBot, Claude-User and Claude-SearchBot bots: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
• Perplexity, Perplexity Crawlers: https://docs.perplexity.ai/guides/bots