Google PageSpeed Started Checking llms.txt
Why Google’s new Lighthouse audit for llms.txt is less about SEO today and more about preparing websites for AI agents tomorrow.
Why a small Markdown file suddenly appeared inside Lighthouse audits and whether developers should care.
A quiet change recently appeared in Google Lighthouse.
Among the usual performance, accessibility, SEO, and best practice checks, a new category called Agentic Browsing started showing up. One of its first recommendations is surprisingly simple. Check whether your website exposes a valid llms.txt file.
It doesn't affect Core Web Vitals.
It doesn't improve your Lighthouse score dramatically.
And there is no evidence that it changes search rankings.
So why is Google checking it?
What is llms.txt?
The idea behind llms.txt is similar to robots.txt, but instead of helping search crawlers discover pages, it helps AI systems understand a website's structure.
Rather than crawling hundreds of pages looking for documentation or important resources, an AI agent can read a single Markdown file that points directly to the content that matters.
A typical file might look like this.
JS Tools
Documentation for browser-based developer utilities.
Important Resources
Documentation
API
Guides
Examples
The format intentionally uses Markdown because it is lightweight, human readable, and extremely easy for language models to parse.
Why did Google add this to Lighthouse?
That's the interesting part.
Google has not announced that Search uses llms.txt for indexing.
OpenAI has not confirmed using it inside ChatGPT.
Anthropic has not said Claude relies on it either.
In fact, Google's John Mueller compared the idea to the old keywords meta tag, suggesting that simply creating the file doesn't automatically provide SEO benefits.
So why add an audit?
Most likely because Google sees AI agents becoming a larger part of how people interact with websites.
Traditional SEO focuses on search engines ranking pages.
Agentic Browsing focuses on AI systems navigating websites efficiently.
Those are related problems, but they are not the same.
Where does llms.txt already matter?
Developer tools are currently the strongest use case.
AI coding assistants frequently need to understand project documentation.
When someone asks an assistant how to integrate a library or configure an API, it usually searches documentation before generating an answer.
A well structured llms.txt file gives those assistants a clear starting point instead of forcing them to crawl an entire documentation website.
If your product has API documentation, SDK guides, tutorials, or developer references, adding llms.txt is a sensible investment.
What makes a good llms.txt?
The current recommendations are intentionally simple.
Include a single H1 heading.
Organize content using Markdown headings.
Link only to pages that provide long term value.
Avoid marketing pages or temporary announcements.
A clean example might look like this.
Project Documentation
Getting Started
Installation
Quick Start
Authentication
API Reference
REST API
Webhooks
Rate Limits
Examples
JavaScript
Python
PHP
Think of it as a table of contents for machines rather than for humans.
Who should create one?
The priority depends on the type of website.
Developer platforms, SaaS products, SDKs, and API documentation should probably add it today.
Technical blogs can benefit because AI assistants increasingly reference high quality articles when answering programming questions.
Corporate websites and online stores have less to gain right now because there is still no evidence that llms.txt improves visibility in Google Search.
Checking your Markdown
Since llms.txt is simply a Markdown document, validating its structure is straightforward.
If you're already maintaining documentation in Markdown, converting and exporting files is easy with tools like Markdown to Word, which can also help review larger documentation before publishing.
Keeping headings consistent and avoiding malformed Markdown makes the file easier for both humans and automated systems to process.
Should you spend time on it?
Creating the file usually takes less than fifteen minutes.
That makes it one of the lowest cost improvements you can make for a documentation website.
It probably won't increase traffic tomorrow.
It probably won't move your search rankings.
But it does prepare your content for a future where AI agents consume documentation directly instead of relying entirely on traditional search engines.
Whether that future arrives in six months or several years remains an open question.
The direction, however, is becoming increasingly difficult to ignore.
