An agent answering an API question may need one authentication guide and one endpoint reference. Loading an entire website also brings in material unrelated to that task.
The llms.txt proposal gives publishers a place to describe their content and link to useful Markdown pages.
What goes in the file?
The file contains a project name and can include a summary, guidance and grouped links to further information.
The proposal uses Markdown so people and programs can process the same file. Its link lists point to content the agent can fetch when needed.
A documentation index might group authentication, message sending and delivery events under separate headings. Short descriptions help the agent distinguish those destinations before downloading them.
The proposal also allows indexes below the site root. An index under a documentation path covers the content beneath that path.
Is it the same as robots.txt or a sitemap?
These files serve different purposes: access guidance, URL discovery and a curated route into content.
| File | Purpose |
|---|---|
robots.txt | Gives automated clients crawling rules |
| Sitemap | Lists URLs for search engines to discover |
llms.txt | Describes content and links to useful detail for agents |
The proposal's comparison with existing standards describes llms.txt as complementary to the other files. Publishing it does not grant access to protected content.
It also does not expose an executable action. An agent needs a separate API or MCP connection to perform workspace operations.
Does publishing it prove that agents use it?
Publishing a readable index establishes availability. Evidence of use requires observing a client retrieve and use that content.
Chrome's Lighthouse audit checks for server errors when retrieving the file. A missing file receives a not-applicable result because the file is optional. Passing that check does not measure an agent's answer quality.
For your own evaluation, record the task and the documents the client fetched. Check whether the agent reached the correct answer or completed the intended action. Compare equivalent runs when assessing whether the index helped.
A request in an access log proves a fetch. It does not, by itself, prove that the client used the content in its answer. A citation establishes a different observation again. Keep these outcomes separate when reporting results.
Which Bird documentation should I give an agent?
You can provide Bird's llms.txt index when an agent needs to discover documentation and fetch relevant pages.
The machine-readable documentation guide describes the index, individual Markdown pages and the full corpus. The index supplies links; individual pages let the client retrieve a focused answer.
llms-full.txt concatenates the documentation with source URLs. That larger form suits a workflow that explicitly needs the corpus, subject to the client's context limits.
To perform a messaging task after retrieving the instructions, connect Bird's MCP server. Documentation supplies information; the authenticated connection supplies access to workspace operations.