The llms.txt standard provides AI assistants with a concise summary of your site's content. Think of it as robots.txt for AI inference—not blocking access, but guiding AI to your most useful documentation.
Jeremy Howard proposed the standard in September 2024. Unlike robots.txt, llms.txt uses Markdown and targets AI tools at inference time rather than crawlers at training time.
llms.txt solves a specific problem: LLM context windows are too small to process entire websites. Your Vue documentation might be thousands of pages, but an AI assistant needs a curated entry point.
llms.txt is useful for:
llms.txt is overkill for:
| Feature | robots.txt | llms.txt |
|---|---|---|
| Purpose | Block/allow crawling | Guide AI to useful content |
| Format | Custom syntax | Markdown |
| When used | Training data collection | Inference (answering questions) |
| Crawler support | All major crawlers | Limited—mostly AI coding tools |
| Required | No | No |
Important: AI crawlers don't currently request llms.txt during inference. GPTBot, ClaudeBot, and PerplexityBot use pre-built datasets and respect robots.txt, not llms.txt.
The primary use case today is AI coding assistants (Cursor, Claude Code) and MCP servers that explicitly fetch llms.txt to understand project documentation.
llms.txt uses a structured Markdown format. Only the H1 title is required—everything else is optional.
# Vue Router Documentation
> Vue Router is the official router for Vue.js. It deeply integrates with Vue.js core to make building Single Page Applications easy.
Key concepts: route matching, nested routes, navigation guards, route meta fields.
## Getting Started
- [Installation](https://router.vuejs.org/installation.html): Install Vue Router via npm
- [Quick Start](https://router.vuejs.org/guide/): Basic router setup
- [Dynamic Routing](https://router.vuejs.org/guide/essentials/dynamic-matching.html): Route params and patterns
## API Reference
- [Router Instance](https://router.vuejs.org/api/#router-instance-methods): push, replace, go, back, forward
- [Route Object](https://router.vuejs.org/api/#the-route-object): params, query, hash, matched
- [Navigation Guards](https://router.vuejs.org/guide/advanced/navigation-guards.html): beforeEach, beforeResolve, afterEach
## Optional
- [Changelog](https://github.com/vuejs/router/blob/main/CHANGELOG.md)
- [Migration from Vue 2](https://router.vuejs.org/guide/migration/)
H1 Title — The only required element. Name of your project or site.
Blockquote — Brief summary with key information for understanding the rest of the file.
Body content — Paragraphs, lists, or any Markdown except headings. Provides context about the project.
H2 sections — File lists with links to detailed documentation. Each entry is a Markdown link with optional description:
- [Link Text](https://url): Optional description of what this page covers
Optional section — An H2 titled "Optional" marks content that AI can skip if context is limited.
The simplest approach: create a static file in your public directory.
public/
llms.txt
# My Vue App
> A Vue 3 application with TypeScript and Vite.
## Documentation
- [API Reference](/docs/api): REST API endpoints
- [Components](/docs/components): Vue component library
- [Getting Started](/docs/setup): Installation and configuration
## Optional
- [Changelog](/changelog)
Your file will be available at https://yoursite.com/llms.txt.
The spec also supports /llms-full.txt for comprehensive documentation when context limits aren't a concern:
# My Vue App - Full Documentation
> Complete documentation including all API endpoints, components, and guides.
[Full content of your docs here, potentially thousands of lines]
AI tools can choose between the concise /llms.txt or comprehensive /llms-full.txt based on their needs.
For large documentation sites, generate llms.txt from your content at build time:
import { readdir, readFile, writeFile } from 'node:fs/promises'
import { join } from 'node:path'
interface DocPage {
title: string
path: string
description?: string
}
async function generateLlmsTxt() {
const docsDir = './docs'
const pages: DocPage[] = []
// Scan docs directory for markdown files
const files = await readdir(docsDir, { recursive: true })
for (const file of files) {
if (!file.endsWith('.md'))
continue
const content = await readFile(join(docsDir, file), 'utf-8')
const titleMatch = content.match(/^#\s+(.+)$/m)
const descMatch = content.match(/^>\s+(.+)$/m)
if (titleMatch) {
pages.push({
title: titleMatch[1],
path: `/docs/${file.replace('.md', '')}`,
description: descMatch?.[1]
})
}
}
// Generate llms.txt content
const llmsTxt = `# My Vue Documentation
> API reference and guides for My Vue App.
## Documentation
${pages.map(p => `- [${p.title}](${p.path})${p.description ? `: ${p.description}` : ''}`).join('\n')}
`
await writeFile('./public/llms.txt', llmsTxt)
}
generateLlmsTxt()
Add to your build process:
{
"scripts": {
"build": "npm run generate:llms && vite build",
"generate:llms": "tsx scripts/generate-llms-txt.ts"
}
}
Several documentation frameworks have llms.txt plugins:
npm install vitepress-plugin-llms
import { defineConfig } from 'vitepress'
import llmstxt from 'vitepress-plugin-llms'
export default defineConfig({
vite: {
plugins: [llmstxt()]
}
})
npm install docusaurus-plugin-llms
export default {
plugins: ['docusaurus-plugin-llms']
}
https://yoursite.com/llms.txtThere's no official validator yet, but the llms-txt-hub directory lists sites implementing the standard.
The standard has 2,000+ GitHub stars and growing adoption among documentation sites. Notable implementers include Answer.AI, fast.ai, and various open source projects.
However, adoption by AI providers is limited. As of late 2025:
The spec is still emerging. Implementing llms.txt today is forward-looking—it positions your docs for better AI integration as adoption grows.
llms.txt complements Generative Engine Optimization but serves a different purpose:
| GEO | llms.txt |
|---|---|
| Optimizes content for AI citations | Provides structured entry point to docs |
| Targets AI search (ChatGPT, Perplexity) | Targets AI coding tools and MCP servers |
| Uses schema.org, content structure | Uses Markdown file format |
| Improves visibility in AI responses | Improves AI understanding of your project |
For maximum AI visibility, implement both:
Nuxt has dedicated support for llms.txt via nuxt-llms:
export default defineNuxtConfig({
modules: ['nuxt-llms'],
llms: {
domain: 'https://example.com',
title: 'My Nuxt App',
description: 'Documentation for My Nuxt App',
sections: [
{
title: 'Getting Started',
links: [
{ title: 'Installation', href: '/docs/installation' },
{ title: 'Configuration', href: '/docs/configuration' }
]
}
]
}
})
Duplicate Content
Duplicate content wastes crawl budget and splits ranking signals. Here's how to find and fix it with canonical tags, redirects, and parameter handling.
SPA SEO
Why Vue SPAs struggle with search engines and how to fix it. Learn when you need SSR, when prerendering works, and when SPA is fine as-is.