AI Directives · Nuxt Robots · Nuxt SEO

[NuxtSEO](https://nuxtseo.com/ "Home")

- [Modules](https://nuxtseo.com/docs/nuxt-seo/getting-started/introduction)
- [Tools](https://nuxtseo.com/tools)
- [Pro](https://nuxtseo.com/pro)
- [Learn SEO](https://nuxtseo.com/learn-seo/nuxt) [Releases](https://nuxtseo.com/releases)

[1.4K](https://github.com/harlan-zw/nuxt-seo)

[Nuxt SEO on GitHub](https://github.com/harlan-zw/nuxt-seo)

[User Guides](https://nuxtseo.com/docs/robots/getting-started/introduction)

[API](https://nuxtseo.com/docs/robots/api/use-robots-rule)

[Releases](https://nuxtseo.com/docs/robots/releases/v6)

Robots

- [Switch to Robots](https://nuxtseo.com/docs/robots/getting-started/introduction)
- [Switch to Nuxt SEO](https://nuxtseo.com/docs/nuxt-seo/getting-started/introduction)
- [Switch to Sitemap](https://nuxtseo.com/docs/sitemap/getting-started/introduction)
- [Switch to OG Image](https://nuxtseo.com/docs/og-image/getting-started/introduction)
- [Switch to Schema.org](https://nuxtseo.com/docs/schema-org/getting-started/introduction)
- [Switch to Link Checker](https://nuxtseo.com/docs/link-checker/getting-started/introduction)
- [Switch to SEO Utils](https://nuxtseo.com/docs/seo-utils/getting-started/introduction)
- [Switch to Site Config](https://nuxtseo.com/docs/site-config/getting-started/introduction)
- [Switch to Skew Protection](https://nuxtseo.com/docs/skew-protection/getting-started/introduction)
- [Switch to AI Ready](https://nuxtseo.com/docs/ai-ready/getting-started/introduction)

Search…```k`` /`

v6.0.7

- Playgrounds
- [Discord Support](https://discord.com/invite/275MBUBvgP)

### Getting Started

- [Introduction](https://nuxtseo.com/docs/robots/getting-started/introduction)
- [Installation](https://nuxtseo.com/docs/robots/getting-started/installation)
- [Troubleshooting](https://nuxtseo.com/docs/robots/getting-started/troubleshooting)

### Core Concepts

- [Disabling Site Indexing](https://nuxtseo.com/docs/robots/guides/disable-indexing)
- [Disable Page Indexing](https://nuxtseo.com/docs/robots/guides/disable-page-indexing)
- [How Nuxt Robots Works](https://nuxtseo.com/docs/robots/guides/how-it-works)
- [Robots.txt Recipes](https://nuxtseo.com/docs/robots/guides/robot-recipes)
- [Config using Robots.txt](https://nuxtseo.com/docs/robots/guides/robots-txt)
- [Config using Nuxt Config](https://nuxtseo.com/docs/robots/guides/nuxt-config)
- [Config Using Route Rules](https://nuxtseo.com/docs/robots/guides/route-rules)
- [Bot Detection](https://nuxtseo.com/docs/robots/guides/bot-detection)
- [AI Directives](https://nuxtseo.com/docs/robots/guides/ai-directives)

### Advanced

- [Yandex: Clean-param](https://nuxtseo.com/docs/robots/advanced/yandex)
- [Nuxt Content](https://nuxtseo.com/docs/robots/advanced/content)
- [Nuxt I18n](https://nuxtseo.com/docs/robots/advanced/i18n)

Core Concepts

# AI Directives

[Copy for LLMs](https://nuxtseo.com/docs/robots/guides/ai-directives.md)

AI Directives allow you to express preferences about how AI systems, search engines, and automated tools should interact with your content. Two standards are supported:

- **[Content-Usage](https://ietf-wg-aipref.github.io/drafts/draft-ietf-aipref-vocab.html)** - IETF standard with broader automation categories
- **[Content-Signal](https://contentsignals.org/)** - Cloudflare's widely-deployed implementation focused on AI use cases

Both can be used together in your robots.txt file and are enforced through the robots.txt protocol.

[](https://nuxtseo.com/tools/robots-txt-generator)**Test AI crawler blocking** - Our [Robots.txt Generator](https://nuxtseo.com/tools/robots-txt-generator) includes presets for GPTBot, ClaudeBot, and other AI crawlers.

**Important:** AI directives rely on voluntary compliance by crawlers and AI systems. They are not enforced by web servers and should be combined with other protection methods for sensitive content.

## [Content-Usage (IETF aipref-vocab)](#content-usage-ietf-aipref-vocab)

The Content-Usage directive follows the [IETF AI Preferences specification](https://ietf-wg-aipref.github.io/drafts/draft-ietf-aipref-vocab.html), providing a standardized way to express automation preferences.

### [Categories](#categories)

| Category | Description | Example Use Case |
| --- | --- | --- |
| `bots` | Automated Processing | General bot access and crawling |
| `train-ai` | Foundation Model Production | Training large language models |
| `ai-output` | AI Output | AI generated responses and content |
| `search` | Search | Indexing for search results |

### [Values](#values)

- `y` - Allow this category of use
- `n` - Disallow this category of use

### [Syntax](#syntax)

robots.txt

```
User-agent: *
Content-Usage: <category>=<value>[, <category>=<value>]
Content-Usage: /path/ <category>=<value>[, <category>=<value>]
```

### [Examples](#examples)

#### [Block AI Training Globally](#block-ai-training-globally)

robots.txt

```
User-agent: *
Allow: /
Content-Usage: train-ai=n
```

#### [Allow Bots, Block AI Training](#allow-bots-block-ai-training)

robots.txt

```
User-agent: *
Allow: /
Content-Usage: bots=y, train-ai=n
```

#### [Path-Specific Rules](#path-specific-rules)

robots.txt

```
User-agent: *
Allow: /
Content-Usage: train-ai=n
Content-Usage: /docs/ train-ai=y
Content-Usage: /api/ train-ai=n
```

### [Programmatic Configuration](#programmatic-configuration)

**Object Format (Recommended)** - Type-safe with autocomplete:

nuxt.config.ts

```
export default defineNuxtConfig({
  robots: {
    groups: [
      {
        userAgent: '*',
        allow: '/',
        contentUsage: {
          'train-ai': 'n'
        }
      }
    ]
  }
})
```

## [Content-Signal (Cloudflare/IETF aipref-contentsignals)](#content-signal-cloudflareietf-aipref-contentsignals)

Content-Signal is [Cloudflare's implementation](https://blog.cloudflare.com/content-signals-policy/) based on [IETF aipref-contentsignals](https://www.ietf.org/archive/id/draft-romm-aipref-contentsignals-00.html).

### [Categories](#categories-1)

| Category | Description | Example Use Case |
| --- | --- | --- |
| `search` | Search Applications | Indexing for search results and snippets |
| `ai-input` | AI Input | RAG, grounding, generative AI search answers |
| `ai-train` | AI Training | Training or fine-tuning AI models |

### [Values](#values-1)

- `yes` - Allow this category of use
- `no` - Disallow this category of use

### [Syntax](#syntax-1)

robots.txt

```
User-agent: *
Content-Signal: <category>=<value>[, <category>=<value>]
Content-Signal: /path/ <category>=<value>[, <category>=<value>]
```

### [Examples](#examples-1)

#### [Block AI Training, Allow Search](#block-ai-training-allow-search)

robots.txt

```
User-agent: *
Allow: /
Content-Signal: ai-train=no, search=yes
```

#### [Block All AI Usage](#block-all-ai-usage)

robots.txt

```
User-agent: *
Allow: /
Content-Signal: ai-train=no, ai-input=no, search=yes
```

#### [Path-Specific Rules](#path-specific-rules-1)

robots.txt

```
User-agent: *
Allow: /
Content-Signal: ai-train=no, search=yes
Content-Signal: /docs/ ai-input=yes
Content-Signal: /api/ ai-train=no, ai-input=no, search=no
```

### [Programmatic Configuration](#programmatic-configuration-1)

**Object Format (Recommended)** - Type-safe with autocomplete:

nuxt.config.ts

```
export default defineNuxtConfig({
  robots: {
    groups: [
      {
        userAgent: '*',
        allow: '/',
        contentSignal: {
          'ai-train': 'no',
          'search': 'yes'
        }
      }
    ]
  }
})
```

## [Vendor-Specific AI Tokens](#vendor-specific-ai-tokens)

While `Content-Usage` and `Content-Signal` are emerging standards, some major AI providers offer specific User-Agent tokens to opt-out of AI training while maintaining search visibility.

These are highly effective as they are strictly adhered to by their respective companies.

### [Google-Extended](#google-extended)

[Google-Extended](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers#google-extended) is a standalone token that allows you to control whether your content is used to help improve Google's generative AI APIs and services (Gemini, Vertex AI).

- **Does NOT** affect your site's ranking in Google Search.
- **Does NOT** stop Googlebot from crawling your site for indexing.

robots.txt

```
User-agent: Google-Extended
Disallow: /
```

### [Applebot-Extended](#applebot-extended)

[Applebot-Extended](https://support.apple.com/en-us/119829) allows you to opt-out of having your website content used to train Apple's foundation models that power generative AI features across Apple products.

- **Does NOT** affect your site's ranking in Apple Search (Spotlight, Siri).
- **Does NOT** stop Applebot from crawling your site.

robots.txt

```
User-agent: Applebot-Extended
Disallow: /
```

### [Dataset Crawlers](#dataset-crawlers)

Some crawlers are specifically designed to build massive datasets used for training many different AI models. Blocking these can be a broad-stroke approach to AI protection.

- **CCBot (Common Crawl)**: Used by many open-source and commercial models (including early GPT versions).
- **Bytespider**: Crawler for ByteDance (TikTok) AI models.
- **Diffbot**: AI-powered knowledge extraction.

robots.txt

```
User-agent: CCBot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: Diffbot
Disallow: /
```

## [Using Both Together](#using-both-together)

You can use both Content-Usage and Content-Signal in the same robots.txt for comprehensive coverage:

robots.txt

```
User-agent: *
Allow: /
Content-Usage: bots=y, train-ai=n
Content-Signal: ai-train=no, search=yes
```

Object Format (Recommended)

String Format

```
export default defineNuxtConfig({
  robots: {
    groups: [
      {
        userAgent: '*',
        allow: '/',
        contentUsage: {
          'train-ai': 'n'
        },
        contentSignal: {
          'ai-train': 'no',
          'search': 'yes'
        }
      }
    ]
  }
})
```

```
export default defineNuxtConfig({
  robots: {
    groups: [
      {
        userAgent: '*',
        allow: '/',
        contentUsage: ['bots=y, train-ai=n'],
        contentSignal: ['ai-train=no, search=yes']
      }
    ]
  }
})
```

## [Examples](#examples-2)

### [Block All AI Training](#block-all-ai-training)

Content-Usage

Content-Signal

```
User-agent: *
Allow: /
Content-Usage: train-ai=n
```

```
User-agent: *
Allow: /
Content-Signal: ai-train=no
```

### [Documentation-Only Training](#documentation-only-training)

Content-Usage

Content-Signal

```
User-agent: *
Allow: /
Content-Usage: train-ai=n
Content-Usage: /docs/ train-ai=y
```

```
User-agent: *
Allow: /
Content-Signal: ai-train=no
Content-Signal: /docs/ ai-train=yes
```

[Edit this page](https://github.com/nuxt-modules/robots/edit/main/docs/content/2.guides/5.ai-directives.md)

[Markdown For LLMs](https://nuxtseo.com/docs/robots/guides/ai-directives.md)

Did this page help you?

[Bot Detection Detect and classify bots with server-side header analysis and client-side browser fingerprinting.](https://nuxtseo.com/docs/robots/guides/bot-detection) [Yandex: Clean-param Learn how to use the `clean-param` directive to remove query parameters from URLs with Yandex.](https://nuxtseo.com/docs/robots/advanced/yandex)

On this page

- [Content-Usage (IETF aipref-vocab)](#content-usage-ietf-aipref-vocab)
- [Content-Signal (Cloudflare/IETF aipref-contentsignals)](#content-signal-cloudflareietf-aipref-contentsignals)
- [Vendor-Specific AI Tokens](#vendor-specific-ai-tokens)
- [Using Both Together](#using-both-together)
- [Examples](#examples-2)

[GitHub](https://github.com/harlan-zw/nuxt-seo) [ Discord](https://discord.com/invite/275MBUBvgP)

### [NuxtSEO](https://nuxtseo.com/ "Home")

- [Getting Started](https://nuxtseo.com/docs/nuxt-seo/getting-started/introduction)
- [MCP](https://nuxtseo.com/docs/nuxt-seo/guides/mcp)

Modules

- [Robots](https://nuxtseo.com/docs/robots/getting-started/introduction)
- [Sitemap](https://nuxtseo.com/docs/sitemap/getting-started/introduction)
- [OG Image](https://nuxtseo.com/docs/og-image/getting-started/introduction)
- [Schema.org](https://nuxtseo.com/docs/schema-org/getting-started/introduction)
- [Link Checker](https://nuxtseo.com/docs/link-checker/getting-started/introduction)
- [SEO Utils](https://nuxtseo.com/docs/seo-utils/getting-started/introduction)
- [Site Config](https://nuxtseo.com/docs/site-config/getting-started/introduction)
- [Skew Protection](https://nuxtseo.com/docs/skew-protection/getting-started/introduction)
- [AI Ready](https://nuxtseo.com/docs/ai-ready/getting-started/introduction)

### [NuxtSEO Pro](https://nuxtseo.com/pro "Home")

- [Getting Started](https://nuxtseo.com/pro)
- [Dashboard](https://nuxtseo.com/pro/dashboard)
- [Pro MCP](https://nuxtseo.com/docs/nuxt-seo-pro/mcp/installation)

### [Learn SEO](https://nuxtseo.com/learn-seo "Learn SEO")

Nuxt

- [Mastering Meta](https://nuxtseo.com/learn-seo/nuxt/mastering-meta)
- [Controlling Crawlers](https://nuxtseo.com/learn-seo/nuxt/controlling-crawlers)
- [Launch & Listen](https://nuxtseo.com/learn-seo/nuxt/launch-and-listen)
- [Routes & Rendering](https://nuxtseo.com/learn-seo/nuxt/routes-and-rendering)
- [Staying Secure](https://nuxtseo.com/learn-seo/nuxt/routes-and-rendering/security)

Vue

- [Vue SEO Guide](https://nuxtseo.com/learn-seo/vue)
- [Mastering Meta](https://nuxtseo.com/learn-seo/vue/mastering-meta)
- [Controlling Crawlers](https://nuxtseo.com/learn-seo/vue/controlling-crawlers)
- [SPA SEO](https://nuxtseo.com/learn-seo/vue/spa)
- [SSR Frameworks](https://nuxtseo.com/learn-seo/vue/ssr-frameworks)
- [SEO Checklist](https://nuxtseo.com/learn-seo/checklist)
- [Pre-Launch Warmup](https://nuxtseo.com/learn-seo/pre-launch-warmup)
- [Backlinks & Authority](https://nuxtseo.com/learn-seo/backlinks)

### [Tools](https://nuxtseo.com/tools "SEO Tools")

- [Social Share Debugger](https://nuxtseo.com/tools/social-share-debugger)
- [Robots.txt Generator](https://nuxtseo.com/tools/robots-txt-generator)
- [Meta Tag Checker](https://nuxtseo.com/tools/meta-tag-checker)
- [HTML to Markdown](https://nuxtseo.com/tools/html-to-markdown)
- [XML Sitemap Validator](https://nuxtseo.com/tools/xml-sitemap-validator)
- [Schema.org Validator](https://nuxtseo.com/tools/schema-validator)
- [Keyword Idea Generator](https://nuxtseo.com/tools/keyword-generator)
- [Keyword Research](https://nuxtseo.com/tools/keyword-research)
- [SERP Analyzer](https://nuxtseo.com/tools/serp-analyzer)
- [Domain Rankings](https://nuxtseo.com/tools/domain-rankings)

Copyright © 2023-2026 Harlan Wilton - [MIT License](https://github.com/harlan-zw/nuxt-seo/blob/main/license) · [mdream](https://mdream.dev)