Guides

Config using Nuxt Config

Copy for LLMs
Last updated by
Harlan Wilton
 in feat: new Content-Usage directive (#226).

If you need programmatic control, you can configure the module using nuxt.config.

Simple Configuration

The simplest configuration is to provide an array of paths to disallow for the * user-agent. If needed you can provide allow pat You can simply add the path or path pattern to hs as well.

  • disallow - An array of paths to disallow for the * user-agent.
  • allow - An array of paths to allow for the * user-agent.
nuxt.config.ts
export default defineNuxtConfig({
  robots: {
    // provide simple disallow rules for all robots `user-agent: *`
    disallow: ['/secret', '/admin'],
    allow: '/admin/login'
  }
})

This will generate the following output:

robots.txt
User-agent: *
Disallow: /secret
Disallow: /admin
Allow: /admin/login

Group Configuration

When targeting specific robots, you can use the groups option to provide granular control.

  • groups - A stack of objects to provide granular control (see below).
nuxt.config.ts
export default defineNuxtConfig({
  // add more granular rules
  robots: {
    groups: [
      // block specific robots from specific pages
      {
        userAgent: ['AdsBot-Google-Mobile', 'AdsBot-Google-Mobile-Apps'],
        disallow: ['/admin'],
        allow: ['/admin/login'],
        comments: 'Allow Google AdsBot to index the login page but no-admin pages'
      },
    ]
  }
})

This will generate the following output:

robots.txt
# Allow Google AdsBot to index the login page but no-admin pages
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google-Mobile-Apps
Disallow: /admin
Allow: /admin/login

Content Signals Configuration

You can configure Content Signals (AI usage preferences) programmatically using the contentUsage option in your groups:

nuxt.config.ts
export default defineNuxtConfig({
  robots: {
    groups: [
      {
        userAgent: '*',
        allow: '/',
        contentUsage: [
          'ai=n', // Disable AI usage globally
          '/docs/ train-ai=y', // Allow AI training on docs
          '/api/ ai=n train-ai=n' // Disable all AI usage for API
        ]
      }
    ]
  }
})

This will generate:

robots.txt
User-agent: *
Allow: /
Content-Usage: ai=n
Content-Usage: /docs/ train-ai=y
Content-Usage: /api/ ai=n train-ai=n

Content-Usage Options

The contentUsage field accepts an array of strings with the following formats:

  • Global preferences: 'ai=n', 'train-ai=y'
  • Path-specific preferences: '/path/ ai=n', '/docs/ train-ai=y'

See the Content Signals guide for more detailed information about Content Signals and supported AI preferences.

Edit this page
Markdown For LLMs
Did this page help you?

Nuxt I18n

How to use the Nuxt Robots module with Nuxt I18n.

Bot Detection

Detect and classify bots with server-side header analysis and client-side browser fingerprinting.