NuxtSEO
Core Concepts

If you need programmatic control, you can configure the module using nuxt.config.

Simple Configuration

The simplest configuration is to provide an array of paths to disallow for the * user-agent. If needed you can provide allow paths as well.

  • disallow - An array of paths to disallow for the * user-agent.
  • allow - An array of paths to allow for the * user-agent.
nuxt.config.ts
export default defineNuxtConfig({
  robots: {
    // provide simple disallow rules for all robots `user-agent: *`
    disallow: ['/secret', '/admin'],
    allow: '/admin/login'
  }
})

This will generate the following output:

robots.txt
User-agent: *
Disallow: /secret
Disallow: /admin
Allow: /admin/login

Group Configuration

When targeting specific robots, you can use the groups option to provide granular control.

  • groups - A stack of objects to provide granular control (see below).
nuxt.config.ts
export default defineNuxtConfig({
  // add more granular rules
  robots: {
    groups: [
      // block specific robots from specific pages
      {
        userAgent: ['AdsBot-Google-Mobile', 'AdsBot-Google-Mobile-Apps'],
        disallow: ['/admin'],
        allow: ['/admin/login'],
        comments: 'Allow Google AdsBot to index the login page but no-admin pages'
      },
    ]
  }
})

This will generate the following output:

robots.txt
# Allow Google AdsBot to index the login page but no-admin pages
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google-Mobile-Apps
Disallow: /admin
Allow: /admin/login

AI Directives Configuration

Configure AI usage preferences using contentUsage and contentSignal:

export default defineNuxtConfig({
  robots: {
    groups: [
      {
        userAgent: '*',
        allow: '/',
        contentUsage: {
          'bots': 'y',
          'train-ai': 'n'
        },
        contentSignal: {
          'ai-train': 'no',
          'search': 'yes'
        }
      }
    ]
  }
})
  • contentUsage - IETF standard: bots, train-ai, ai-output, search with y/n
  • contentSignal - Cloudflare: search, ai-input, ai-train with yes/no
Edit this page
Markdown For LLMs
Did this page help you?

Config using Robots.txt

Configure your generated robots.txt file with a robots.txt file.

Config Using Route Rules

Learn how to configure robots through route rules.