Bot Detection · Nuxt Robots · Nuxt SEO

[NuxtSEO](https://nuxtseo.com/ "Home")

- [Modules](https://nuxtseo.com/docs/nuxt-seo/getting-started/introduction)
- [Tools](https://nuxtseo.com/tools)
- [Pro](https://nuxtseo.com/pro)
- [Learn SEO](https://nuxtseo.com/learn-seo/nuxt) [Releases](https://nuxtseo.com/releases)

[1.4K](https://github.com/harlan-zw/nuxt-seo)

[Nuxt SEO on GitHub](https://github.com/harlan-zw/nuxt-seo)

[User Guides](https://nuxtseo.com/docs/robots/getting-started/introduction)

[API](https://nuxtseo.com/docs/robots/api/use-robots-rule)

[Releases](https://nuxtseo.com/docs/robots/releases/v6)

Robots

- [Switch to Robots](https://nuxtseo.com/docs/robots/getting-started/introduction)
- [Switch to Nuxt SEO](https://nuxtseo.com/docs/nuxt-seo/getting-started/introduction)
- [Switch to Sitemap](https://nuxtseo.com/docs/sitemap/getting-started/introduction)
- [Switch to OG Image](https://nuxtseo.com/docs/og-image/getting-started/introduction)
- [Switch to Schema.org](https://nuxtseo.com/docs/schema-org/getting-started/introduction)
- [Switch to Link Checker](https://nuxtseo.com/docs/link-checker/getting-started/introduction)
- [Switch to SEO Utils](https://nuxtseo.com/docs/seo-utils/getting-started/introduction)
- [Switch to Site Config](https://nuxtseo.com/docs/site-config/getting-started/introduction)
- [Switch to Skew Protection](https://nuxtseo.com/docs/skew-protection/getting-started/introduction)
- [Switch to AI Ready](https://nuxtseo.com/docs/ai-ready/getting-started/introduction)

Search…```k`` /`

v6.0.7

- Playgrounds
- [Discord Support](https://discord.com/invite/275MBUBvgP)

### Getting Started

- [Introduction](https://nuxtseo.com/docs/robots/getting-started/introduction)
- [Installation](https://nuxtseo.com/docs/robots/getting-started/installation)
- [Troubleshooting](https://nuxtseo.com/docs/robots/getting-started/troubleshooting)

### Core Concepts

- [Disabling Site Indexing](https://nuxtseo.com/docs/robots/guides/disable-indexing)
- [Disable Page Indexing](https://nuxtseo.com/docs/robots/guides/disable-page-indexing)
- [How Nuxt Robots Works](https://nuxtseo.com/docs/robots/guides/how-it-works)
- [Robots.txt Recipes](https://nuxtseo.com/docs/robots/guides/robot-recipes)
- [Config using Robots.txt](https://nuxtseo.com/docs/robots/guides/robots-txt)
- [Config using Nuxt Config](https://nuxtseo.com/docs/robots/guides/nuxt-config)
- [Config Using Route Rules](https://nuxtseo.com/docs/robots/guides/route-rules)
- [Bot Detection](https://nuxtseo.com/docs/robots/guides/bot-detection)
- [AI Directives](https://nuxtseo.com/docs/robots/guides/ai-directives)

### Advanced

- [Yandex: Clean-param](https://nuxtseo.com/docs/robots/advanced/yandex)
- [Nuxt Content](https://nuxtseo.com/docs/robots/advanced/content)
- [Nuxt I18n](https://nuxtseo.com/docs/robots/advanced/i18n)

Core Concepts

# Bot Detection

[Copy for LLMs](https://nuxtseo.com/docs/robots/guides/bot-detection.md)

## [Introduction](#introduction)

The modern web is full of [bots](https://nuxtseo.com/learn/controlling-crawlers). Start detecting them to better understand your traffic and optimize your Nuxt app for both human users and automated agents.

Nuxt Robots provides bot detection that works on both server and client side, from simple heuristics using HTTP header `User-Agent` checks to advanced client-side detection using [BotD](https://github.com/fingerprintjs/BotD).

## [Bot Categories](#bot-categories)

The module classifies bots into categories based on their intended purpose and trustworthiness. **Trusted bots** are legitimate services that respect robots.txt and provide value to websites, while **untrusted bots** include automation tools, scrapers, and potentially malicious crawlers.

### [Search Engine Bots (trusted)](#search-engine-bots-trusted)

Search engines that index content for public search results and respect standard web protocols.

- **Google**: `googlebot`, `google.com/bot.html`
- **Bing**: `bingbot`, `msnbot`
- **Apple**: `applebot`, `apple.com/go/applebot`
- **DuckDuckGo**: `duckduckbot`, `duckduckgo.com`
- **Baidu**: `baiduspider`, `baidu.com`
- **Yandex**: `yandexbot`
- **Yahoo**: `slurp`

### [Social Media Bots (trusted)](#social-media-bots-trusted)

Social platforms that crawl content for link previews, cards, and social sharing features.

- **Facebook**: `facebookexternalhit`, `facebook.com`
- **Twitter**: `twitterbot`, `twitter`
- **LinkedIn**: `linkedinbot`, `linkedin`
- **Pinterest**: `pinterest`, `pinterestbot`
- **Discord**: `discordbot`, `discordapp`

### [SEO & Analytics Bots (trusted)](#seo-analytics-bots-trusted)

Professional SEO and analytics services that provide legitimate website analysis and insights.

- **Ahrefs**: `ahrefsbot`, `ahrefs.com`
- **Majestic**: `mj12bot`, `majestic12.co.uk/bot`
- **SEMrush**: `semrushbot`, `semrush.com/bot`
- **Screaming Frog**: `screaming frog`, `screamingfrog.co.uk`
- **Moz**: `rogerbot`

### [AI & ML Bots (trusted)](#ai-ml-bots-trusted)

AI companies and research organizations training models or providing AI-powered services.

- **OpenAI**: `gptbot`, `openai.com`
- **Anthropic**: `anthropic`, `claude`
- **Google**: `google-extended` (AI training only), `googlebot-news`
- **Apple**: `applebot-extended` (AI training only)
- **Common Crawl**: `ccbot`, `commoncrawl.org`
- **ByteDance**: `bytespider`
- **Perplexity**: `perplexitybot`, `perplexity.ai`
- **Cohere**: `cohere`, `cohere.com`
- **Diffbot**: `diffbot`

### [Automation Tools (untrusted)](#automation-tools-untrusted)

Browser automation and testing frameworks that may be used for legitimate testing or malicious scraping.

- **Selenium**: `selenium`, `webdriver`
- **Puppeteer**: `puppeteer`, `headless`
- **Playwright**: `playwright`
- **PhantomJS**: `phantomjs`
- **Headless Browser**: `headless`

### [HTTP Tools (untrusted)](#http-tools-untrusted)

Command-line HTTP clients and programmatic request libraries often used for automated data extraction.

- **cURL**: `curl`
- **wget**: `wget`
- **Python Requests**: `python-requests`, `python`

### [Security Scanners (untrusted)](#security-scanners-untrusted)

Network scanning and vulnerability assessment tools that may indicate malicious reconnaissance.

- **Nmap**: `nmap`, `insecure.org`
- **Nikto**: `nikto`
- **ZGrab**: `zgrab`
- **Masscan**: `masscan`
- **WPScan**: `wpscan`

### [Scraping Tools (untrusted)](#scraping-tools-untrusted)

Dedicated web scraping frameworks designed for automated data collection.

- **Scrapy**: `scrapy`, `scrapy.org`
- **Generic Scraper**: `scraper`

Missing a bot? Submit a quick PR :) [View and contribute to bot definitions →](https://github.com/nuxt-modules/robots/blob/main/src/const.ts)

## [Nitro Bot Detection](#nitro-bot-detection)

Since server-side detection only uses HTTP headers, detection can only work for bots that correctly identify themselves in the `User-Agent` header.

You can detect bots inside a Nitro route, middleware, or API handler.

```
import { getBotDetection } from '#robots/server/composables/getBotDetection'

export default defineEventHandler((e) => {
  const detection = getBotDetection(e)

  if (detection.isBot) {
    return { message: \`${detection.botName} bot detected\`, category: detection.botCategory }
  }

  return { message: 'Human user' }
})
```

For full behavior, please consult the [`getBotDetection`](https://nuxtseo.com/docs/robots/nitro-api/get-bot-detection) API docs.

## [Nuxt Bot Detection](#nuxt-bot-detection)

When using bot detection in Nuxt, it will use the `User-Agent` header by default. You can optionally use the [BotD](https://github.com/fingerprintjs/BotD) fingerprinting library to detect advanced automation tools by setting `fingerprint: true`.

```
<script setup lang="ts">
import { useBotDetection } from '#robots/app/composables/useBotDetection'

const { isBot, botName, botCategory, trusted } = useBotDetection({
  fingerprint: true, // detects using botd
})
</script>

<template>
  <div v-if="isBot">
    Bot detected: {{ botName }} ({{ botCategory }})
  </div>
</template>
```

See the [`useBotDetection()`](https://nuxtseo.com/docs/robots/api/use-bot-detection) API docs for full usage details.

## [Fingerprinting with BotD](#fingerprinting-with-botd)

When using `fingerprint: true`, the composable will load the [BotD](https://github.com/fingerprintjs/BotD) library when the window is idle and perform client-side fingerprinting to detect advanced bots and automation tools.

### [Performance Considerations](#performance-considerations)

This fingerprinting is computationally expensive for end users' CPUs, so you should be mindful of when you enable it. For example, you may consider only enabling it for sensitive pages where bot detection is critical.

That said, the composable aims to be performant and will cache the result in the user's local storage under the `'__nuxt_robots:botd'` key when a bot is detected, so the fingerprinting only runs once per detected bot.

```
localStorage.getItem('__nuxt_robots:botd') // returns the cached bot detection result - used internally already
```

### [Watching For Fingerprinting](#watching-for-fingerprinting)

The properties returned from the composable are all `ref`s. It's important to watch these for changes if you're using fingerprinting, as the results will not be immediately available when the composable is called.

```
import { useBotDetection } from '#robots/app/composables/useBotDetection'
import { watch } from 'vue'

const { isBot } = useBotDetection({
  fingerprint: true,
})

watch(isBot, (detected) => {
  if (detected) {
    console.log(\`Bot detected!\`)
  }
})
```

Alternatively you can use the `onFingerprintResult` callback to handle the result when fingerprinting completes.

```
import { useBotDetection } from '#robots/app/composables/useBotDetection'

const botd = useBotDetection({
  fingerprint: true,
  onFingerprintResult(result) {
    // Fingerprinting completed
    console.log('Detection result:', result)
  },
})
```

[Edit this page](https://github.com/nuxt-modules/robots/edit/main/docs/content/2.guides/4.bot-detection.md)

[Markdown For LLMs](https://nuxtseo.com/docs/robots/guides/bot-detection.md)

Did this page help you?

[Config Using Route Rules Learn how to configure robots through route rules.](https://nuxtseo.com/docs/robots/guides/route-rules) [AI Directives Control how AI systems interact with your content using Content-Usage and Content-Signal directives.](https://nuxtseo.com/docs/robots/guides/ai-directives)

On this page

- [Introduction](#introduction)
- [Bot Categories](#bot-categories)
- [Nitro Bot Detection](#nitro-bot-detection)
- [Nuxt Bot Detection](#nuxt-bot-detection)
- [Fingerprinting with BotD](#fingerprinting-with-botd)

[GitHub](https://github.com/harlan-zw/nuxt-seo) [ Discord](https://discord.com/invite/275MBUBvgP)

### [NuxtSEO](https://nuxtseo.com/ "Home")

- [Getting Started](https://nuxtseo.com/docs/nuxt-seo/getting-started/introduction)
- [MCP](https://nuxtseo.com/docs/nuxt-seo/guides/mcp)

Modules

- [Robots](https://nuxtseo.com/docs/robots/getting-started/introduction)
- [Sitemap](https://nuxtseo.com/docs/sitemap/getting-started/introduction)
- [OG Image](https://nuxtseo.com/docs/og-image/getting-started/introduction)
- [Schema.org](https://nuxtseo.com/docs/schema-org/getting-started/introduction)
- [Link Checker](https://nuxtseo.com/docs/link-checker/getting-started/introduction)
- [SEO Utils](https://nuxtseo.com/docs/seo-utils/getting-started/introduction)
- [Site Config](https://nuxtseo.com/docs/site-config/getting-started/introduction)
- [Skew Protection](https://nuxtseo.com/docs/skew-protection/getting-started/introduction)
- [AI Ready](https://nuxtseo.com/docs/ai-ready/getting-started/introduction)

### [NuxtSEO Pro](https://nuxtseo.com/pro "Nuxt SEO Pro")

- [Getting Started](https://nuxtseo.com/pro)
- [Dashboard](https://nuxtseo.com/pro/dashboard)
- [Pro MCP](https://nuxtseo.com/pro/docs/getting-started/mcp-setup)

### [Learn SEO](https://nuxtseo.com/learn-seo "Learn SEO")

Nuxt

- [Mastering Meta](https://nuxtseo.com/learn-seo/nuxt/mastering-meta)
- [Controlling Crawlers](https://nuxtseo.com/learn-seo/nuxt/controlling-crawlers)
- [Launch & Listen](https://nuxtseo.com/learn-seo/nuxt/launch-and-listen)
- [Routes & Rendering](https://nuxtseo.com/learn-seo/nuxt/routes-and-rendering)
- [Staying Secure](https://nuxtseo.com/learn-seo/nuxt/routes-and-rendering/security)

Vue

- [Vue SEO Guide](https://nuxtseo.com/learn-seo/vue)
- [Mastering Meta](https://nuxtseo.com/learn-seo/vue/mastering-meta)
- [Controlling Crawlers](https://nuxtseo.com/learn-seo/vue/controlling-crawlers)
- [SPA SEO](https://nuxtseo.com/learn-seo/vue/spa)
- [SSR Frameworks](https://nuxtseo.com/learn-seo/vue/ssr-frameworks)
- [SEO Checklist](https://nuxtseo.com/learn-seo/checklist)
- [Pre-Launch Warmup](https://nuxtseo.com/learn-seo/pre-launch-warmup)
- [Backlinks & Authority](https://nuxtseo.com/learn-seo/backlinks)

### [Tools](https://nuxtseo.com/tools "SEO Tools")

- [Social Share Debugger](https://nuxtseo.com/tools/social-share-debugger)
- [Robots.txt Generator](https://nuxtseo.com/tools/robots-txt-generator)
- [Meta Tag Checker](https://nuxtseo.com/tools/meta-tag-checker)
- [HTML to Markdown](https://nuxtseo.com/tools/html-to-markdown)
- [XML Sitemap Validator](https://nuxtseo.com/tools/xml-sitemap-validator)
- [Schema.org Validator](https://nuxtseo.com/tools/schema-validator)
- [Keyword Idea Generator](https://nuxtseo.com/tools/keyword-generator)
- [Keyword Research](https://nuxtseo.com/tools/keyword-research)
- [SERP Analyzer](https://nuxtseo.com/tools/serp-analyzer)
- [Domain Rankings](https://nuxtseo.com/tools/domain-rankings)

Copyright © 2023-2026 Harlan Wilton - [MIT License](https://github.com/harlan-zw/nuxt-seo/blob/main/license) · [mdream](https://mdream.dev)