Disabling Site Indexing

Introduction

Disabling certain environments of your site from being indexed is an important practice to avoid SEO issues.

For example, you don't want your staging or preview environments to be indexed by search engines as they will cause duplicate content issues as well as confuse end-users.

If you need to disable specific pages from being indexed, refer to the Disabling Page Indexing guide.

Disable Indexing Completely

In some cases, such as internal business tools, or sites that are not ready for the public, you may want to disable indexing completely.

You can achieve this by setting the indexable option to false in your site config.

export default defineNuxtConfig({
  site: { indexable: false }
})

Handling Staging Environments

Staging environments are great for testing out code before it goes to production. However, we definitely don't want search engines to index them.

To control the indexing of these sites we will make use of the env Site Config, which defaults to production.

NUXT_SITE_ENV=staging

Nuxt Robots will disable indexing for any sites which don't have a production environment, so feel free to set this to whatever makes sense for your project.

Verifying Indexing

To verify that your site is being not being indexed, you can check the generated robots.txt file, it should look something like this.

User-agent: *
Disallow: /

A robots meta tag should also be generated that looks like:

<meta name="robots" content="noindex, nofollow">

For full confidence you can inspect the URL within Google Search Console to see if it's being indexed.