Features 🚀
Robots.txt Config
The robots.txt standard is important for search engines to understand which pages to crawl and index.
To match closer to the standard, Nuxt Robots now allows you to configure the module by using a
public/_robots.txt
This file will be parsed and used to configure the module.
If you need programmatic control, you can still configure the module using nuxt.config.ts, Route Rules and Nitro hooks.
Read more at Robots.txt Config.
groups New Config:
- Type:
{ userAgent: []; allow: []; disallow: []; comments: [] }[] - Default:
[]
Define more granular rules for the robots.txt. Each group is a set of rules for specific user agent(s).
export default defineNuxtConfig({
robots: {
groups: [
{
userAgent: ['AdsBot-Google-Mobile', 'AdsBot-Google-Mobile-Apps'],
disallow: ['/admin'],
allow: ['/admin/login'],
comments: 'Allow Google AdsBot to index the login page but no-admin pages'
},
]
}
})
blockNonSeoBots New Config:
- Type:
boolean - Default:
false
Blocks some non-SEO bots from crawling your site. This is not a replacement for a full-blown bot management solution, but it can help to reduce the load on your server.
See const.ts for the list of bots that are blocked.
export default defineNuxtConfig({
robots: {
blockNonSeoBots: true
}
})
Improved header / meta tag integration
Previously, only routes added to the
This has been changed to include all
credits New Config:
- Type:
boolean - Default:
true
Control the module credit comment in the generated robots.txt file.
# START nuxt-robots (indexable) <- credits
...
# END nuxt-robots <- credits
export default defineNuxtConfig({
robots: {
credits: false
}
})
debug New Config:
- Type:
boolean - Default:
false
Enables debug logs.
export default defineNuxtConfig({
robots: {
debug: true
}
})
Deprecations
Nuxt Site Config Integration
The module now integrates with the nuxt-site-config module.
The
For most sites, you won't need to provide any further configuration, everything will just work.
If you need to modify
the default config, the easiest way is to do so through the
export default defineNuxtConfig({
site: {
url: 'https://example.com',
indexable: true
}
})