Index of every page on this site: llms.txt

Use Cloudflare AI Search with Nuxt AI Ready

Connect your published Nuxt pages to Cloudflare AI Search using its website data source. Install Nuxt AI Ready first. Your public pages must render HTML through SSR or prerendering.

AI Search crawls each HTML page, converts it to Markdown, and creates its own search index. Its website crawler does not depend on Nuxt AI Ready's .md URLs or runtime database. Your domain must belong to the same Cloudflare account as the AI Search instance. See the website data source requirements.

Prepare the public pages

Set your production URL and keep content permissions explicit:

nuxt.config.ts
export default defineNuxtConfig({
  modules: ['nuxt-ai-ready'],
  site: {
    url: 'https://example.com',
  },
  aiReady: {
    contentSignal: {
      search: true,
      aiInput: true,
      aiTrain: false,
    },
  },
  sitemap: {
    urls: [
      { loc: '/docs/start', lastmod: '2026-10-01' },
    ],
  },
})

Use your page's actual edit date for lastmod. The fixed date above is illustrative. This website source does not need an MCP server. Set up MCP separately if needed. For CMS pages, return their edit dates through your existing sitemap source. Avoid setting every page's date to the current time on each request. See Nuxt Sitemap sources.

Content Signals describe usage permissions. They do not bypass authentication, bot protection, or Disallow rules. Keep private pages out of the public sitemap and protect them through your application's access controls. See Content Signals.

After deployment, inspect the HTML page, sitemap, and crawler rules:

curl --fail-with-body -H 'Accept: text/html' https://example.com/docs/start
curl --fail-with-body https://example.com/sitemap.xml
curl --fail-with-body https://example.com/robots.txt

The page should return its readable content in HTML. Follow sitemap-index links until you find the page's <loc> and <lastmod> entries. Check the actual sitemap URL if your site uses a custom path or application base path.

The crawler uses Cloudflare-AI-Search. Confirm its matching robots group permits the intended public paths. An ordinary curl request cannot prove that Cloudflare's verified crawler will pass your WAF rules. If crawling receives challenges or blocks, use the verified bot rule in Cloudflare's website crawler guide.

Connect the website data source

In your Cloudflare account, create an AI Search instance and add your website as a data source. Use the dashboard setup guide for the current account workflow.

Choose the sitemap parse type when your sitemap covers the pages you want indexed. AI Search reads the sitemap declarations in robots.txt; you can also configure a specific sitemap URL. It checks source <lastmod> dates during syncs. Without <lastmod>, it uses <changefreq> or its daily fallback. The discover parse type follows links and uses cache age instead of sitemap dates. See AI Search parse types.

Use Static sites rendering when SSR or prerendering already supplies the content in HTML. Use Rendered sites when the crawler needs JavaScript to produce the page content. These options control Cloudflare's fetch method. See rendering modes.

Restrict the source to the public section you want searchable. Path filters match the full URL, so a docs include pattern is **/docs/**. Add exclusions for drafts or other unwanted public content. See path filtering.

After the first sync, inspect the Items and Jobs tabs. Find one expected page and query a distinctive sentence from it. If it is missing, check the page response, sitemap coverage, path filters, and crawler access first.

Refresh after publishing

Publish the changed page and its updated sitemap date before triggering an AI Search sync. Inspect the next job, then query the changed sentence again.

External website sources sync every six hours by default. You can change the interval or trigger a job through the dashboard or Create job API. Triggers have a thirty-second minimum interval. See AI Search syncing.

Nuxt AI Ready runtime indexing and Cloudflare AI Search syncing are separate operations. If you also use the module's MCP tools, refresh its stored pages through runtime indexing. An AI Search sync does not refresh that database.

AI Search can expose its own search endpoint to MCP clients. That endpoint queries Cloudflare's index; Nuxt AI Ready's MCP tools read the module's stored pages. Choose the index your client should use. See AI Search MCP.

Was this page helpful?