Guides

RAG Content API

Last updated by
Harlan Wilton
in doc: guide skeletons.

Bulk JSONL API exports prerendered pages as newline-delimited JSON chunks, optimized for vector indexing and RAG pipelines.

Endpoint

# Default
curl https://yoursite.com/content.jsonl

# Custom route
curl https://yoursite.com/_ai-ready/bulk

Configuration

// Minimal example
aiReady: { bulkRoute: '/content.jsonl' }

// Full config: /docs/ai-ready/api/config#bulkroute

Served with application/x-ndjson content type.

JSONL Format

Each line is a BulkChunk object:

interface BulkChunk {
  id: string                    // SHA256-based chunk ID
  route: string                 // Page route
  chunkIndex: number            // Chunk position (0-based)
  content: string               // Markdown content
  headers?: Record<string, string>  // Heading hierarchy
  loc?: {                       // Location in HTML
    lines: {
      from: number
      to: number
    }
  }
  title: string                 // Page title
  description: string           // Meta description
}

Example Output

{"id":"a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4y5z6-0","route":"/docs/getting-started","chunkIndex":0,"content":"# Getting Started\n\nNuxt AI Ready makes your website discoverable...","headers":{"h1":"Getting Started"},"loc":{"lines":{"from":1,"to":15}},"title":"Getting Started","description":"Quick start guide"}
{"id":"a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4y5z6-1","route":"/docs/getting-started","chunkIndex":1,"content":"## Installation\n\nInstall the module...","headers":{"h1":"Getting Started","h2":"Installation"},"loc":{"lines":{"from":16,"to":32}},"title":"Getting Started","description":"Quick start guide"}

Usage

Fetch and Parse

async function fetchBulkContent(siteUrl: string) {
  const response = await fetch(`${siteUrl}/content.jsonl`)
  const text = await response.text()
  return text.trim().split('\n').map(line => JSON.parse(line))
}

Reassemble Pages

function reassemblePage(chunks: BulkChunk[], route: string) {
  return chunks
    .filter(chunk => chunk.route === route)
    .sort((a, b) => a.chunkIndex - b.chunkIndex)
    .map(chunk => chunk.content)
    .join('\n\n')
}

Build-Time Hook

nitroApp.hooks.hook('ai-ready:chunk', async (context) => {
  const embedding = await generateEmbedding(context.chunk.content)
  await vectorDb.insert({ id: context.chunk.id, embedding })
})

// Hook details: /docs/ai-ready/api/nuxt-hooks#ai-ready-chunk

Chunk ID Generation

Deterministic SHA256-based IDs ensure stability across rebuilds:

function generateVectorId(route: string, chunkIndex: number): string {
  const hash = createHash('sha256')
    .update(route)
    .digest('hex')
    .substring(0, 48)
  return `${hash}-${chunkIndex}`
}

Same route + index = same ID. Enables efficient incremental updates.

Advanced Configuration

// Chunk size
aiReady: { mdreamOptions: { chunkSize: 512 } }

// Route filtering
nitro: { prerender: { ignore: ['/admin/**'] } }

// Full config: /docs/ai-ready/api/config
Did this page help you?