---
title: "RAG Setup"
description: "Vectorize your site's markdown for semantic search and RAG pipelines."
canonical_url: "https://nuxtseo.com/docs/ai-ready/advanced/rag-example"
last_updated: "2026-09-25T20:11:52.698Z"
---

Use `llms-full.txt` to build a small vector-search example from your published pages.
Nuxt AI Ready exports the Markdown. Your script creates embeddings and stores them separately.

The example uses paid OpenAI API calls and an in-memory sqlite-vec database. Run it on a small set of short pages first.
It retrieves context for RAG; it does not generate an answer or provide a production ingestion service.

Install the script dependencies and set `OPENAI_API_KEY` in your environment:

```bash
pnpm add -D openai better-sqlite3 sqlite-vec tsx @types/better-sqlite3
```

Put the Fetch, Generate, sqlite-vec, and Query examples below in one `scripts/vectorize.ts` file.
Use Upstash instead of the [SQLite](https://sqlite.org) sections only if you already have an Upstash Vector index.

## Fetch Markdown Content

Each page starts with a `---` boundary and metadata. The exporter removes leading YAML frontmatter and trims the Markdown body.
This parser targets Nuxt AI Ready’s export format, not arbitrary Markdown. It checks the response and required metadata:

```ts
const RE_PAGE_BOUNDARY = /^---\n\n(?=- \*\*Page:\*\* )/gm
const RE_METADATA = /^- \*\*(Page|Source|Description):\*\* (.*)$/

export function parseLlmsFullTxt(content: string, llmsFullUrl: string) {
  const [, ...pageBlocks] = content.replace(/\r\n/g, '\n').split(RE_PAGE_BOUNDARY)
  return pageBlocks.map((block) => {
    const [metadataBlock = '', ...markdownParts] = block.trim().split('\n\n')
    const metadata = Object.fromEntries(
      metadataBlock.split('\n').flatMap((line) => {
        const match = RE_METADATA.exec(line)
        return match ? [[match[1], match[2]]] : []
      }),
    )
    if (!metadata.Page || !metadata.Source)
      throw new Error('A page needs Page and Source metadata.')

    const source = new URL(metadata.Source, llmsFullUrl)
    if (!['http:', 'https:'].includes(source.protocol))
      throw new Error('Page sources must use HTTP or HTTPS.')

    return {
      title: metadata.Page,
      description: metadata.Description || '',
      source: source.href,
      route: source.pathname,
      markdown: markdownParts.join('\n\n').trim(),
    }
  }).filter(page => page.markdown.length > 0)
}

const llmsFullUrl = 'https://yoursite.com/llms-full.txt'
const response = await fetch(llmsFullUrl)
if (!response.ok)
  throw new Error(`Markdown export request failed: ${response.status}`)

const pages = parseLlmsFullTxt(await response.text(), llmsFullUrl)
if (pages.length === 0)
  throw new Error('The export contains no non-empty pages.')
```

## Generate Embeddings

This example uses `text-embedding-3-small`, whose default vectors have 1,536 dimensions.
Keep that dimension consistent with your vector table. See [OpenAI’s embedding guide](https://developers.openai.com/api/docs/guides/embeddings).

Each input must fit the model’s token limit. Split long pages before this step; a heading split alone does not guarantee that limit.
Calls run sequentially here. A provider error stops the script; it does not retry or resume automatically.

```ts
import OpenAI from 'openai'

const openai = new OpenAI()

async function embed(text: string) {
  const response = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: text,
  })
  const embedding = response.data[0]?.embedding
  if (!embedding)
    throw new Error('The embedding response contains no vector.')
  return embedding
}

const vectors = []
for (const page of pages) {
  vectors.push({
    id: page.source,
    embedding: await embed(page.markdown),
    metadata: { title: page.title, source: page.source },
  })
}
```

## Store in Vector DB

### sqlite-vec (Local)

This table exists only while the script runs. Use it to test retrieval before choosing persistent storage.
The [sqlite-vec JavaScript guide](https://alexgarcia.xyz/sqlite-vec/js.html) covers loading the extension.

```ts
import Database from 'better-sqlite3'
import * as sqliteVec from 'sqlite-vec'

const db = new Database(':memory:')
sqliteVec.load(db)

db.exec(`
  CREATE VIRTUAL TABLE pages USING vec0(
    id TEXT PRIMARY KEY,
    embedding FLOAT[1536]
  )
`)

const insert = db.prepare('INSERT INTO pages VALUES (?, ?)')
for (const v of vectors) {
  insert.run(v.id, new Float32Array(v.embedding))
}
```

### Upstash Vector (Serverless)

This is an alternative to SQLite, not another step in the local example.
Install `@upstash/vector` and set `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` for your index.
Its configured dimensions must match the embedding model. See [Upstash’s upsert reference](https://upstash.com/docs/vector/sdks/ts/commands/upsert).

```ts
import { Index } from '@upstash/vector'

const index = new Index()

await index.upsert(vectors.map(v => ({
  id: v.id,
  vector: v.embedding,
  metadata: v.metadata
})))
```

## Query

For the local SQLite example, retrieve the closest vectors and recover their source Markdown.
The `k` constraint uses sqlite-vec’s [KNN query syntax](https://alexgarcia.xyz/sqlite-vec/features/knn.html).

```ts
async function search(query: string, topK = 5) {
  const queryEmbedding = await embed(query)

  const results = db.prepare(`
    SELECT id, distance
    FROM pages
    WHERE embedding MATCH ? AND k = ?
    ORDER BY distance
  `).all(new Float32Array(queryEmbedding), topK) as { id: string, distance: number }[]

  return results
}

const relevant = await search('how do I configure meta tags?')
const context = relevant.map((result) => {
  const page = pages.find(page => page.source === result.id)
  return page ? `Source: ${page.source}\n\n${page.markdown}` : ''
}).filter(Boolean).join('\n\n')
console.log(context)
db.close()
```

## Chunking Strategy

The example embeds one short page at a time. For larger pages, create smaller inputs before generating embeddings.
This simple heading splitter preserves heading text. Use a Markdown parser if headings can appear inside code fences:

```ts
const RE_HEADING_SPLIT = /(?=^##\s)/m

function chunkByHeading(markdown: string, route: string) {
  const sections = markdown.split(RE_HEADING_SPLIT)
  return sections.filter(section => section.trim()).map((section, i) => ({
    id: `${route}#${i}`,
    content: section.trim(),
    route
  }))
}
```

| Strategy       | When to use                                            |
| -------------- | ------------------------------------------------------ |
| Page-level     | Short pages that fit the embedding model’s input limit |
| Heading-level  | Long docs, precise retrieval needed                    |
| Sliding window | Dense technical content, overlap matters               |

## Build Script

After prerendering, you can read the local export instead of fetching the deployed URL:

```ts
// scripts/vectorize.ts
import { readFileSync } from 'node:fs'

const llmsFull = readFileSync('.output/public/llms-full.txt', 'utf-8')
const pages = parseLlmsFullTxt(llmsFull, 'https://yoursite.com/llms-full.txt')
```

Replace the fetch block with that local read. Keep `parseLlmsFullTxt` and the remaining example steps.
Run it after the static files exist:

```json
{
  "scripts": {
    "generate": "nuxt generate && tsx scripts/vectorize.ts"
  }
}
```

## Sitemap

See the full [sitemap](/sitemap.md) for all pages.