Re-embedding a Knowledge Base (Switching the Embedding Model)

Overview

Re-embedding regenerates the vector embeddings for all of a company's ingested content using a different embedding model, then atomically switches the company's live vector collection to the new model. Use it when you need to:

• Upgrade to a higher-quality / higher-dimensional embedding model (e.g. text-embedding-ada-002 → text-embedding-3-large).

• Change the embedding dimension of a tenant's vector store.

• Re-baseline embeddings after a model or provider change.

What it does under the hood (operation type SWITCH_EMBEDDING_MODEL):

1. preparing — creates a new, temporary vector collection sized for the new model's dimension.

2. reembedding — streams every chunk, re-embeds it with the new model, and writes the vectors into the new collection (in batches).

3. swap — once the new collection is complete, atomically points the company's live collection at it and updates the relational store.

4. cleanup — removes the old collection.

The job is checkpointed (a worker restart resumes from the last position) and non-destructive by default: the old collection and live search keep working throughout the run, and the swap only happens at the very end.

Who can run it

• Permission:chat.admin.all (Zitadel) and CONTENT:MANAGE.

• Scope: the companyId is taken from the authenticated admin user — you run it as an admin of the company being re-embedded.

Before you start (prerequisites)

Step 1 — Dry run (preview, makes no changes)

mutation {
  createMaintenanceJob(input: {
    operationType: "SWITCH_EMBEDDING_MODEL"
    newEmbeddingModel: "text-embedding-3-large"
    newEmbeddingDimension: 3072
    dryRun: true
  }) { id status }
}

A dry run validates the request, estimates the number of chunks to process, and completes without changing any data. Read the job back (Step 3) — its result contains the plan, including estimatedChunks, the current collection, and the target model/dimension. Use the estimate to size the run (see Sizing).

Step 2 — Run the switch

mutation {
  createMaintenanceJob(input: {
    operationType: "SWITCH_EMBEDDING_MODEL"
    newEmbeddingModel: "text-embedding-3-large"
    newEmbeddingDimension: 3072
    destructive: false
    dryRun: false
  }) { id status createdAt }
}

Input fields

Invoking it (cURL) — authenticate with a chat.admin.all bearer token; companyId is derived from the token:

curl -X POST "$INGESTION_GRAPHQL_URL" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"mutation { createMaintenanceJob(input:{operationType:\"SWITCH_EMBEDDING_MODEL\",newEmbeddingModel:\"text-embedding-3-large\",newEmbeddingDimension:3072,destructive:false,dryRun:false}){ id status } }"}'

Step 3 — Monitor

A background worker claims the job within ~60 seconds of creation. Track it with:

query {
  maintenanceJob(jobId: "mjob_xxx") {
    id status phase processed total failed errorMessage startedAt completedAt result
  }
}

List all jobs for the company with maintenanceJobs { id status phase processed total operationType createdAt }.

Cancel a running job with cancelMaintenanceJob(jobId: "mjob_xxx").

Sizing & duration

Throughput is bounded by the embedding model's tokens-per-minute (TPM) quota:

estimated minutes  ≈  (total_chunks × avg_tokens_per_chunk) ÷ TPM_quota
chunks per minute  ≈  TPM_quota ÷ avg_tokens_per_chunk

Worked example: ~750,000 TPM quota, ~400 tokens/chunk → ~1,800 chunks/min → a 1.9M-chunk tenant ≈ ~17 hours. A few-thousand-TPM quota would make the same tenant take many days — raise the quota for large tenants.

Re-embedding is processed sequentially (one batch at a time). The single biggest lever on duration is the embedding deployment's TPM quota.

Safety, atomicity & rollback

• Non-destructive by default: users keep searching on the existing model for the entire run; the switch is atomic at the very end. If the job fails before the swap, nothing changes for users.

• Checkpoint / resume: progress is recorded continuously; a worker restart resumes from the last checkpoint rather than starting over.

• destructive: true deletes the old collection up front — only use it when you cannot keep both collections and you accept that there is no rollback.

• Rollback after a completed switch: run the switch again, targeting the previous model and dimension (this is a full re-embed back to the old model).

Troubleshooting

Configuration reference

Known limitations

• One maintenance job per company at a time (enforced by a per-company lock).

• A FAILED job is not auto-resumed — re-trigger it to restart (it re-embeds into a fresh collection; the partial collection from the failed run is garbage-collected).

• Runtime scales with corpus size and embedding quota — large tenants take hours; plan accordingly and monitor.

This runbook documents the SWITCH_EMBEDDING_MODEL maintenance operation. Two sibling operations (RESHARD_COLLECTION, REBUILD_INDEX) exist for other vector-store maintenance and are out of scope here.