Re-sharding a Collection

Overview

Re-sharding recreates a company's vector collection with a different shard count and atomically swaps the live collection to it. It copies the existing vectors as-is — there is no re-embedding and no model change — so it is faster than re-embedding but still scales with the number of points.

Use it to scale a vector collection: as a tenant's corpus grows, a higher shard count improves indexing/query parallelism and distributes the collection across the vector-DB cluster.

Phases

copying → verifying → swapping → cleanup

It captures the source collection's layout (shard count, replication factor), creates a new collection with the target shard count, copies all points, verifies the point count matches the source, swaps the live collection, and removes the old one.

Prerequisites

• Target shard count decided. Shard count scales with collection size and the number of vector-DB nodes — consult Unique on sizing if unsure.

• Vector-DB capacity for a second copy of the collection during the copying phase.

• Dry run first (estimates point count and shows the target shard count).

• Permissions, monitoring, lock, and resilience: see the hub.

Run it

Dry run (preview):

mutation {
  createMaintenanceJob(input: {
    operationType: "RESHARD_COLLECTION"
    shardNumber: 12
    dryRun: true
  }) { id status }
}

Execute:

mutation {
  createMaintenanceJob(input: {
    operationType: "RESHARD_COLLECTION"
    shardNumber: 12
    destructive: false
    dryRun: false
  }) { id status createdAt }
}

Monitor

phase progresses copying → verifying → swapping → cleanup; processed / total track points copied. See the hub for the maintenanceJob query, statuses, and cancelMaintenanceJob.

Safety & rollback

• Non-destructive by default — search keeps serving on the current collection until the atomic swap; a failure before the swap leaves users unaffected.

• Vectors are copied unchanged (same model/dimension), so search quality is identical after re-sharding.

• Rollback after completion: re-shard back to the previous shard count.

Troubleshooting