Usage or change of custom embedding model

2 min read

Please note that only one embedding model per organization can be configured. Changing the embedding model and / or the embedding dimensions requires to re-embed all ingested files.

Unique offers customers to have multiple or custom embedding models on an Unique instance. So for example Company A can use the default embedding model provided by Unique and Company B can use a custom provided embedding model.

Setup

To setup additional embedding models you need to contact the maintainer of the instance because it requires to set an environment variable on the node-ingestion application. The environment variable (ADDITIONAL_EMBEDDING_MODELS_JSON) has those properties:

json

ADDITIONAL_EMBEDDING_MODELS_JSON='[{
  "endpoint": "https://myInstance.openai.azure.com/",
  "apiKey": "mySecretApiKey",
  "apiVersion": "2024-02-01",
  "deploymentName": "text-embedding-ada-002"
}]'

Available Azure OpenAI Embedding Models

Overview

The text-embedding-3-small and text-embedding-3-large are the recommended embedding models (see below). The dimensions of these embedding models can be reduced by providing a emdeddingDimension .

Please note that only one embedding model per organization can be configured. Changing the embedding model and / or the embedding dimensions requires to re-embed all ingested files.

Deployment Name	Tokens Per Minute*	Price per 1k tokens	Max Dimensions	Dimensions Configurable
`text-embedding-ada-002`	1 Mio.	$0.000121	1536	❌
`text-embedding-3-small`	1 Mio.	$0.000025	1536	✅
`text-embedding-3-large`	1 Mio.	$0.000158	3072	✅
Table 1: Embedding model overview (see pricing details here) *per region and subscription

Recommendation

We recommend using the text-embedding-3-small when a cost efficient approach is favored and the text-embedding-3-large when performance is favored. Please note that text-embedding-3-large comes with higher costs and embedding latency. The dimensions of both embedding models can be reduced. We do not recommend reducing the dimensions below 1536 dimensions.

Eval benchmark	`text-embedding-ada-002`	`text-embedding-3-small`	`text-embedding-3-large`
MIRACL average	31.4	44.0	54.9
MTEB average	61.0	62.3	64.6
Table 2: Embedding model benchmark (source OpenAI)

Configuration

When the environment variable is set and the application has been restarted then we can start configuring a company of Unique AI to use the new additional embedding model.
To configure an company to use a different embedding model an update of the company entry in the CompanyMeta table is required to specify this additional embedding model and embedding dimension. This can be done via API request:

bash

curl --location 'https://gateway.<tenantName>.unique.app/ingestion/graphql' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation CompanyMetaUpdate($embeddingModel: String) {\n  companyMetaUpdate(embeddingModel: $embeddingModel) {\n    companyId\n  }\n}","variables":{"embeddingModel":"text-embedding-ada-002", "emdeddingDimension": 3072}}'

It will change the company entry based on the user this token belongs to. After this change the generation of the search embedding vector during a semantic search and also the ingestion of new data will use this specified embedding model.

Re-Embed existing content

If the company has already ingested data, a re-embedding of this data is required to align with the new embedding model. Otherwise you end up having discrepancy embedding vectors because they might be created with a different model. This re-embedding can be done with the content admin endpoint to re-embed all contents.

This is documented in this section: Working with Content Admin Endpoints