Technical Insight

To fulfil this use case Unique created two entities. One is called LanguageModelGroup and one is called ExternalLanguageModel. The LanguageModelGroup is an object containing mainly the groupName and is connected to the ExternalLanguageModel with an n:m relationship. The groupName basically replaces the hardcoded LLM enums of Unique. This fulfils the requirement that clients want to manage the LXMs by themselves.

The ExternalLanguageModel contains all the information which is needed for Unique to be able to call this custom provided LXM. This gives the ability to clients to be able to connect custom LXM with Unique.

The reason why we have the possibility to group together multiple ExternalLanguageModel into a LanguageModelGroup is that load can be randomly distributed to multiple instances. In case the token usage is 500k tokens but the limit per model instance/deployment is capped at 300k tokens there is the possibility to deploy a 2nd instance to fully cover the usage request.

Chat API URL

For the following documentation you might require the Chat Backend API URL as location for the request examples. This URL is marked as <chatGraphqlUrl> and depends on your tenant. It can be one of the following:

For CH and EU: https://gateway.unique.app/chat-gen2/graphql

For US: https://api.us1.unique.app/chat/graphql

How to Create and Connect

Currently Unique does not offer a user interface for it. So the following curl request will create an ExternalLanguageModel and connect it with the LanguageModelGroup. If the LanguageModelGroup is not existing yet, Unique will create a new LanguageModelGroup. If there is already one with this groupName Unique will add it to the existing one.

Create Graphql-Mutation

curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation Mutation($groupName: String!, $input: ExternalLanguageModelCreateInput!) {\n  externalLanguageModelCreate(groupName: $groupName, input: $input) {\n    id\n    internalDescription\n    languageModelGroups {\n      id\n      groupName\n    }\n  }\n}","variables":{"groupName":"<groupName>","input":{"version":"<modelVersion>","model":"<modelName>","maxTokens":<maxToken>,"internalDescription":"<internalDescription>","endpoint":"<endpoint>","deploymentName":"<deploymentName>","authHeaderValue":"<apiToken>","apiVersion":"<apiVersion>"}}}'

Field	Description	Azure OpenAI example
tenantName*	Unique Single Tenant Base URL	oleole
token*	token A token for your tenant of a user with role chat.admin.all Refer to How To Get An API Token to learn how to get a token	Bearer S0VLU0UhIExFQ0tFUiEK...
groupName*	The LanguageModelGroup that is referenced in further Unique configurations.	MY_OWN_MODEL_GPT_4_RAVEN
modelVersion*	The model version deployed at your endpoint
modelName*	The model name deployed at your endpoint
maxToken*	The maximum amount of tokens that Unique will send in one request to the endpoint	3000
internalDescription*	A description of the endpoint for Unique internal reference. You can use it later to reference information of yours to recognise the LXM.	GPT-4-Turbo deployed at experimental Azure subscription Tango-001.
endpoint*	The endpoint of your LXM	https://your-endpoint.openai.azure.com/
deploymentName*	The deployment name (gets added to the endpoint by the @azure/openai library) that you chose when deploying the model in a specific version
apiToken (* even if just a artificial - otherwise it uses Azure WID)	The API token, see #Security This is getting saved encrypted in the DB and only decrypted when Unique does the API request. There will be no possibility to access or change it afterwards, it is immutable write-only.
apiVersion*	API Version that the @azure/openai library will use to talk to the model.	2024-02-01
oauth2	If the endpoint oauth2 should be used as authentication the query above needs to be extended with the following fields: oauth2ClientId, oauth2ClientSecret, oauth2Endpoint, oauth2Scope(optional). Follow this documentation for more information about Oauth2 authentication for LLM calls.

* RequiredFields

Azure OpenAI GPT-4o full example

curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {TOKEN}' \
--data '{"query":"mutation Mutation($groupName: String!, $input: ExternalLanguageModelCreateInput!) {\n  externalLanguageModelCreate(groupName: $groupName, input: $input) {\n    id\n    internalDescription\n    languageModelGroups {\n      id\n      groupName\n    }\n  }\n}","variables":{"groupName":"GPT_4O_PTU_CUSTOM","input":{"version":"2024-05-13","model":"gpt-4o","maxTokens":126000,"internalDescription":"Uniques GPT 4o QA deployment in Sweden","endpoint":"https://unique-openai-083.openai.azure.com/","deploymentName":"gpt-4o-2024-05-13","authHeaderValue":"<secret>","apiVersion":"2024-02-01"}}}' \

Renaming an Existing GroupName

During this process, after changing to the new groupName, the previous groupName will no longer be valid and all LXM configurations using the old groupName will fail! Until you have manually changed all previous configurations to the new one.

This renaming does not automatically rename all configurations done with the old groupName!

Due to this limitation, Unique recommends to first create a new group (including all the same ExternalLanguageModels - basically do the “Create API” chapter) under the new name, migrate all assistants and then execute a Delete Call for not deprecated objects.

To rename an existing groupName you need to know the current groupName. Then with this curl you can rename the groupName to a different new groupName.

curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation LanguageModelGroupUpdate($groupName: String!, $input: LanguageModelGroupUpdateInput!) {\n  languageModelGroupUpdate(groupName: $groupName, input: $input) {\n    groupName\n    id\n  }\n}","variables":{"groupName":"<currentGroupName>","input":{"groupName":"<newGroupName>"}}}'

Now the LanguageModelGroup.groupName is renamed to the new name and all models attached to it are now available under the new groupName. The old groupName is not valid anymore!

Deleting an Existing LanguageModelGroup

Deleting a LanguageModelGroup which is still in use will disrupt the system. Unique does not reverse engineer if a group is used. If a group gets deleted while linked to an assistant, the assistant will subsequently fail.

This case is not covered by Uniques SLAs!

When you want to delete a LanguageModelGroup make first sure that the groupName related to this group is not used anymore in any LXM configuration. Otherwise this configuration will not work anymore.

After you made sure that this groupName is not used anymore you can use the following curl to delete the LanguageModelGroup.

curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation LanguageModelGroupDelete($groupName: String!) {\n  languageModelGroupDelete(groupName: $groupName) {\n    id\n    groupName\n  }\n}","variables":{"groupName":"<groupNameToDelete>"}}'

Deleting an Existing ExternalLanguageModel

Deleting the last ExternalLanguageModel assigned to an LanguageModelGroup which is still in use use will disrupt the system. We can then no longer find a LXM assigned to this group. Unique does not reverse engineer if a group is used. If a group gets deleted while linked to an assistant, the assistant will subsequently fail.

This case is not covered by Uniques SLAs!

First make sure you do not delete the last ExternalLanguageModel assigned to a LanguageModelGroup which is still in use. You can do this with the following curl which will fetch all ExternalLanguageModels and its LanguageModelGroups.

curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"query ExternalLanguageModelFindMany {\n  externalLanguageModelFindMany {\n    id\n    internalDescription\n    languageModelGroups {\n      groupName\n    }\n  }\n}"}'

Also in the response of this list of ExternalLanguageModels you also can find the id of the ExternalLanguageModel you would like to delete. This id then can be used to delete the ExternalLanguageModel with the following curl:

curl --location '<chatGraphqlUrl>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation ExternalLanguageModelDelete($externalLanguageModelDeleteId: String!) {\n  externalLanguageModelDelete(id: $externalLanguageModelDeleteId) {\n    id\n    internalDescription\n  }\n}","variables":{"externalLanguageModelDeleteId":"<IdOfModelToDelete>"}}'

Other APIs

There is also the possibility for other kind of mutations via API on the ExternalLanguageModel and the LanguageModelGroup. Consult Unique support to learn more.

Connect

With this groupName you passed you are now able to change the languageModel attribute on the modules. From now on, this module configuration will use one of the ExternalLanguageModel, which is bound to this LanguageModelGroup.

Security

Connectivity

Naturally, as of now, the model must be reachable over the same network. For Unique PaaS and Single Tenants that means; the internet! Customer Managed or On Premise Tenants can leverage this functionality as well where their models must just be reachable from the deployments networks.

API Token

Unique uses the AES-256-cbc algorithm with a 32 bytes random string to encrypt and decrypt the API Token symmetrically. Each Unique Deployment has its own unique encryption key. This encryption key is stored in an Azure KeyVault vault of the environment and will be loaded on startup into the chat application. The same key is used for all organisations/companies within the one deployment/tenant.

By enabling clients to directly store the key encrypted, Unique must never receive the key and store it. Unique also does not see or need the key as the key gets decrypted just-in-time to send the request. Technically Unique employees can equally not retrieve the key from the database except in specially triggered incidents by the client where Unique employees get temporary access to inspect the clients data.

Company Scoping

The complete Connect custom LXM (LLM) with Unique AIfeature is company-scoped. Companies cannot reuse LanguageModelGroups or ExternalLanguageModels from other companies. Of course, LanguageModelGroups and ExternalLanguageModels can be added to two or more companies though but that requires Administrators to explicitly send and define the same LXM for two or more companies/organisation.

Limitations

• Reserved key words: There are reserved key words for groupNames which cannot be used. Basically are that the ones that Unique provides you as default starting with AZURE_GPT_

• API only: Configurations can only be done via API currently.

• GPT API standards: For the LXM network request Unique is using the @azure/openai library. Means the APIs of your LXM needs to be compatible with the GPT standards. This is also known as https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-api?tabs=python.

• Connecting one model to two groups: Its not possible to connect one ExternalLanguageModel to multiple LangaugeModelGroup. The API call for this mutation is currently missing and is in progress.