Language Model Info

2 min read

In any space or module, the language model is configured as described in this page.

For Azure and LiteLLM-deployed models, only the model name is required. For custom-deployed models, additional parameters must be provided in JSON format to fully define the model configuration.

Overview

The Language Model Info schema is used to configure different language models within our tool.

For Azure OpenAI models or models deployed via LiteLLM, setup requires only the model name, no additional configuration is needed. For custom models, a full configuration is required as a JSON, as outlined in the sections below.

If you are using Azure or LiteLLM, the rest of this documentation can be skipped and you can can refer directly to the LLM Overview.

Example Configuration

Azure Model Configuration

For Azure models, a simple configuration can be achieved by specifying the model name.

json

{
  "language_model": "AZURE_GPT_4_0613"
}

Custom Model Configuration

Custom models require a full configuration setup, detailing all necessary parameters:

json

{
  "language_model": {
    "name": "Custom_Model_Name",
    "version": "1.0",
    "provider": "CUSTOM",
    "encoder_name": "o200k_base",
    "token_limits": {
      "token_limit_input": 5000,
      "token_limit_output": 1500
    },
    "capabilities": ["function_calling", "streaming"],
    "info_cutoff_at": "2023-10-01",
    "published_at": "2023-09-01",
    "retirement_at": null,
    "deprecated_at": null,
    "retirement_text": null
  }
}

Fields Documentation

The following table describes the different parameters and which once are required.

Field Name	Description	Type	Default Value	Required
`name`	The name of the language model, can be an Azure model name or a custom name	string	N/A	Yes
`version`	The version of the language model	string	N/A	Yes
`provider`	The provider of the language model, either AZURE or CUSTOM	string	N/A	Yes
`encoder_name`	The encoder name used for the model	string [enum]	`cl100k_base`	No
`token_limits`	Defines input and output token limits for the model	integer	`{7000, 1000}`	No
`capabilities`	Lists the capabilities of the model. See below for available capabilities.	array	`["streaming"]`	No
`info_cutoff_at`	Date when information cutoff occurs	string	`null`	No
`published_at`	Date when the model was published	string	`null`	No
`retirement_at`	Date when the model is retired	string	`null`	No
`deprecated_at`	Date when the model is deprecated	string	`null`	No
`retirement_text`	Text describing the retirement details	string	`null`	No

Nested Types

token_limits

Field Name	Type	Description	Default Value	Required
token_limit_input	integer	Maximum number of input tokens	7000	Yes
token_limit_output	integer	Maximum number of output tokens	1000	Yes

Enums and Constants

Enumerated types and constants define fixed values to standardize data inputs.

Enum Name: The identifier for the enumeration.
Possible Values: All potential values the enum can represent.
Description: Explanation of the enum's role.

Enum Name	Possible Values	Description
EncoderName	`o200k_base`, `cl100k_base` , `qwen`, `deepseek`	Defines the encoder used for the model
LanguageModelName	see LLM Availability Overview	Lists available Azure language models
LanguageModelProvider	`AZURE`, `CUSTOM`	Specifies the provider of the language model
ModelCapabilities	json `"function_calling", "parallel_function_calling", "reproducible_output", "structured_output", "vision", "streaming", "reasoning"`	Enumerates the capabilities that the model can support

Full Json Schema

json

{
  "$defs": {
    "EncoderName": {
      "enum": [
        "o200k_base",
        "cl100k_base"
      ],
      "title": "EncoderName",
      "type": "string"
    },
    "LanguageModelName": {
      "enum": [
        "AZURE_GPT_35_TURBO_0125",
        "AZURE_GPT_4o_2024_0513",
        "AZURE_GPT_4o_2024_0806",
        "AZURE_GPT_4o_MINI_2024_0718",
        "AZURE_o1_PREVIEW_2024_0912",
        "AZURE_o1_2024_1217",
        "AZURE_o1_MINI_2024_0912",
        "AZURE_o3_MINI_2025_0131",
        "AZURE_GPT_45_PREVIEW_2025_0227"
      ],
      "title": "LanguageModelName",
      "type": "string"
    },
    "LanguageModelProvider": {
      "enum": [
        "AZURE",
        "CUSTOM"
      ],
      "title": "LanguageModelProvider",
      "type": "string"
    },
    "LanguageModelTokenLimits": {
      "properties": {
        "token_limit_input": {
          "title": "Token Limit Input",
          "type": "integer"
        },
        "token_limit_output": {
          "title": "Token Limit Output",
          "type": "integer"
        }
      },
      "required": [
        "token_limit_input",
        "token_limit_output"
      ],
      "title": "LanguageModelTokenLimits",
      "type": "object"
    },
    "ModelCapabilities": {
      "enum": [
        "function_calling",
        "parallel_function_calling",
        "reproducible_output",
        "structured_output",
        "vision",
        "streaming",
        "reasoning"
      ],
      "title": "ModelCapabilities",
      "type": "string"
    }
  },
  "properties": {
    "name": {
      "anyOf": [
        {
          "$ref": "#/$defs/LanguageModelName"
        },
        {
          "type": "string"
        }
      ],
      "title": "Name"
    },
    "version": {
      "title": "Version",
      "type": "string"
    },
    "provider": {
      "$ref": "#/$defs/LanguageModelProvider"
    },
    "encoder_name": {
      "$ref": "#/$defs/EncoderName",
      "default": "cl100k_base"
    },
    "token_limits": {
      "$ref": "#/$defs/LanguageModelTokenLimits",
      "default": {
        "token_limit_input": 7000,
        "token_limit_output": 1000
      }
    },
    "capabilities": {
      "default": [
        "streaming"
      ],
      "items": {
        "$ref": "#/$defs/ModelCapabilities"
      },
      "title": "Capabilities",
      "type": "array"
    },
    "info_cutoff_at": {
      "anyOf": [
        {
          "format": "date",
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Info Cutoff At"
    },
    "published_at": {
      "anyOf": [
        {
          "format": "date",
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Published At"
    },
    "retirement_at": {
      "anyOf": [
        {
          "format": "date",
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Retirement At"
    },
    "deprecated_at": {
      "anyOf": [
        {
          "format": "date",
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Deprecated At"
    },
    "retirement_text": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Retirement Text"
    }
  },
  "required": [
    "name",
    "version",
    "provider"
  ],
  "title": "LanguageModelInfo",
  "type": "object"
}