Document Search V2

7 min read

Functionality

This module is designed to answer user questions using documents stored in the Knowledge Center. It starts by converting the user’s query into a search string, which is then embedded and used to perform a semantic search (via VectorDB) and/or a full-text search (via PostgreSQL).

Based on the retrieved internal content, the module generates a response including references to the corresponding documents.

Reference in Code in AI Module Template

SearchInVectorDBV2

Configuration settings

Default Configuration

json
{
    "additionalLLMOptions": {},
    "languageModel": "AZURE_GPT_4o_2024_1120",
    "searchType": "COMBINED",
    "maxTokens": 30000,
    "scopeIds": null,
    "scopeToChatOnUpload": false,
    "chunkedSources": false,
    "historyIncluded": true,
    "keyWordExtractionTemperature": 0.0,
    "systemPromptSearch": "You are helping the employees with their questions. You will find below a question, some information sources and the past conversation (they are delimited with XML tags).\n\nAnswer the employee's question using ONLY facts from the sources or past conversation. Information helping the employee's question can also be added.\n\nIf not specified, format the answer using an introduction followed by a list of bullet points. The facts you add should ALWAYS help answering the question.\n\nSTRICTLY reference each fact you use. A fact is preferably referenced by ONLY ONE source e.g [sourceX]. If you use facts from past conversation, use [conversation] as a reference.\n\nHere is an example on how to reference sources (referenced facts must STRICTLY match the source number):\n- Some information retrieved from source N°X.[sourceX]\n- Some information retrieved from source N°Y and some information retrieved from source N°Z.[sourceY][sourceZ]\n- Some information retrieved from past conversation.[conversation]",
    "triggerPromptSearch": "new_question:\n```\nUSER_MESSAGE\n```\n\nsources:\n```\nSEARCH_CONTEXT\n```\n\npast_conversation:\n```\n<conversation>HISTORY</conversation>\n```\n\nnew_question:\n```\nUSER_MESSAGE\n```\n\nAnswer concisely in LANGUAGE and ALWAYS reference each of your facts:",
    "systemPromptChatUpload": "You are helping the employees with their questions. You will find below my question and an uploaded document (delimited with XML tags).\n\nYour task is to assist me, an employee, by providing me responses to my question, based on PURELY the information available in the uploded document as your only information source.\nSTRICTLY reference each fact you use. Here is an example on how to reference used facts:\n###\n- Information retrieved from source X.[sourceX]\n- Information retrieved from source Y.[sourceY]\n###\n\nYou are reluctant of making any claims unless they are stated by the uploaded document or past conversation. If there is a situation where you cannot provide an answer based solely on the available sources, please inform me accordingly.\n\nIf the question is talking about 'it', 'this document' or 'the document', the question is refering to all content in the uploaded document.\nIf the question is asking about the content of the document (e.g. 'What is it about?', 'What is the content of this document?'), provide a concise summary of one or two paragraphs.",
    "triggerPromptChatUpload": "question:\n```\nUSER_MESSAGE\n```\n\nuploaded document:\n```\nSEARCH_CONTEXT\n```\n\nquestion:\n```\nUSER_MESSAGE\n```\n\nAnswer in LANGUAGE.\nAnswer using ONLY information from the uploaded document and ALWAYS reference each of your facts:",
    "systemPromptSearchString": "Below is a history of the previous conversation and a question asked by the user (delimitated by XML tags). Follow these steps:\n\nStep 1: Translate the user question to english.\n\nStep 2: Verify if the new question relates with the previous conversation. If the new question does not relate then say for Step 2 '<not_a_follow_up>', otherwise say '<follow_up>'.\n\nStep 3: Generate a search query in English optimised for a vector database search by combining the english translation with relevant information from the previous conversation. The query must be a sentence, instruction or question and in English.\n\nStep 4: Output ALWAYS a JSON object structured like: {\"translation\": user question translated to english, \"relation\": <not_a_follow_up> or <follow_up>,  \"search_query\": updated search query}\n\nExample:\n{\n\"translation\": \"How many live there?\",\n\"relation\": \"<follow_up>\",\n\"search_query\": \"How many Tweeka live in Columbia (South America)?\"\n}",
    "triggerPromptSearchString": "Previous conversation:\n```\nLAST_3_MESSAGES\n```\n\nUser question:\n<new_question>USER_MESSAGE</new_question>\n\nOutput in JSON format:",
    "rerankerConfig": null,
    "searchLanguage": "english",
    "limit": 50,
    "evaluationConfig": {
        "displayName": "Hallucination-Level",
        "metricConfigs": [
            {
                "enabled": false,
                "name": "hallucination",
                "languageModel": "AZURE_GPT_4o_2024_1120",
                "customPrompts": {
                    "systemPrompt": "\nYou will receive a question, references, a conversation between a user and an agent, and an output. \nThe output is the answer to the question. \nYour task is to evaluate if the output is fully supported by the information provided in the references and conversation, and provide explanations on your judgement in 2 sentences.\n\nUse the following entailment scale to generate a score:\n[low] - All information in output is supported by the references/conversation, or extractions from the references/conversation.\n[medium] - The output is supported by the references/conversation to some extent, but there is at least some information in the output that is not discussed in the references/conversation. For example, if an instruction asks about two concepts and the references/conversation only discusses either of them, it should be considered a [medium] hallucination level.\n[high] - The output contains information that is not part of the references/conversation, is unrelated to the references/conversation, or contradicts the references/conversation.\n\nMake sure to not use any external information/knowledge to judge whether the output is true or not. Only check whether the output is supported by the references/conversation, and not whether the output is correct or not. Also do not evaluate if the references/conversation contain further information that is not part of the output but could be relevant to the question. If the output mentions a plot or chart, ignore this information in your evaluation.\n\nYour answer must be in JSON format:\n{\n \"reason\": Your explanation of your judgement of the evaluation,\n \"value\": decision, must be one of the following: [\"high\", \"medium\", \"low\"]\n}                                                  \n",
                    "userPrompt": "\nHere is the data:\n\nInput:\n'''\n$input_text\n'''\n\nReferences:\n'''\n$contexts_text\n'''\n\nConversation:\n'''\n$history_messages_text\n'''\n\nOutput:\n'''\n$output_text\n'''\n\nAnswer as JSON:\n",
                    "systemPromptDefault": "\nYou will receive a question and an output. \nThe output is the answer to the question. \nThe situation is that no references could be found to answer the question. Your task is to evaluate if the output contains any information to answer the question,\nand provide a short explanations of your reasoning in 2 sentences. Also mention in your explanation that no references were provided to answer the question.\n\nUse the following entailment scale to generate a score:\n[low] - The output does not contain any information to answer the question.\n[medium] - The output contains some information to answer the question, but does not answer the question entirely. \n[high] - The output answers the question.\n\nIt is not considered an answer when the output relates to the questions subject. Make sure to not use any external information/knowledge to judge whether the output is true or not. Only check that the output does not answer the question, and not whether the output is correct or not.\nYour answer must be in JSON format:\n{\n \"reason\": Your explanation of your reasoning of the evaluation,\n \"value\": decision, must be one of the following: [\"low\", \"medium\", \"high\"]\n}\n",
                    "userPromptDefault": "                                                  \nHere is the data:\n\nInput:\n'''\n$input_text\n'''\n\nOutput:\n'''\n$output_text\n'''\n\nAnswer as JSON:\n"
                },
                "scoreToLabel": {
                    "LOW": "GREEN",
                    "MEDIUM": "YELLOW",
                    "HIGH": "RED"
                },
                "scoreToTitle": {
                    "LOW": "No Hallucination Detected",
                    "MEDIUM": "Hallucination Warning",
                    "HIGH": "High Hallucination"
                }
            }
        ],
        "hideScoreValue": false
    },
    "chunkRelevancySortConfig": {
        "enabled": false,
        "relevancyLevelsToConsider": [
            "high",
            "medium",
            "low"
        ],
        "relevancyLevelOrder": {
            "high": 0,
            "medium": 1,
            "low": 2
        },
        "languageModel": "AZURE_GPT_4o_2024_1120",
        "fallbackLanguageModel": "AZURE_GPT_4o_2024_1120",
        "structuredOutputConfig": {
            "enabled": false,
            "extract_fact_list": false,
            "reason_description": "A brief explanation justifying your evaluation decision.",
            "value_description": "Assessment of how relevant the facts are to the query. Must be one of: ['low', 'medium', 'high'].",
            "fact_description": "A fact is an information that is directly answers the user's query. Make sure to emphasize the important information from the fact with bold text.",
            "fact_list_description": "A list of relevant facts extracted from the source that supports or answers the user's query."
        },
        "maxTasks": null
    },
    "chatOnly": false,
    "numberOfHistoryInteractionsToInclude": 2,
    "sourceFormatConfig": {
        "sourceTemplate": "<source${index}>${document}${info}${text}</source${index}>",
        "sections": {
            "document": "<|document|>{}<|/document|>\n",
            "info": "<|info|>{}<|/info|>\n"
        }
    }
}

General parameters

Parameter

Description

Type

Default

languageModel

Specifies the language model used

string

AZURE_GPT_4o_2024_1120

additionalLLMOptions

Any additional parameters like seed that wish to specify in the LLM call. Eg.
'additionalLLMOptions': {'seed': 42}

object

{}

searchType

Defines the type of search to be performed (VECTOR or COMBINED)

string

COMBINED

limit

Number of chunks returned by search type.

integer

50

maxTokens

Maximum number of tokens used by sources and previous conversation in the LLM call

integer

3000

scopeIds

Optional scope identifiers to limit the search

[string]

null

scopeToChatOnUpload

Indicates if the scope should be switched to the current chat upon file upload

boolean

false → Scope switching off

chatOnly

If true, restricts search to files uploaded to chat (Irrespective of whether a file was uploaded or not)

boolean

false

chunkedSources

Indicates if chunks of the same document are appended as individual sources (true) to the LLM content or merged to one source (false)
We recommend setting this parameter to true for GPT-4 and false for GPT-3.5.

boolean

false

historyIncluded

Flag that allows to include previous chat conversation in GPT-calls only if the new user input is a follow-up question (false)

boolean

true → History always included

keyWordExtractionTemperature

Temperature setting for keyword extraction

integer

0

evaluationConfig

Object containing configuration for answer evaluation (e.g. hallucination check)

object

See
Hallucination Evaluation

chunkRelevancySort

Enable the sorting of retrieved chunks based on their relevance to the user input

object

See
Chunk Relevancy Sort

sourceFormatConfig

SourceFormatConfig allows you to include metadata into the chunk. This does not improve vector/combined search, but helps bring metadata into the LLM’s context at the chunk level. Note that the keywords “document” and “info” are not supported from the metadata.

object

"sourceFormatConfig": {
    "sections": {
      "key": "<|KeyFromMetaData|>{}<|/keyFromMetaData|>\n",
      "document": "<|document|>{}<|/document|>\n"
    },
    "source_template": "<source${index}>${document}${key}${text}</source${index}>"
}

numberOfHistoryInteractionsToInclude

Number of history interactions to be included into the new LLM call

integer

2

searchLanguage

Language to be used for full-text search

string

english

rerankerConfig

Reranker configuration

object

null

alwaysUseExtractedSearchQuery

Force use of extarcted search query even if not a followup promopt.

boolean

false

Prompts

Parameter

Description

Type

systemPromptSearch

triggerPromptSearch

System and trigger prompt used to interpret user input and form search queries

string

systemPromptChatUpload

triggerPromptChatUpload

System and trigger prompt used for chat upload scenarios

string

systemPromptSearchString

triggerPromptSearchString

System and trigger prompt used to extract the search string from the user question

string

Reserved keywords in Trigger Prompts

Keyword

Purpose/Replaced BY

USER_MESSAGE

The input user message

SEARCH_CONTEXT

Search results returned by the knowledge base/uploaded documents

HISTORY

Past messages of this chat

DATE_TIME

Current date and time in ISO format

LANGUAGE

Language of the user message

For example a trigger prompt can look like :

py
DEFAULT_VECTOR_DB_SEARCH_TRIGGER_PROMPT = """
Current Date and Time: DATE_TIME

new_question:
```
USER_MESSAGE
```

sources:
```
SEARCH_CONTEXT
```

past_conversation:
```
<conversation>HISTORY</conversation>
```

new_question:
```
USER_MESSAGE
```

Answer concisely in LANGUAGE and ALWAYS reference each of your facts:"""

(Tool) Definition

json
{
  "type": "function",
  "function": {
    "name": "SearchInVectorDBV2",
    "parameters": {
      "type": "object",
      "required": [
        "instruction",
        "language"
      ],
      "properties": {
        "language": {
          "type": "string",
          "description": "The language used by the user in their prompt (e.g. English, French, German, etc.)."
        },
        "instruction": {
          "type": "string",
          "description": "The semantic query to search. Should be the form of a semantic query containing all relevant information."
        }
      }
    },
    "description": "Search in the employee knowledge base for information on policies, procedures, benefits, groups, or specific people. This should be your go-to tool if no other tools are applicable."
  }
}
Last updated