Analytics

9 min read

Analytics reports (e.g., user engagement) are available via API or also via Unique UI. The article explains which reports are available out of the box and how to get them. Unique continuously enriches the service with additional reports.

Definitions

Active Users: A user with a minimum of one interaction (question & answer) over a defined period of time

Unique Users: A user counted only once no matter how many times they interacted (question & answer) over a defined period of time.

User information anonymization

You can anonymize user email addresses in analytics exports by enabling the environment variable ANONYMIZED_CHAT_INTERACTIONS_EXPORT.

Set: ANONYMIZED_CHAT_INTERACTIONS_EXPORT=true
Effect: Email addresses will be anonymized in the export output.

When enabled, exported data will return anonymized identifiers in place of raw email addresses, as shown below.

Anonymization causes identifiers such as email addresses to be randomized every time you export analytics data, if this behavior is unwanted as it may become difficult to correlate data across exports you can enable idempotent anonymization ANONYMIZED_CHAT_INTERACTIONS_EXPORT_TYPE which are stable pseudonyms

Set: ANONYMIZED_CHAT_INTERACTIONS_EXPORT_TYPE=idempotent
Effect: User ID or Email addresses will be anonymized using a stable pseudonym

Available Reports

Usage statistics

Chat Interactions
Active Users
Document Reference Statistics

Work with Reports

How to download Reports

A user with the required role is able to request an analytics export.
A user can filter the data request:
- By date
- with for a specific Assistant by spec
When a user requests for a report, the system will create an order entry in the Analytics order table. After the request is completed, a report is available for the user to download.
A user can delete the report after it has been processed or downloaded. You can also filter down report for a specific Assistant before exporting.

Interpret reports

Chat Interactions

With this analytics you can evaluate the following:

Which assistant is most used
Number of interactions per assistant
Most active user

Email: The email of the user, who interacted with the assistant
Date: The date of the interaction
Assistant: The assistant used for the interaction
Count: Number of interactions (question & answer) the user had on the indicated date with the selected assistant

Active Users

With this analytics you can evaluate the overall engagement of the users on the platform:

How many unique daily active users (DAU) are on the platform?
How many unique weekly active users (WAU) are on the platform?
How many unique monthly active users (MAU) are on the platform?
How did the DAU/WAU/MAU develop over time?

Period: month (relevant for MAU), week (relevant for WAU), day (relevant for DAU)
Date: The date of the interaction. The date for the weekly interaction always indicates the end of a 7-day period, e.g. week 10.02.24 means the week 04.02.-10.02.2024.
Value: Number of unique users who had at least one interaction (request) in the specified period. A user who has 10 interactions ( questions & answers) within one day is only counted once (unique user).

Reference Statistics

With this analytics you can evaluate the overall engagement of the users on the platform:

Who uploaded which documents?
Which documents are most asked about?
Which documents are uploaded to which scopes?

File Uploaded Date: The date the file was initially ingested (connected via Sharepoint or manually uploaded)
File name: The name of the file (as also shown in the knowledge center)
File uploader: The email address of the user who uploaded or connected the file. For Sharepoint it is currently just one generic user ID, which will be changed in the future.
Source: UNIQUE_BLOB_STORAGE refers to the manual upload of files and MICROSOFT_365_SHAREPOINT to documents ingested via Sharepoint
Reference Count: The number a document was referenced on the platform when answering a question.
Scope: The scope a file was uploaded to

Detailed Chat Interactions

This report gives you a detailed export of the chat interactions. The report contains mainly every message of the user and answer of Unique AI in the specified time frame. The user is as default anonymised. But on special request by a customer Unique can change this to contain the users data. (Not available on multi tenant instance)

This exported data can help identify frequently asked questions/prompts and also missing knowledge when Unique AI is not able to provide an answer.

Username: The name of the user or anonymised: Unknown
Timestamp: The date and time the user sent the message
Space: Which space the user has used for this message
Chat ID: Identifying which messages belongs together (the same chat)
Device Category: Which device the user used (mobile, desktop, tablet)
Message: The message that was sent
Answer: The answer of Unique AI

Note the Message and Answer columns are returned if the user requesting the data has the chat.data.admin role

Tool Names: The tools invoked by Unique AI while producing the answer.

Tool Count: Number of tool calls the assistant made during the interaction.
Skill Names: The skills used by Unique AI while producing the answer.
Referenced sources count: How many distinct sources were cited in the response.
Subagents: Number of sub-agents spawned to handle parallel or specialized subtasks within the interaction.
MCP tool names: The specific MCP tools invoked, listed by name.
Prompt length: Size of the user's input prompt, measured in characters or tokens.
Answer length: Size of the assistant's final response, measured in characters.
Loop iterations: Number of internal reasoning/execution cycles the agent ran before producing the final answer.
Model name: Identifier of the language model that served the interaction.
Total response time: End-to-end latency from prompt submission to final answer, typically in seconds.
Artifacts count: Number of output files or deliverables generated.
File types: The formats of any generated.
Output size: Total size of all generated artifacts.
Cost (USD): Cost of the interaction based on model and token usage.
Total token usage: Combined input + output tokens consumed across the entire interaction.

Ingestion Report

The Ingestion Report helps you to understand which users are using the ingestion services and how. The report delivers the following metrics:

Total Contents (Files) ingested per user for the given timeframe
Total pages ingested per user for the given timeframe

UserId: The unique identifier of the user who ingested the content
Email: The email address of the user who uploaded/ingested content
ContentCount: Number of documents/content items the user ingested within the selected date range
PageCount: Total number of pages processed from all content ingested by the user

Date Range

Default range: Last 180 days
Maximum allowed range: 365 days
Custom start and end dates can be specified

Privacy Features

The report respects company privacy settings:

Pseudonymization: User identifiers may be anonymized based on company configuration
Authorization: Requires CHAT_FEEDBACK_READ or CHAT_DATA_ADMIN role to access

Product Metrics

This report provides key engagement metrics — DAU, WAU, and MAU — representing the number of unique users who interact with the Chat or Agentic Table interfaces over specific time periods (daily, weekly, and monthly).

Daily Active Users (DAU) measure how consistently users engage with the Chat or Agentic Table in their day-to-day work.
An increase in DAU indicates that users are finding recurring value in these interfaces, while a decline may suggest reduced relevance or potential usability issues.
Weekly Active Users (WAU) highlight short-term engagement patterns, showing how often users return within a week.
WAU is especially useful for identifying cyclical usage — for example, users engaging around specific workflows or reporting periods.
Monthly Active Users (MAU) represent the overall reach and adoption of the Chat and Agentic Table over time.
A steady or growing MAU signals healthy user retention and expansion, while a decline could point to disengagement or lack of sustained adoption.

The tables below provide an overview of all metrics currently being tracked:

Chat Metrics

Metric Name	Explanation
`space_interaction`	This is triggered when a new space is created
`context_window_interaction`	This is triggered when a user open the context window
`select_files_to_chat`	This is triggered when a user selects a file to chat.

RAG Metrics

Metric Name	Explanation
`kb_visit`	Triggered when a user visits the knowledge base
`kb_document_upload`	This is triggered when a user uploads a file to the knowledge base
`kb_scope_created`	This is triggered when a user creates a new folder
`kb_document_search`	This is triggered when a user searches a file in the knowledge base

Agentic Table Metrics

Metric Name	Explanation
`agentic_table_sheet_creation`	Triggered when a user creates a new sheet.
`agentic_table_library_sheet_creation`	Triggered when a new answer library sheet is created within an Agentic Table space.
`agentic_table_sheet_deletion`	Triggered when a user deletes a sheet via the Landing page.
`agentic_table_sheet_completion`	Triggered when a user marks a non-library sheet as completed.
`agentic_table_library_sheet_completion`	Triggered when a user marks a library sheet as completed.
`agentic_table_cell_update`	Triggered whenever a user edits an existing cell or adds a new one in a sheet.
`agentic_table_metadata_added_existing_sheet`	Triggered when sources or question files are added to an existing sheet.
`agentic_table_metadata_added_new_sheet`	Triggered whenever sources or question files are added during the creation of a new sheet.
`agentic_table_row_submitted_initial`	Triggered when rows are submitted for the first time (initial run). Unlike other metrics, the value increases per submitted row, not per unique user.
`agentic_table_row_submitted`	Triggered when rows are submitted (including re-runs). Unlike other metrics, the value increases per submitted row, not per unique user.

Important Note

All metrics are tracked per unique user. This means that multiple actions by the same user within a given day count as a single event for that metric.

Screenshot 2025-10-21 at 17.23.25.png — Example Product Metrics Export

Tool call and MCP metrics

This section defines every tool-call and MCP metric exported in the Unique AI metrics report (the separate metric document an admin can download under Metrics).

The prefix rule (read this first)

The prefix decides what the number counts:

MAU- / WAU- / DAU- = unique active user counts only (distinct users who did the action in the period).
monthly- / weekly- / daily- = total call counts and company-level stats (raw volumes and ratios).

In this export, the raw counts exist at daily, weekly and monthly granularity, while every rate, ratio and distribution is daily only.

All ratios derive from three flags stamped on each tool call in the debug info: is_forced (tool selected by the user vs. chosen by the model), is_exclusive (the only tool used in that chat), and loop_iteration (which orchestrator iteration made the call).

Raw counts

Metric key	Description	Formula
`{period}-tool_calls:{tool}`	Number of invocations of a built-in tool (WebSearch, InternalSearch, UploadedSearch, code_interpreter, TodoWrite).	count of calls where name = {tool}
`{period}-mcp_calls:{server}`	Calls routed to a given MCP server.	count of MCP calls where server = {server}
`{period}-mcp_tool_calls:{server}:{tool}`	Calls to one specific tool on one specific MCP server.	count where server = {server} and tool = {tool}
`daily-mcp_total_calls`	All MCP calls across every server that day.	sum of `mcp_calls:{server}`

Do not double count. Every MCP call is listed twice: once under mcp_tool_calls:{server}:{tool} and once under the generic tool_calls:mcp_.... The total counts them once. Never add the MCP and tool namespaces together.

Rates and ratios (daily)

Metric key	Description	Formula	Range
`daily-tool_calls_average_per_chat`	Engagement intensity: mean tool invocations per conversation (numerator includes MCP calls).	total tool calls / total chats	0 to n (can exceed 1)
`daily-tool_calls_usage_rate`	Adoption: share of all chats that fired at least one tool.	chats with at least 1 tool call / total chats	0 to 1
`daily-mcp_usage_rate`	Share of all chats that used at least one MCP call.	chats with at least 1 MCP call / total chats	0 to 1
`daily-tool_calls_forced_ratio`	Orchestration health: how often tools were user-forced rather than model-chosen. High means the model is not auto-selecting well, or users are overriding it.	forced tool calls / total tool calls	0 to 1
`daily-mcp_forced_ratio`	Same signal for MCP calls.	forced MCP calls / total MCP calls	0 to 1
`daily-tool_calls_exclusive_ratio`	How often a tool was used in isolation vs. combined with others. Low means multi-source retrieval is the norm.	exclusive tool calls / total tool calls	0 to 1

Distributions (daily)

Metric key	Description	Formula
`daily-tool_calls_distribution:{tool}`	Per-tool intensity across all chats. The set sums to `average_per_chat`, not to 1.	`tool_calls:{tool}` / total chats
`daily-mcp_distribution:{server}`	As exported: average calls to that server per MCP-using chat. This is NOT a share and does not sum to 1.	`mcp_calls:{server}` / number of MCP-using chats