Logging Architecture Overview

10 min read

Overview

Unique AI and its deployment models generate a variety of logs to support effective monitoring, security, and operational visibility. Understanding these logs is critical to maintaining system integrity and achieving compliance with industry standards. This document provides a detailed overview of the different types of logs produced, their sources, and how Unique AI, or in some deployment scenarios, the client, utilizes these logs for auditing and analysis.

Layer Descriptions

Before diving into the specifics of each layer in the logging architecture, it is important to understand the structure of the table used to describe the logs at every layer. Each table provides a consistent framework that outlines the key aspects of the logs generated within that layer. The table format is reused across all layers for clarity and consistency, and it includes the following columns:

Type:
- This column defines the type of log or the category of the activity being captured. It helps to quickly identify the nature of the log, such as security-related events, system modifications, or application-specific actions. Each layer may contain various types of logs depending on its scope and function within the system.
Description:
- The description provides a brief but informative explanation of what each log type entails. This includes details on the kind of events that the log records, how the log supports auditing, compliance, or troubleshooting, and its relevance to overall system monitoring. This column helps the user understand the purpose and context behind each type of log.
Data Class:
- Data classification is crucial for understanding the sensitivity and regulatory requirements related to the logged data. This column categorizes the data to help users manage it according to legal, security, or operational policies. For example, some logs may contain sensitive information subject to retention and access controls, while others may be less critical (see link )
Examples:
- To further illustrate each log type, this column provides concrete examples of scenarios or events that would trigger the generation of a log. These examples help clarify how each log type functions in real-world situations. For instance, an example may detail how a user interaction or a system event creates a specific log entry.
Log Destination:
- This column specifies where the logs are stored based on the deployment model. Different deployment models—such as Unique Multi-Tenant, Unique Single-Tenant, and Customer Managed Tenant—may handle log storage and management differently. This column outlines whether logs are retained in shared environments, customer-managed infrastructure, or single-tenant setups. It also provides guidance on log retention periods and access controls.

Layers

Now that we have a clear understanding of the structure and categories used to describe logs within each layer, we can dive deeper into the specific layers of the logging architecture. Each layer plays a distinct role in the overall system and provides valuable insights through its logs. Below is an overview of the different layers and the types of logs they generate.

1. Provider Layer

This is the foundation that consists of cloud infrastructure components such as Azure, AWS, or on-premises equivalents. It manages infrastructure-level resources like identity management, authentication, and privilege activities.

Key Logs :

Type	Description	Data Class	Examples	Log destination
Unique Multi Tenant	Unique Single Tenant	Customer Managed Tenant
IDP Audit Logs	The range of Identity Providers (IDPs) is extensive, covering solutions such as Azure Entra ID, Active Directory Federation Services (ADFS), and custom implementations. In more complex environments, multiple IDPs may be deployed, including scenarios involving local Active Directory (AD) or LDAP.	C2	Who created the user account (user@domain.com), when it was created, under whose approval, and which ticket was associated with the request? When was the user assigned to the 'AI Engineers' team, under whose approval, and in response to which access request?	Logs are captured by Azure Activity Logs and retained for 90 days.	Logs are highly customized for each client, with respective providers responsible for their creation, storage, and retention. Clients are accountable for the proper handling of these logs. Unique can be consulted for guidance and should be informed as necessary.
IDP Privilege Escalation Logs	Security best practices, such as the Principle of Least Privilege, are integral to responsible system design. These strategies often involve Just-In-Time (JIT) access or Privilege Escalation mechanisms. It is essential to log all actions associated with these mitigations for auditing and compliance purposes.	C2	Who activated the 'Break Glass Administrator' role, when it was activated, and which linked request authorized this action? When did user@domain.com escalate their permissions for network zone NZ1, and what was the reason for the escalation?

Type

Description

Data Class

Examples

Log destination

Unique Multi Tenant

Unique Single Tenant

Customer Managed Tenant

IDP
Audit Logs

The range of Identity Providers (IDPs) is extensive, covering solutions such as Azure Entra ID, Active Directory Federation Services (ADFS), and custom implementations. In more complex environments, multiple IDPs may be deployed, including scenarios involving local Active Directory (AD) or LDAP.

Who created the user account (user@domain.com), when it was created, under whose approval, and which ticket was associated with the request?
When was the user assigned to the 'AI Engineers' team, under whose approval, and in response to which access request?

Logs are captured by Azure Activity Logs and retained for 90 days.

Logs are highly customized for each client, with respective providers responsible for their creation, storage, and retention.

Clients are accountable for the proper handling of these logs. Unique can be consulted for guidance and should be informed as necessary.

IDP
Privilege Escalation Logs

Security best practices, such as the Principle of Least Privilege, are integral to responsible system design. These strategies often involve Just-In-Time (JIT) access or Privilege Escalation mechanisms. It is essential to log all actions associated with these mitigations for auditing and compliance purposes.

Who activated the 'Break Glass Administrator' role, when it was activated, and which linked request authorized this action?
When did user@domain.com escalate their permissions for network zone NZ1, and what was the reason for the escalation?

Industry Standard Reference: Azure Activity Logs

2. Resource Layer

This layer consists of the essential compute, network, and storage resources that support the application. It handles the infrastructure necessary to manage workload communications and operations. The Resource Layer is responsible for logging both Control Plane and Data Plane activities.

Key Logs:

Type	Description	Data Class	Examples	Log destination
Unique Multi Tenant	Unique Single Tenant	Customer Managed Tenant
Control Plane Activity Logs	All resources provide Control Plane actions. These actions are recorded in the Activity Log, detailing which user performed the action and when it occurred	C2	Who increased the database server size from 4 CPUs to 8 CPUs? Who modified the size of the Kubernetes Node Pool?	Logs are captured by Azure Activity Logs and retained for 90 days.	Logs are highly customized for each client, with respective providers responsible for their creation, storage, and retention. Clients are accountable for managing these logs appropriately. Unique may be consulted for support and should be notified as needed.
Data Plane Activity Logs	Not all resources support Data Plane actions. For those that do, the Activity Log captures which user performed the action and when it occurred.	C2	Who, if possessing escalated privileges, accessed the key 'some-key' in the last n weeks? Who rotated the 'rabbitmq-password' secret yesterday?

Type

Description

Data Class

Examples

Log destination

Unique Multi Tenant

Unique Single Tenant

Customer Managed Tenant

Control Plane
Activity Logs

All resources provide Control Plane actions. These actions are recorded in the Activity Log, detailing which user performed the action and when it occurred

Who increased the database server size from 4 CPUs to 8 CPUs?
Who modified the size of the Kubernetes Node Pool?

Logs are captured by Azure Activity Logs and retained for 90 days.

Logs are highly customized for each client, with respective providers responsible for their creation, storage, and retention.

Clients are accountable for managing these logs appropriately. Unique may be consulted for support and should be notified as needed.

Data Plane
Activity Logs

Not all resources support Data Plane actions. For those that do, the Activity Log captures which user performed the action and when it occurred.

Who, if possessing escalated privileges, accessed the key 'some-key' in the last n weeks?
Who rotated the 'rabbitmq-password' secret yesterday?

Industry Standard Reference: Azure Control and Data Plane

3. Kubernetes Layer

Kubernetes orchestrates the deployment and management of containerized applications. This layer captures logs that show how resources within the Kubernetes cluster are used and modified.

Key Logs:

Type	Description	Data Class	Examples	Log destination
Unique Multi Tenant	Unique Single Tenant	Customer Managed Tenant
Audit Logs	Kubernetes, including but not limited to Azure AKS, operates as its own ecosystem with distinct Control Plane and Data Plane components that generate dedicated audit logs.	C2	Who scaled the 'node-chat' service from 2 to 4 pods, and when did this action occur? Who deleted the Pod Disruption Budget for a specific service, and when was it done?	Currently under discussion.	Logs are streamed to the Azure Log Analytics Workspace. Data is stored in the Analytics Table with a retention period of 31 days.	Clients utilizing Kubernetes typically have dedicated logging mechanisms for Kubernetes auditing, ensuring that these logs are directed to the client's designated logging destination.

Type

Description

Data Class

Examples

Log destination

Unique Multi Tenant

Unique Single Tenant

Customer Managed Tenant

Audit Logs

Kubernetes, including but not limited to Azure AKS, operates as its own ecosystem with distinct Control Plane and Data Plane components that generate dedicated audit logs.

Who scaled the 'node-chat' service from 2 to 4 pods, and when did this action occur?
Who deleted the Pod Disruption Budget for a specific service, and when was it done?

Currently under discussion.

Logs are streamed to the Azure Log Analytics Workspace.

Data is stored in the Analytics Table with a retention period of 31 days.

Clients utilizing Kubernetes typically have dedicated logging mechanisms for Kubernetes auditing, ensuring that these logs are directed to the client's designated logging destination.

Industry Standard Reference: Kubernetes Audit Logs

4. Workload Layer

The Workload Layer contains the actual applications running on Kubernetes. Logs from this layer are critical for tracking user interactions, service requests, and application performance.

Key Logs:

Type	Description	Data Class	Examples	Log destination
Type	Description	Data Class	Examples	Unique Multi Tenant	Unique Single Tenant	Customer Managed Tenant
Service Audit Logs	Unique AI provides a range of APIs, both public and internal. Audit logs meticulously document every request to ensure compliance with legal regulations and requirements.	C3	Who listed all ingested files in a failed state, when did it happen, how was it done, and from where was the request made? Who listed all users, including their last names, when did it occur, how was it done, and from where was the request made?	Currently under discussion.	Data is stored in isolated Azure Blob Containers and retained for the legal hold period, which defaults to 5 years. These containers are append-only and tamper-proof.	Clients have the option to provide a dedicated disk for mounting and writing files, or they can integrate an additional logging drain, which will be implemented at the standard daily rate.
Service Application Logs	Unique AI generates standard application logs (such as debug, info, warn, and error logs, when enabled). These logs are valuable for investigating incidents and verifying application functionality.	C1	How many database connections were created by the 'node-chat' service? How many results were returned from the Vector Database?	Logs are streamed to Azure Disks via `stdout` and `stderr`. Logs are retained for 31 days.	Logs are streamed to the Azure Log Analytics Workspace from `stdout` and `stderr`. A basic table of logs is retained for 7 days.	Clients using Kubernetes typically have dedicated logging drains for `stdout` and `stderr`, ensuring that these logs are directed to the client's designated logging destination.
IDP Audit Events	Unique utilizes an existing Identity Provider (IDP) solution, which generates its own audit events to track all actions related to authentication and authorization.	C2	Who assigned the 'chat.admin' role to user@domain.com last week? Who issued themselves a token, when did it happen, and potentially from which device?	Data is stored in the corresponding PostgreSQL database and retained for the legal hold period, which defaults to 5 years.	Data is stored in the corresponding PostgreSQL database and retained for the legal hold period, which defaults to 5 years.	Data is stored in the corresponding PostgreSQL database and retained for the legal hold period, which defaults to 5 years. Additional post-write storage or archival solutions are available at the standard daily rate.

Industry Standard Reference: Application-specific monitoring tools like Azure Monitor.

5. DevOps Tooling Layer

This layer contains the tools and processes used during the development lifecycle, including version control, build systems, and deployment pipelines. Logs from this layer ensure that the development process is traceable and secure.

Key Logs:

Type	Description	Data Class	Examples	Log destination
Type	Description	Data Class	Examples	Unique Multi Tenant	Unique Single Tenant	Customer Managed Tenant
Source Control Audit Logs	Nearly all organizations, including clients and Unique, manage as much as possible through code. This includes the product, its configuration, the infrastructure configuration, and even the provider setup. As a result, access to these codebases must be properly audited.	C2	When was a specific user invited, under whose approval, and which access request was associated with the invitation? How frequently does a specific user log in, and from which geographical region?	Logs are sourced and managed by Unique through GitHub Enterprise and retained for 6 months		Logs are highly customized for each client, with respective providers responsible for their creation, storage, and retention. Clients are accountable for the proper management of these logs. Unique may be consulted for guidance and should be informed as needed.
Commit History	All changes to the codebase are automatically logged by the version control system.	C1	Who modified the product to function as 'B' instead of 'A'? Who changed the configuration for the Vector Database?	Logs generated by Git itself are retained as long as the repository exists, remaining immutable as Unique enforces a no-rewrite policy on the main branches.
Test Automation Audit Logs	All changes to version control undergo testing in accordance with Unique's testing procedures. Logs from these tests are maintained for traceability and auditing.	C2	When change 'X' occurred, which tests were executed and what were the results? If tests failed but the change was still approved, who granted the exception and why?	Logs related to actions in GitHub are also sourced and managed by Unique through GitHub Enterprise and GitHub Actions, with a retention period of 90 days.
Test Automation Logs	Test executions generate logs that are informational and can be used to troubleshoot issues in case of test failures.	C1	Log output from the test framework while testing case 'A' or 'B.'	Logs produced by GitHub Actions are retained for 90 days.
Build Audit Logs	When a change is built (including its artifacts), the actions taken are logged for traceability and auditing purposes.	C2	When was change 'C' built, and who triggered the build? Which artifacts were generated to proceed further along the pipeline?	Sourced and managed by Unique using GitHub Enterprise and GitHub Actions, with a retention period of 90 days.
Build Logs	Build processes also generate logs, which are informational and assist in debugging if a build fails.	C1	Log output from the build system while applying the tested change from version control.	Generated by Git and GitHub Actions, with a retention period of 90 days.
Deployment Audit Logs	Whenever a system is updated, the relevant actions are logged for traceability and auditing. This applies to both deployments and releases, even in environments where these steps are separated.	C2	When and by whom was the deployment approved or executed, and under which change request (who triggered the automation)? Deployments require segregated duties and sign-off, so the person who signed off is also logged. Who reverted a deployment, when, and for what reason?	Logs are produced by ArgoCD (the GitOps orchestrator) and are retained for the legal hold period, which defaults to 5 years.
Deployment Logs	Deployment processes generate logs that are informational and can be used to troubleshoot issues in the event of a deployment failure.	C1	Log output from the deployment system while applying the tested and built change.

Industry Standard Reference: GitHub Enterprise Logs for managing source control audits.

Connection Between the Layers

Each of the layers builds on the one below it, creating a hierarchy that supports full operational visibility:

Provider Layer manages the cloud infrastructure and supports the Resource Layer.
The Resource Layer provides compute, network, and storage resources for the Kubernetes Layer to orchestrate containerized applications.
Kubernetes Layer manages the lifecycle of workloads, providing infrastructure for the Workload Layer to function.
Finally, the DevOps Tooling Layer ensures traceability and control over the entire development and deployment pipeline, connecting all other layers for comprehensive monitoring.