Unique maintains a Reference Architecture for CMT, where we collect the most simple but secure setup with an acceptable timeline. Unique has seen with various FSI clients, that the landscape / Landing Zone surroundings always look different and thus has created this checklist so clients can internally review, what is needed, missing or different in order to accommodate Unique.

Most clients run more than one deployments of Unique on site, probably DEV and PROD or even an additional in between like QA.

Unique advises to start with a less-governed or a bit looser tenant in order to get to know Unique and the surrounding resources. Of course, Unique recommends not to use this cluster for sensitive workloads but to just train and PoC the deployment!

In a second iteration or cluster, the client can harden the setup or modify it to their standards and policies.

Starting directly with a hardened or isolated setup will from experience manyfold the timeline and the first run of Unique on the clients tenant!

Note that this list is not conclusive as Unique discovers new cases with every tenant implemented. The list will be extended with every further discovery.

Even though the list appears to be extensive, multiple clients have shown that efficient installations are possible when the involved staff has all necessary knowledge, tools and permissions in place!

Introduction

• Purpose: This checklist is designed to assist clients in preparing their Azure environments for the installation of the Unique application. It outlines necessary steps and checks to ensure a smooth and secure implementation.

• Scope: This checklist provides guidance for initial setups and advanced configurations, focusing primarily on Azure environments.

• Audience: Intended for IT administrators and cloud architects who are responsible for setting up and securing Azure environments in preparation for deploying Unique.)

General

Even though Unique can advise clients at a rate, having fundamental if not extensive experience with the resources used is crucial.

• If ClickOps instead of Code or Automation is used, Unique cannot assist in setup of further clusters as they will derive from the only config Unique is aware about.

• Unique recommends using always RBAC (for AKS and KeyVault as well as all other resources), using older access principles can have a functionality and security impact and is not encouraged by Unique.

• Advanced understanding of Azure Entra ID incl. Privileged Access Management, Azure ARM, its RBAC permissions system as well as the different resources themselves and the Azure hierarchy around them.

• Expertise with Networking within Azure from Public IPs, over Application Gateways, to Virtual Networks and maybe their peering down to subnets and their Network Security Groups etc.

• Strong expertise of Azure Kubernetes Service including all its implied dependencies

  • few examples: Virtual Machine Scale Sets, Storage Classes, Disk Encryption Sets, Managed Prometheus, Managed Grafana etc.

• If the client relies on custom certificates or even custom certificate authorities, being familiar with the internals of the client regarding this is a must

  • Also the Unique mobile recording app cannot deal with custom certificates as today (can be investigated in a budgeted integration project whereas the Minimum Duration is ~8 weeks).

• Familiarity of staff with tooling helm, helmfile, kubectl, probably az and maybe k9s

• If Intune and Unique mobile recording app is used, client must be familiar with Intune and its implications on Azure Entra ID, Conditional Access Policies, Enterprise Applications, App registrations etc.

• Unique relies on the Azure Application Gateway Ingress Controller, if not already, the client must make themselves familiar with it.

• Similar projects have been implemented a handful of times within the clients environment already and processes and practises are known especially around AKS and its security. Unique can be the clients guinea pig but the experience and timeline will be accordingly…

• Clients are familiar with Uniques Infrastructure requirements/pre-requisites.

Landing Zone / Networking

• DEV or first setup should not restrict egress traffic even though Unique does not actively need it

• For initial setup, the Network Security Groups should be kept as Unique terraform code and later potentially hardened by the customer

• The outside world of Uniques resources or Landing Zone is unknown. In case clients do not want to use Uniques Application Gateway, up-stream prepend another Gateway and potentially use a private gateway for Uniques resources the client must manage these customisations related to networking outside Uniques reach or Landing Zone by themselves and ensure they have a concept, process, permissions and implementations ready or already applied.

• Practices of the client on how ingresses secure their traffic with Certificates must be known and ready to use in the cluster if cert-manager can not be used (needs cluster to be reachable from the internet)

  • Certificates must be pre-created or be ready to be created within each namespace so ingresses can reference them (Uniques helm charts as well as the public ones support this use case)

  • If Unique mobile recording app is used it won’t work as long as the client does not somehow force-install the custom certificates or CA onto the clients devices

• Client has clear documentation and plan as well as experience wether the solution is multi Availability Zones or not

  • Has influence mainly on the Storage Classes and the VM placement

• Client ensures that the DNS and Zone delegations are correctly setup so that the url x.client.com will work and that all sub-domains are registered as defined in the terraform code

  • Have the URL ready where the Application Gateway will listen

  • Unique uses subdomains as in example: id.x.client.com, gateway.x.client.com. If clients do not have this option, x-id.client.com or similar flatter structures work as well as long as the IP addresses are correctly bound/mapped.

• Internal URLs (as in Private Application Gateways or DNS Zones etc) or making the cluster inaccessible from the internet might lead to issues that will need investigation

  • The SharePoint integration works with Power Automate which is part of Microsoft’s 365 suite.

  • This is considered to be “public” or “outside” of the “internal” environment where the Unique solution is deployed.

  • The calls from Power Automate to the Unique solution need to go through a general API gateway.

  • An API Management service might need to be configured / setup so that the traffic can be allowed to pass to the “internal” environment.

    • https://learn.microsoft.com/en-us/azure/api-management/

  • Private clusters will not work with the Unique Mobile App (only affects recording tenants)

  • These customisations are to be made by the client under potential advisory at a rate.

• Prohibiting the use of Public IPs needs to be solved by the client by customising the setup on their side like in

  • https://learn.microsoft.com/en-us/azure/aks/egress-outboundtype

  • https://learn.microsoft.com/en-us/azure/aks/outbound-rules-control-egress#required-outbound-network-rules-and-fqdns-for-aks-clusters

• Ability for Kubernetes Nodes to pull images from the Internet

  • Or already have provisioned an own registry with a sync job to gather Uniques images after their release.

• Managed Identities / Workload Identities need to have permission to use the necessary services in the Landing Zone

  • e.g. the workload identities for the backend micro services that need access to the LLMs

  • The client needs to ensure that the workload identities have the needed permissions for the dedicated APIs (by either following the terraform setup that Unique provides or setting it up themselves)

Management Group / Subscription

• Contributor or Owner rights on the management group, primary subscription + the following Data Actions in either management group or subscriptions

data_actions = [
  # Virtual Machine User Login
  "Microsoft.Compute/virtualMachines/login/action",
  "Microsoft.HybridCompute/machines/login/action",
  # Key Vault Secrets Officer
  "Microsoft.KeyVault/vaults/secrets/*",
  # Key Vault Crypto Officer
  "Microsoft.KeyVault/vaults/keys/*",
  "Microsoft.KeyVault/vaults/keyrotationpolicies/*",
  # Azure Kubernetes Service RBAC Cluster Admin
  "Microsoft.ContainerService/managedClusters/*",
  # Grafana Viewer
  "Microsoft.Dashboard/grafana/ActAsGrafanaViewer/action"
]

Resources (Groups)

• Azure VM (via Bastion or not) that can access all Unique-Provisioned KeyVaults and the Kubernetes Private API, the identity of the VM must be KeyVault Reader on same vaults

  • VM needs tooling, namely helm, helmfile, kubectl, probably az and maybe k9s

  • VM needs internet access to download public helm charts from ArtifactHub

    • or all charts need to be helm pulled onto the machine and the helmfiles customised accordingly

• Accessible Log Analytics Workspace where all Pod logs are visible

• Accessible Managed Grafana with all default metrics

• Not using cert-manager but own certificates are possible but need further configuration on the clients side to inject the certificates and/or the CA certificates into every deployment

• Cert-Manager won’t work if the cluster is inaccessible from the internet at all

  • Can be switched to DNS challenges but still the Zone would need to face the internet

  • If not exposed, needs investigation how clients normal approach is and this approach must be used and potentially ingresses adapted

• If deployments (PV[C]) or cluster use encrypted disks, helm charts must be customised on clients side to reference the correct encryption set

• If the AKS Network derives from Uniques recommendation (terraform) it must be ensured that enough IP-addresses are allocatable in the range for all pods to start

• Unique is known to be challenging to be installed in a shared cluster due to the CRDs that need to be created for https://tyk.io/docs/tyk-oss-gateway/ or https://cert-manager.io if used. The client must provide a cluster where CRDs can be installed OR pre-install the CRDs themselves.

• If subnet sizes vary from the recommendation, scaling can be impacted and must be solved by the client

Special use cases

• If SSO Login is needed, Entra ID Application Registration must be created or requested or process must be known.

• If Unique has no access or accounts in the clients IDP/Entra, they can never investigate or triage a bug except via labour intensive screen sharing or cumbersome log analysis.
  The support or RUN model chosen will then determine how bug triage or investigations take place.

Licenses

• If Intune is required, client needs an Intune license on the tenant. Custom Intune setups might break Uniques Mobile App if used and all related efforts consumed from Unique are charged at a rate.