Confluence Connector - Flows

Content Sync Flow

The content sync flow runs periodically (default: every 15 minutes) to synchronize labeled Confluence pages and their attachments to Unique.

Overview

Sequence Diagram

The connector is stateless -- it does not maintain local state between sync cycles. Change detection is performed by the Unique platform's file diff API, called once per Confluence space.

Scope Hierarchy

The connector manages a two-level scope hierarchy:

1. Root scope -- Must pre-exist in the Unique platform. Configured via ingestion.scopeId. The connector grants itself access at initialization and marks the scope as owned by this Confluence instance on the first sync cycle. Subsequent sync cycles verify that the ownership mark still matches the configured Confluence instance.

2. Space scopes -- Created automatically as children of the root scope, one per Confluence space key. Access is inherited from the root scope.

Root Scope Ownership Validation

To prevent misconfiguration where the same Confluence instance is connected to multiple Unique organizations, or the same root scope is pointed at a different Confluence instance, the connector tags the root scope with an identifier derived from the Confluence instance on the first sync cycle:

• Cloud: the identifier is derived from the cloudId configured in the tenant YAML.

• Data Center: the identifier is fetched from the Data Center's application manifest endpoint (/rest/applinks/1.0/manifest). No authentication is required.

On every subsequent sync cycle, the connector reads the ownership mark from the root scope and compares it against the current Confluence instance. A mismatch aborts the tenant sync cycle with a fatal error.

Discovery Phase

Pages are discovered through a CQL-based label search, then attachments are optionally extracted from the already-fetched page objects.

Important: CQL search results are filtered by the authenticated user's permissions. Pages in spaces the service account cannot access are silently excluded from results. On Data Center, the service account's space access can optionally be restricted to specific spaces. When no spaces are specified, the service account has read access to all spaces. If space restrictions are configured, pages in excluded spaces never appear in discovery results and no error is produced.

CQL Queries

The connector uses Confluence Query Language (CQL) to discover pages. The exact CQL differs by instance type:

The space type filter varies by instance type. See the Configuration Guide for details on which space types are scanned per platform.

The type != attachment clause excludes attachments from top-level CQL results since they are fetched via the expand=children.attachment parameter on the page objects themselves.

Descendant Discovery

Pages labeled with ingestAllLabel trigger a descendant search:

1. Collect all page IDs that carry the ingestAllLabel

2. Batch IDs into groups of 100

3. For each batch, execute CQL: ancestor IN ({batch}) AND type != attachment

4. Deduplicate results with labeled pages by page ID

Attachment Extraction

When attachments.mode is enabled (default), attachments are extracted from the already-fetched page objects. Confluence inlines up to 25 attachments per page via the expand=children.attachment parameter during the CQL search. If a page has more than 25 attachments, additional attachments are fetched via pagination (Cloud uses the v2 attachments endpoint; Data Center uses v1 pagination via _links.next).

An attachment is accepted if: - Its mediaType (reported by the Confluence API) is in the allowedMimeTypes list (defaults cover PDF, the major Office formats, plain text, CSV, HTML, PNG, and JPEG) - Its file size does not exceed maxFileSizeMb (default: 200 MB) - The maxItemsToScan capacity has not been exhausted (pages count first, attachments use remaining capacity)

Images embedded in a page body (via the editor's drag/drop, paste, or "Insert image" actions) are stored by Confluence as regular page attachments and surface in the same expand=children.attachment results, so they are ingested through this path when image/png or image/jpeg is in allowedMimeTypes. Images inserted as external URLs are not attachments and are not ingested.

When attachments.imageOcr is enabled (default), each image content registration includes ingestionConfig.jpgReadMode = DOC_INTELLIGENCE_DEFAULT, which the Unique ingestion service merges over its environment defaults and the destination scope's own config (request body has highest precedence). This forces OCR-based processing on the worker so chunks are produced; without it, the worker default (NO_INGESTION) returns zero chunks and the worker raises FAILED_IMAGE.

Content Type Ingestion Map

The connector uses label-based discovery via CQL. After fetching, it skips three content types: database, whiteboard, and embed.

Content that passes the filter has its body.storage HTML extracted and ingested. Items with empty bodies are skipped. Descendants of skipped content types (such as sub-pages under a database) are still discovered and ingested.

Confluence Cloud

Confluence Data Center

Whiteboard, Database, Embed, Folder, and Live Doc are Cloud-only features and do not exist in Data Center.

File Diff Mechanism

The connector uses the Unique platform's server-side file diff API (/v2/content/file-diff) to detect changes between sync cycles. The connector does not compute local content hashes -- instead, it sends each item's key, url, and updatedAt timestamp to the diff endpoint, which returns categorized results. The diff is called once per Confluence space.

Change Detection Logic

File Diff Item Attributes

Each item sent to the diff API contains:

Safety Checks

After each space's file diff is computed, two guards run to prevent accidental full deletion of content. Both abort the current tenant sync cycle when triggered.

Partial deletions (removing some but not all files) are always allowed. To intentionally remove all content for a tenant, set the tenant status to deleted (see Configuration -- Tenant Status).

Ingestion Pipeline

The ingestion pipeline follows a 3-step process for each item: register, upload, and finalize.

Page Ingestion

For each new or updated page:

1. Fetch content -- The connector retrieves the full page content (HTML storage representation). The page is skipped if it is not found, if fetching fails, or if the page body is empty.

2. Extract labels -- Confluence labels are extracted from the page, excluding the configured sync labels. Labels are sorted alphabetically for deterministic ordering.

3. Register -- Sends metadata to the Unique ingestion API.

4. Upload -- The page body (Confluence storage format HTML) is uploaded to Unique.

5. Finalize -- Triggers downstream processing in Unique.

If any step fails for a page, the error is logged and that page is skipped. Other pages continue processing.

Attachment Ingestion

For each new or updated attachment:

1. Skip zero-byte -- Attachments with zero file size are skipped.

2. Register -- Sends metadata to the Unique ingestion API.

3. Download -- Streams the attachment from Confluence.

4. Upload -- The stream is uploaded to Unique with the original content type and content length.

5. Finalize -- Triggers downstream processing in Unique.

If any step fails, the stream is destroyed and the error is logged. Other attachments continue processing.

Deletion

Deleted items identified by the file diff are processed after ingestion:

1. Build content keys for each deleted item

2. Resolve keys to content IDs

3. Delete content by IDs

If no content is found for the given keys, a warning is logged and no deletion occurs.

Removed Space Cleanup

The file diff runs once per Confluence space that appears in the current discovery results. Spaces that have disappeared from discovery entirely (because the space was deleted in Confluence, or all ingest labels were removed from its pages) are not diffed and would otherwise leave orphaned content and scopes in Unique.

After the per-item deletion step, the connector detects orphaned space scopes by comparing the root scope's children against the set of Confluence space keys discovered during the current sync. For each orphaned space, the connector deletes both the space's files and the space scope itself. A failure for one space does not prevent the cleanup of other orphaned spaces.

Concurrency and Progress

Ingestion concurrency is controlled by processing.concurrency (default: 1). Pages are ingested first, then attachments -- not in parallel. All items in a batch are attempted even if some fail.

Progress is logged periodically. After all items complete, a summary is logged with the total, succeeded, and failed counts.

Error Handling

Error Handling Strategy

The connector applies scenario-specific behavior to keep sync cycles stable:

Retry Logic

Transient HTTP errors are retried automatically with backoff. Requests follow redirects (up to 10 redirections). Non-2xx responses that exhaust retries are thrown as errors with the status code and response content.

Related Documentation

• Technical Reference - Architecture overview, key concepts, ingestion pipeline

• Architecture - System components and infrastructure

• Configuration - Scheduler and processing settings

Standard References

• Confluence Cloud REST API - Atlassian Confluence Cloud API documentation

• Confluence Data Center REST API - Atlassian Confluence Data Center API documentation

• Confluence Query Language (CQL) - CQL reference for content search queries