Core Sync Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• dev/knowledge/sync-architecture.md
• dev/knowledge/schema-mapping.md
• dev/knowledge/incremental-and-cache.md
• infrahub_sync/cli.py
• dev/constitution.md
• infrahub_sync/generator/init.py

Infrahub Sync moves data from a source system of record to a destination system of record. The architecture uses the diffsync framework to load objects into in-memory stores, calculate differences, and reconcile the destination to match the source. The system prioritizes idempotency, safety, and reproducibility by enforcing a read-only path as the default behavior.

Sources: dev/knowledge/sync-architecture.md:5-9, dev/constitution.md:15-18

System Components

The architecture consists of four primary functional blocks: the DiffSync foundation, Adapters, the Potenda engine, and the code-generation path.

DiffSync Foundation

DiffSync manages object representation and comparison. It uses two core concepts:

• Adapter: Loads objects into a store and performs create, update, and delete operations.
• Model: Defines a typed record with _identifiers for stable identity and _attributes for comparison.

Sources: dev/knowledge/sync-architecture.md:11-19

Adapters and Mixins

Every system connector honors a standard contract through two mixins: DiffSyncMixin for adapters and DiffSyncModelMixin for models. An adapter acts as a source when reading data and as a destination when writing data. The direction is fixed per run, ensuring the source remains read-only.

Sources: dev/knowledge/sync-architecture.md:21-31, dev/constitution.md:46-48

The Potenda Engine

The Potenda engine orchestrates the synchronization process in three stages:

Stage	Action	Description
Load	load_both_sides()	Iterates through models and triggers model_loader() for both source and destination.
Diff	diff()	Compares stores and produces a structured set of reconciliation actions.
Sync	sync()	Applies actions (create/update/delete) against the destination in dependency order.

Sources: dev/knowledge/sync-architecture.md:33-46

Data Flow and Orchestration

The following diagram illustrates the lifecycle of a synchronization event, from configuration loading to destination reconciliation.

The diagram shows how the Potenda engine uses the Schema Mapping from config.yml to drive the Adapter interactions.
Sources: dev/knowledge/sync-architecture.md:33-46, dev/knowledge/schema-mapping.md:5-10

Code Generation Path

Infrahub Sync avoids manual model creation by generating DiffSync model classes dynamically.

1. Configuration: A user defines a config.yml with a schema_mapping block.
2. Generation: The infrahub-sync generate command reads the config and destination schema to render Python classes using Jinja2 templates.
3. Loading: The plugin_loader resolves the adapter class and attaches generated models to the adapter at runtime.

Sources: dev/knowledge/sync-architecture.md:48-57, infrahub_sync/generator/init.py:180-190

Incremental Sync and Caching

To optimize performance for large datasets, the architecture supports incremental extraction and plan caching.

Cursor Tiers

Adapters specify their extraction strategy using CursorTier values.

• NONE (0): Always performs a full extract.
• ID (1): Compares stable ID sets to detect changes.
• TIMESTAMP (2): Filters by modification time (e.g., last_updated__gte) to extract only changed records.

Sources: dev/knowledge/incremental-and-cache.md:9-20

Execution Plan Caching

The engine can decouple the comparison phase from the application phase. The diff command writes a Parquet-formatted plan to disk. The apply command reads this plan to perform the sync without re-extracting data from the source.

Sources: dev/knowledge/incremental-and-cache.md:38-42, infrahub_sync/cli.py:108-112

Safety Guardrails

The architecture includes mechanisms to prevent accidental data corruption or loss.

• Dry-Run by Default: Commands like list, diff, and generate are non-mutating. Mutation requires the explicit sync command and human approval.
• Row-Count Guardrail: The engine compares new destination row counts against a persisted baseline. If the count drops beyond a specific threshold, the run fails unless the --allow-rowcount-drop flag is provided.
• Idempotency: Reconciliation is designed to be safe for re-running. Convergence results in zero changes if the source and destination are already in sync.

Sources: dev/constitution.md:20-25, dev/knowledge/incremental-and-cache.md:52-56, infrahub_sync/cli.py:155-159

CLI Interaction Flow

The CLI manages the user interface to the sync engine, enforcing the safety principles defined in the architecture.

The sequence diagram details the interaction between the CLI, Potenda, and Adapters during diff and sync operations.
Sources: infrahub_sync/cli.py:75-80, infrahub_sync/cli.py:140-145, dev/knowledge/sync-architecture.md:38-46

Summary

Core Sync Architecture provides a framework for infrastructure data reconciliation. By combining the diffsync model with a code-generation pipeline and strict safety guardrails, it enables reproducible and safe data movement across heterogeneous systems. The architecture supports both full and incremental workflows while ensuring that destructive operations remain deliberate and transparent.

Sources: dev/knowledge/sync-architecture.md, dev/constitution.md