DiffSync Framework Integration
Relevant source files
The following files were used as context for generating this wiki page:
Infrahub Sync builds upon the DiffSync framework to reconcile data between infrastructure systems. It represents every object as a model instance with a stable identity, loading objects into in-memory stores to calculate differences and apply changes to a destination system. The integration uses custom mixins to extend DiffSync functionality, providing a declarative approach to synchronization through YAML configurations.
Sources: README.md:12-14, dev/knowledge/sync-architecture.md:6-14
Core Architecture and Components
The integration utilizes three primary components to manage data flow: Adapters, Models, and the Potenda Engine.
Component Overview
| Component | Responsibility | Base Classes |
|---|---|---|
| Adapter | Connects to a system, loads objects into a store, and performs CUD (Create, Update, Delete) operations. | DiffSyncMixin, Adapter |
| Model | Defines a typed record with identifiers (natural key) and attributes (comparable fields). | DiffSyncModelMixin, DiffSyncModel |
| Potenda Engine | Orchestrates the sync lifecycle: loading, diffing, and applying changes. | N/A |
Sources: dev/knowledge/sync-architecture.md:9-15, dev/knowledge/adapter-anatomy.md:9-24
Class Hierarchy and Mixins
Infrahub Sync layers two mixins over DiffSync base classes to provide project-specific features like schema mapping and incremental sync support. Adapters must declare the mixin before the DiffSync base class to ensure mixin methods take precedence.
The diagram shows the inheritance structure where Infrahub Sync mixins extend the standard DiffSync Adapter and Model classes.
Sources: dev/knowledge/adapter-anatomy.md:9-25, dev/guidelines/writing-an-adapter.md:10-21
Synchronization Lifecycle
The Potenda engine orchestrates synchronization in three distinct stages: Load, Diff, and Sync.
1. Load Stage
The engine calls load_both_sides(), which triggers load() on both the source and destination adapters. Adapters use a model_loader() to fetch raw records, apply filters, and convert them into DiffSync model instances. Loading occurs concurrently by default to improve performance.
2. Diff Stage
The engine executes the DiffSync comparison logic. It compares the in-memory stores of the source and destination to produce a structured set of required actions (Create, Update, or Delete).
3. Sync Stage
The engine applies the calculated actions against the destination system. It respects dependency orders—either defined in config.yml or auto-computed from references—to ensure related objects are created in the correct sequence.
This flowchart illustrates the sequential execution of a sync run from the CLI entry point through the Potenda engine's orchestration stages.
Sources: dev/knowledge/sync-architecture.md:25-36, infrahub_sync/cli.py:175-225
Framework Extensions and Flags
Infrahub Sync leverages specific DiffSync flags to control reconciliation behavior. These flags are typically managed via the SyncConfig and CLI options.
- SKIP_UNMATCHED_DST: Prevents deletion of objects found in the destination that are missing in the source (Default behavior).
- SKIP_UNMATCHED_SRC: Skips the creation of objects found in the source that are missing in the destination.
- SKIP_MODIFIED: Prevents updating existing objects even if attributes differ.
Sources: README.md:101-103, dev/constitution.md:25-30
Incremental Extraction and Caching
The integration supports an incremental path to avoid full extracts of large datasets. Adapters declare a CursorTier (NONE, ID, or TIMESTAMP) via cursor_tier_for(). When incremental sync is enabled, the engine uses list_changed_since() to fetch only modified records and reuses a cached diff plan from disk.
Sources: dev/knowledge/incremental-and-cache.md:6-25, infrahub_sync/cli.py:114-125
Declarative Model Generation
Unlike standard DiffSync implementations where models are manually coded, Infrahub Sync generates model classes dynamically. The infrahub-sync generate command reads the schema_mapping from config.yml and uses Jinja2 templates to render Python classes that inherit from DiffSyncModelMixin. This ensures that model attributes and identifiers match the infrastructure schema exactly.
Sources: dev/knowledge/sync-architecture.md:41-49, dev/knowledge/schema-mapping.md:6-25
# Example of a generated model structure based on schema mapping
class InfraDevice(DiffSyncModelMixin, DiffSyncModel):
_kind = "InfraDevice"
_identifiers = ["name"]
_attributes = ["status", "role", "site"]
name: str
status: str | None = None
role: str | None = None
site: str | None = None
Sources: dev/knowledge/schema-mapping.md:12-25, dev/knowledge/adapter-anatomy.md:16-24
Execution Safety and Idempotency
The integration prioritizes safety by enforcing read-only operations by default. Commands like list and diff do not mutate state. The sync command requires explicit user instruction and uses DiffSync's reconciliation logic to ensure idempotency; re-running a sync on a converged system produces no changes.
Sources: dev/constitution.md:21-40, README.md:89-94