Diffing & Merging
Relevant source files
The following files were used as context for generating this wiki page:
Infrahub manages infrastructure data using a graph-based version control system. It integrates changes between branches through a sophisticated diffing and merging pipeline. This system isolates changes, detects conflicts, and manages the lifecycle of dependent artifacts and generators.
The platform uses Git-like branching to allow users to modify the graph in isolation. Merging integrates these isolated changes from a source branch into a target branch, such as develop or stable. During this process, Infrahub captures enriched diffs to drive selective regeneration of infrastructure artifacts.
Sources: AGENTS.md:3-8, dev/guidelines/git-workflow.md:7-12
Merge Orchestration and Diff Capture
The BranchMergeOrchestrator manages the lifecycle of a merge operation. It captures the enriched diff of the source branch before the changes are frozen and integrated into the target branch. This enriched diff provides a granular view of every added, modified, or deleted node in the graph.
Diff Serialization and Caching
The system converts the raw enriched diff into a NodeDiff summary. This summary is serialized into a JSON payload and stored in the InfrahubCache (Redis) with a specific Time-to-Live (TTL).
- Cache Key:
branch_merge:diff_id:{diff_root_uuid}:diff_summary - TTL: Two hours (
KVTTL.TWO_HOURS) - Storage: Neo4j stores the graph and enriched diff persistence, while Redis stores the summary payload.
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:16-30, specs/archive/ifc-2704-incremental-merge-regen/tasks.md:46-52
Merge Lifecycle Flow
This diagram shows how diff data flows from the initial merge request to the post-merge dispatcher.
The flowchart illustrates the sequential steps of capturing, caching, and processing diff data during a branch merge.
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:16-25, specs/archive/ifc-2704-incremental-merge-regen/tasks.md:54-62
Selective Regeneration
Selective regeneration ensures that only affected artifacts and generators run after a merge. This reduces task volume and prevents unnecessary compute cycles. The behavior is controlled by the selective_execution_after_merge configuration flag.
Definition-Level Gates
The system uses predicates to determine which infrastructure definitions require execution.
| Predicate | Purpose |
|---|---|
_query_changed | Checks if the query_id associated with a definition appears in the diff summary. |
_definition_changed | Evaluates if the CoreArtifactDefinition or CoreGeneratorDefinition node itself was modified. |
_transform_changed | Detects changes in the transformation logic or script closure. |
MODIFIED_KINDS | Intersects the kinds modified in the diff with the query_models of a definition. |
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:86-95, specs/infp-409-artifact-regen-triggers/plan.md:11-15
Member-Level Impact Analysis
If a definition is selected, the system further narrows the execution scope to specific members of a group. It reconciles group membership on the target branch and identifies which members are impacted by field-level changes.
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:96-102, specs/archive/ifc-2704-incremental-merge-regen/tasks.md:88-94
Post-Merge Failure Recovery and Fallbacks
Infrahub prioritizes correctness by ensuring no artifact remains stale. If the selective path encounters uncertainty, it reverts to full regeneration.
Fallback Conditions
The PostMergeRegenerationDispatcher triggers full regeneration of all definitions if any of the following occur:
- The
selective_execution_after_mergeflag is disabled. - The merge diff summary is missing from the cache.
- The cache payload is malformed or unreadable.
- A definition has a null
fingerprint, indicating a pre-existing state without tracked changes. - Transformation dependencies are incomplete or unresolved.
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:31-36, specs/archive/ifc-2704-incremental-merge-regen/tasks.md:116-124
Selective vs. Fallback Decision Flow
This diagram shows the decision logic used by the dispatcher to select an execution path.
The flow diagram describes the logic gates used to decide between selective execution and the full regeneration fallback path.
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:16-25, specs/archive/ifc-2704-incremental-merge-regen/tasks.md:116-120
Direct Merge Generator Cascade
Direct merges (merges not originating from a Proposed Change) require specialized handling when generators are involved. Generators can mutate graph data, which in turn might affect artifacts.
Cascaded Execution Logic
- The dispatcher identifies if a direct merge dispatched one or more generators.
- It awaits the completion of all generator runs.
- It captures the nodes modified by those generators, scoped to each generator's tracking group.
- It regenerates artifacts that read the tracked output of those generators.
- If a generator run fails, the system regenerates all artifacts without re-running the generators to ensure consistency.
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:126-135, specs/archive/ifc-2704-incremental-merge-regen/tasks.md:154-162
Summary of Components
| Component | Description |
|---|---|
BranchMergeOrchestrator | Orchestrates the merge and performs initial diff capture. |
NodeDiff | A serialized TypedDict representing graph changes for selection logic. |
InfrahubCache | Redis-backed storage for merge diff summaries. |
PostMergeDispatcher | Threads the cache key and proposed change ID to follow-up tasks. |
RegenerationSelector | Analyzes diff summaries and group members to filter execution. |
Sources: specs/archive/ifc-2704-incremental-merge-regen/plan.md:73-85, specs/archive/ifc-2704-incremental-merge-regen/tasks.md:54-68
Infrahub's diffing and merging infrastructure provides a robust framework for managing infrastructure at scale. By combining enriched diff capture with selective regeneration gates, the platform maintains graph consistency while minimizing post-merge processing time.