Directory Sync Services
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Directory Sync Services in flext-ldap provide high-level mechanisms for synchronizing LDAP Data Interchange Format (LDIF) data into an LDAP directory. This system is designed to handle both single-file imports and complex, multi-phase synchronization workflows where directory objects are updated or created in a specific sequence (e.g., users before groups).
The core of this functionality is implemented through the FlextLdapSync mixin, which integrates with the ldap API facade to provide enterprise-grade synchronization features including batch upserts, progress monitoring via callbacks, and robust error handling with configurable retry logic.
Sources: src/flext_ldap/services/sync.py:12-19, examples/README.md:158-167
Architecture and Core Logic
The synchronization service operates as a mixin within the Method Resolution Order (MRO) of the LDAP public facade. It extends FlextLdapOperations to utilize core directory actions like batch upserting.
Synchronization Flow
The process follows a structured path from LDIF parsing to batch LDAP operations:
The diagram shows the transition from parsing raw LDIF files to executing batch LDAP operations and the higher-level loop managing multiple phases.
Sources: src/flext_ldap/services/sync.py:73-100, src/flext_ldap/services/sync.py:124-142
Key Components
| Component | Purpose |
|---|---|
FlextLdapSync | Primary mixin class providing synchronization logic. |
SyncPhaseConfig | Model defining sync behavior (retries, server type, callbacks). |
BatchUpsert | Core operation that determines whether to add or modify an entry based on existing state. |
LDIF Parser | Integration with flext-ldif to convert files into entry models. |
Sources: src/flext_ldap/services/sync.py:19-21, tests/unit/test_models_sync.py:10-25, src/flext_ldap/protocols.py:228-235
Synchronization Operations
Single Phase Synchronization
The sync_phase_entries method handles the synchronization of a single LDIF file. It parses the file into entry models and performs a batch upsert. If no entries are found, it returns a successful result with zeroed statistics.
Sources: src/flext_ldap/services/sync.py:124-150
Multi-Phase Synchronization
sync_multiple_phases allows for sequential processing of multiple LDIF files, which is critical for maintaining referential integrity in LDAP (e.g., ensuring Organizational Units exist before users).
- Sequential Processing: Files are processed in the order provided in the mapping.
- Error Boundaries: The
stop_on_errorflag determines if a failure in one phase aborts the entire sequence. - Aggregation: Overall success rates and durations are calculated across all phases.
Sources: src/flext_ldap/services/sync.py:73-100, tests/unit/test_sync.py:125-145
Callback and Progress Monitoring
The service supports two types of progress callbacks to allow UI or logging integration during long-running sync tasks.
Callback Signatures
- Single-Phase Signature: Receives
(current, total, dn, stats). - Multi-Phase Signature: Receives
(phase, current, total, dn, stats).
The service automatically normalizes these signatures using _prepare_phase_callback, wrapping multi-phase callbacks to fit the single-phase requirement of the underlying batch engine when necessary.
This sequence illustrates how callbacks are normalized before being passed to the batch operation engine.
Sources: src/flext_ldap/services/sync.py:46-68, src/flext_ldap/services/sync.py:169-191
Data Models and Statistics
The sync service relies on structured models to report detailed outcome metrics.
Phase and Multi-Phase Results
| Field | Type | Description |
|---|---|---|
total_entries | int | Total count of entries found in source LDIF. |
synced | int | Count of entries successfully created or updated. |
failed | int | Count of entries that encountered errors. |
skipped | int | Count of entries skipped (e.g., due to specific rules). |
success_rate | float | Percentage of successful operations. |
duration_seconds | float | Time taken to process the phase or total sync. |
Sources: tests/unit/test_models_sync.py:65-90, src/flext_ldap/services/sync.py:155-165
Error and Retry Logic
Configuration allows for resilient synchronization against transient directory errors.
- Default Retries: Common errors like "session terminated", "not connected", and "socket" issues trigger automatic retries.
- Max Retries: User-defined limit for operation attempts.
- Stop on Error: Boolean flag to control whether a single failure stops the batch or phase.
Sources: src/flext_ldap/services/sync.py:136-140, tests/constants.py:200-210
Conclusion
Directory Sync Services provide the necessary abstraction to transition from static LDIF data to a live LDAP directory. By managing complex callback normalization and phase-based execution, it ensures that large-scale data migrations or updates are performed reliably and with full observability into the synchronization progress and results.