Processing Results
Relevant source files
The following files were used as context for generating this wiki page:
Processing Results in the flext-target-ldap project encompass the mechanisms used to track, validate, and report the outcomes of LDAP directory operations. This system ensures that data synchronization from Singer-based sources to LDAP targets is observable, providing granular counters for successful operations, errors, and validation states.
The architecture relies on specialized result models that integrate with Singer sinks to provide real-time feedback during batch or record-level processing. This includes tracking the number of records processed, identifying specific error messages, and calculating success rates to determine the health of the integration.
Sources: src/flext_target_ldap/_models/processing_result.py:1-29, src/flext_target_ldap/_models/sinks.py:73-85
Result Tracking Models
The core of the reporting system is built on the FlextTargetLdapProcessingCounters and FlextTargetLdapProcessingResult classes. These components provide a mutable state for sinks to record the progress of LDAP operations.
Data Model Fields
The following fields are used to track the lifecycle of record processing:
| Field | Type | Description |
|---|---|---|
processed_count | int | Total number of records attempted in the current session. |
success_count | int | Number of records successfully written to the LDAP server. |
error_count | int | Number of records that failed processing or validation. |
errors | list[str] | A collection of specific error messages encountered during execution. |
Sources: src/flext_target_ldap/_models/processing_result.py:8-13, src/flext_target_ldap/_models/sinks.py:77-85
Processing Flow Logic
Sinks utilize these models to update state based on the outcome of _persist_entry or process_record calls.
The diagram above illustrates how the processing result counters are updated during the lifecycle of an LDAP operation.
Sources: src/flext_target_ldap/_models/sinks.py:165-197
Validation and Business Rules
Before a result is recorded as a success, entries must pass rigorous validation checks defined in the FlextTargetLdapModels and sink validation methods. These checks ensure that results reflect valid LDAP protocol requirements.
Key Validation Components
- DN Validation: Ensures the Distinguished Name is not empty and follows required formats (e.g., containing
=and,). - Object Class Compliance: Automatically ensures the
topobject class is present and validates required attributes for specific classes (e.g.,snandcnforperson). - Result Wrapping: Uses
p.Result[bool](viaflext-core) to wrap operation outcomes, allowing sinks to distinguish between technical failures and business rule violations.
Sources: src/flext_target_ldap/models.py:126-160, src/flext_target_ldap/_models/sinks.py:202-220
CLI and Integration Reporting
When running via the Command Line Interface (CLI), processing results are often reflected through logging and standard error streams. The run_cli method processes Singer messages (SCHEMA, RECORD, STATE) and maintains a seen_dns set to determine whether to perform an add_entry or modify_entry operation, which directly impacts the success/error counters.
CLI Execution Sequence
This sequence shows the interaction between the CLI execution loop and the processing result tracking mechanism.
Sources: src/flext_target_ldap/api.py:164-219, tests/unit/test_target.py:120-143
Summary of Sink Implementations
Different sink types (Users, Groups, Organizational Units) handle processing results by implementing specific logic for building DNs and attributes.
- FlextTargetLdapUsersSink: Tracks results for user creation, mapping fields like
emailstomailandphone_numberstotelephoneNumber. - FlextTargetLdapGroupsSink: Monitors group memberships, mapping
membersto thememberattribute in LDAP. - FlextTargetLdapBaseSink: Provides the foundational
_processing_resultobject used by all specialized sinks.
Sources: src/flext_target_ldap/_models/sinks.py:99-102, src/flext_target_ldap/_models/sinks.py:223-228, src/flext_target_ldap/_models/sinks.py:330-335
Conclusion
Processing results in flext-target-ldap provide a robust framework for monitoring LDAP synchronization. By leveraging mutable counter classes and strict validation models, the system ensures that every record attempted is accounted for, providing developers and operators with clear visibility into the success or failure of data loading tasks.