Data Transformation
Relevant source files
The following files were used as context for generating this wiki page:
Data Transformation in the flext-target-ldap project encompasses the logic and configurations used to convert incoming Singer protocol records into valid LDAP entries. This process involves mapping source fields to LDAP attributes, applying regex-based pattern replacements, and normalizing values to meet directory schema requirements.
The system ensures that data moving from a Singer source is properly formatted for LDAP operations such as adding or modifying entries. This includes handling attribute name conversions, applying default values, and executing specific transformation rules like case normalization.
Transformation Architecture
The transformation logic is embedded within the domain models and sink components. The FlextTargetLdapModels class defines the structural rules for transformation, while sink subclasses like FlextTargetLdapUsersSink and FlextTargetLdapGroupsSink implement the field-level mapping.
Logic Flow
The transformation process follows a structured flow from record ingestion to the preparation of LDAP attributes.
The diagram above illustrates the sequence of operations applied to a Singer record to produce LDAP-compatible attributes.
Sources: src/flext_target_ldap/models.py:34-118, src/flext_target_ldap/_models/sinks.py:228-250
Attribute Mapping Configuration
The AttributeMapping class defines how specific Singer fields are translated into LDAP attributes. It includes metadata for requirement checks, default values, and the association of specific transformation rules.
Mapping Schema
| Field | Type | Description |
|---|---|---|
singer_field_name | String | The name of the field in the source Singer data. |
ldap_attribute_name | String | The corresponding attribute name in the target LDAP directory. |
is_required | Boolean | If true, the transformation fails if the field is missing. |
transformation_rule | String | Reference to a rule (e.g., 'lowercase', 'trim'). |
default_value | String | Value applied if the source field is null. |
Sources: src/flext_target_ldap/models.py:44-77
Transformation Rules
Transformation rules provide granular control over data values using regex patterns. The TransformationRule model allows developers to define find-and-replace logic that is applied during the synchronization process.
Rule Structure
Rules are defined with the following properties:
- Pattern: A regex string used to match content within the source data.
- Replacement: The string to insert in place of the matched pattern.
- Enabled: A toggle to activate or deactivate the rule without deleting it.
Sources: src/flext_target_ldap/models.py:202-230, tests/unit/test_transformation.py:16-25
Transformation Validation
The system validates transformation rules to ensure they adhere to supported operations. Valid operations currently include:
lowercaseuppercasetrimnormalize
Sources: src/flext_target_ldap/models.py:99-105
Sink-Specific Transformations
Different LDAP object types require specific transformation logic, which is handled by specialized sink classes.
User Transformations
The FlextTargetLdapUsersSink uses a predefined internal map to convert standard Singer user fields into standard LDAP attributes.
_USER_FIELD_MAP: ClassVar[t.StrMapping] = {
"emails": "mail",
"phone_numbers": "telephoneNumber",
}
Sources: src/flext_target_ldap/_models/sinks.py:218-221
Group Transformations
The FlextTargetLdapGroupsSink handles the mapping of membership arrays. It specifically transforms source members fields into the LDAP member attribute, which typically requires Distinguished Names (DNs).
Sources: src/flext_target_ldap/_models/sinks.py:330-340
Value Normalization
Before data is sent to the LDAP client, values are normalized into a format compatible with the ldap3 library. This is primarily handled by the FlextTargetLdapClient.to_str_values utility, which ensures that all values—whether single strings or lists—are represented as a sequence of strings.
The diagram shows the relationship between field mappings and the rules applied to them.
Sources: src/flext_target_ldap/models.py:34-230, src/flext_target_ldap/_models/sinks.py:228-233
Summary
Data Transformation in flext-target-ldap is a multi-stage process that ensures data integrity between Singer sources and LDAP targets. By combining declarative attribute mapping with regex-based transformation rules and specialized sink logic, the system provides a robust framework for normalizing user, group, and organizational unit data for directory services.