Implementing Custom Transformers
Relevant source files
The following files were used as context for generating this wiki page:
Custom Transformers in the FLEXT-LDIF library provide a structured way to manipulate, normalize, and migrate LDIF data between different LDAP server implementations. They function as modular units within the infrastructure layer to handle specific data transformation tasks, such as DN normalization, attribute case correction, or server-specific schema adjustments.
These transformers are integrated into a Railway Oriented Programming (ROP) pipeline, allowing for early failure detection and clear audit trails of every modification made to an LDIF entry. This architecture ensures high-fidelity data migration while maintaining RFC compliance.
Sources: src/flext_ldif/_utilities/_transformer_base.py, examples/01_basic_usage.py:20-30, src/README.md
Transformer Architecture
The transformation logic is organized into specialized classes that inherit from base transformer utilities. The system distinguishes between structural transformations (DNs) and content transformations (Attributes).
Base Transformer Components
The FlextLdifUtilitiesTransformer serves as the foundational utility for all transformation logic. It works in conjunction with specialized transformers for attributes and DNs.
The diagram above illustrates the inheritance relationship between the base transformer utility and specialized normalization transformers.
Sources: src/flext_ldif/_utilities/init.py:12-20, src/flext_ldif/_utilities/_transformer_base.py
Transformation Pipeline
Transformers operate within a pipeline where each stage performs a specific task. Typical stages include DN normalization, attribute normalization, and server-specific transformations.
| Stage Name | Constant Reference | Description |
|---|---|---|
| Normalize DN | PROCESSING_STAGE_NORMALIZE_DN | Standardizes DN syntax and case. |
| Normalize Attrs | PROCESSING_STAGE_NORMALIZE_ATTRS | Standardizes attribute names and values. |
| Server Transform | PROCESSING_STAGE_SERVER_TRANSFORM | Applies specific logic for OID, OUD, or AD. |
Sources: src/flext_ldif/constants.py:230-233, examples/01_basic_usage.py:120-135
Core Transformation Logic
Implementing a custom transformer involves defining logic to handle attribute name normalization, value adjustments, and schema compliance.
Attribute Normalization
Normalization often involves removing server-specific suffixes (like ;binary) or replacing characters to ensure compatibility with standard LDAP schemas.
# Example of normalization logic found in tests
result = u.Ldif.normalize_name(
"userCertificate;binary",
suffixes_to_remove=[";binary"]
)
# Result: "userCertificate"
Sources: tests/unit/servers/test_schema_transformer.py:17-22
Schema Transformation
Transformers are used to fix discrepancies in schema definitions, such as incorrect matching rules. For example, a transformer might move a substring rule incorrectly placed in an equality field to the correct location.
The flow above describes how transformers resolve schema matching rule errors.
Sources: tests/unit/servers/test_schema_transformer.py:53-62
Metadata and Audit Trails
A critical feature of FLEXT-LDIF transformers is the preservation of transformation history. Every change is recorded in the ServerMetadata model, which includes a conversion_history and attribute_transformations map.
Metadata Fields for Transformers
Transformers interact with these fields to provide transparency:
- conversion_history: A sequence of mappings recording every step, timestamp, and operation performed (e.g.,
parse,normalize,migrate). - attribute_transformations: A detailed audit trail of per-attribute changes.
- minimal_differences: Tracks exact changes in strings to ensure zero data loss during round-trips.
Sources: src/flext_ldif/_models/domain_metadata.py:180-230
Tracking Conversions
Transformers use helper methods to log their actions within the metadata object:
metadata.add_conversion_note(
operation="oid_to_rfc",
description="Converted OID ACL format to RFC 4515 filter"
)
Sources: src/flext_ldif/_models/domain_metadata.py:270-280
Integration with Railway Oriented Programming
Custom transformers are executed within Result (represented as r or p.Result) containers. This prevents a single transformation failure from crashing the entire batch process while still reporting the error clearly.
The sequence above shows the interaction between the transformation pipeline and the resulting metadata updates.
Sources: examples/01_basic_usage.py:130-150, src/flext_ldif/_models/domain_metadata.py:235-245
Conclusion
Implementing custom transformers in FLEXT-LDIF requires extending the base transformer utilities and adhering to the metadata-rich architecture. By leveraging the built-in normalization patterns and integrating with the audit trail system, developers can create robust migration tools that maintain strict RFC compliance while handling complex server-specific LDAP extensions.