System Utilities
Relevant source files
The following files were used as context for generating this wiki page:
System Utilities in the flext-target-ldif project provide a unified framework for processing Singer records into LDAP Data Interchange Format (LDIF). This module encapsulates data transformation, DN (Distinguished Name) generation, line wrapping, and base64 encoding logic necessary for generating RFC-compliant LDIF files.
The utilities are structured as a unified class FlextTargetLdifUtilities that inherits from both flext_meltano and flext_ldif base utilities, ensuring consistency across the FLEXT ecosystem while providing domain-specific logic for LDAP operations.
Sources: src/flext_target_ldif/utilities.py:19-22, src/flext_target_ldif/models.py:33-40
LDIF Data Processing
The core of the system utilities lies in the LdifDataProcessing class, which handles the structural conversion of raw records into LDIF entries. This includes validating DN formats, formatting attribute values, and handling line wrapping for long strings.
Key Processing Functions
| Function | Purpose |
|---|---|
build_ldif_dn | Constructs a Distinguished Name using a template and record data. |
format_ldif_value | Determines if a value needs base64 encoding or wrapping. |
wrap_ldif_line | Implements RFC 2849 line folding for long attribute values. |
convert_record_to_ldif_entry | Orchestrates the conversion of a Singer record into a multi-line LDIF string. |
validate_ldif_entry | Ensures an LDIF string contains a valid DN and proper structure. |
Sources: src/flext_target_ldif/utilities.py:27-145
Data Transformation Flow
The following diagram illustrates how a Singer record is processed by the utilities before being written to the output file.
This flow ensures that every attribute is normalized, validly encoded, and structured according to LDIF standards.
Sources: src/flext_target_ldif/utilities.py:30-100, src/flext_target_ldif/writer.py:115-150
Record Transformation
The RecordTransformer utility provides specialized normalization logic for standard LDAP attributes. It ensures that data types from Singer streams (like booleans or timestamps) are converted into their LDAP-compatible string representations.
Normalization Logic
The transformer includes built-in handlers for various data types:
- Booleans: Converted to "TRUE" or "FALSE" strings.
- Timestamps: Transformed into ISO format or LDAP-compatible timestamp strings.
- Names: Formatted to ensure proper capitalization (e.g.,
givenName,sn). - Attributes: Maps Singer field names to specific LDAP attributes (e.g.,
emailtomail).
Sources: src/flext_target_ldif/utilities.py:155-210
Mandatory Attribute Injection
The utility includes an add_required_attributes method which ensures that records meet minimum LDAP schema requirements if they are missing. For example, if objectclass is missing, it defaults to ["inetOrgPerson", "person"]. If cn (Common Name) is missing, it attempts to derive it from uid, givenname, or displayname.
Sources: src/flext_target_ldif/utilities.py:241-260
Implementation in Sinks and Writers
While the utilities provide the logic, the FlextTargetLdifWriter and Sink classes use these utilities to perform the actual I/O operations.
The writer maintains a record buffer and leverages the utilities for each entry conversion before finalizing the file.
Sources: src/flext_target_ldif/writer.py:100-140, src/flext_target_ldif/models.py:210-250
Configuration and Defaults
The utilities rely on constants and settings defined within the FlextTargetLdifSettings and TargetLdif.LdifFormatOptions models.
| Option | Default Value | Description |
|---|---|---|
line_length | 78 | Maximum characters per line before folding. |
dn_template | uid={uid},ou=users,dc=example,dc=com | The structure used to generate the entry DN. |
base64_encode | False | Whether to force base64 encoding for all attributes. |
include_version | True | Includes the version: 1 header in the output. |
Sources: src/flext_target_ldif/models.py:53-85, src/flext_target_ldif/target.py:35-40
Summary
System Utilities in flext-target-ldif serve as the bridge between generic JSON record data and the rigid requirements of the LDIF format. By centralizing logic for DN generation, attribute normalization, and RFC 2849 compliance in the FlextTargetLdifUtilities class, the project ensures that any data stream can be reliably exported to LDAP-compatible directory files.