Extending the Writer

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_target_ldif/protocols.py
• src/flext_target_ldif/writer.py
• src/flext_target_ldif/models.py
• src/flext_target_ldif/utilities.py
• src/flext_target_ldif/target.py
• tests/unit/test_writer.py

Extending the Writer in flext-target-ldif involves customizing the logic used to transform Singer records into LDAP Data Interchange Format (LDIF) entries. The core component responsible for this is the FlextTargetLdifWriter, which leverages the flext-ldif infrastructure to ensure RFC 2849 compliance while providing hooks for custom mapping, Distinguished Name (DN) generation, and attribute formatting.

The extension surface primarily exists within the record-to-entry conversion pipeline, which handles data normalization, attribute mapping, and line wrapping. Developers can extend these capabilities by modifying the FlextTargetLdifWriter class or by utilizing the RecordTransformer and LdifDataProcessing utilities within the FlextTargetLdifUtilities namespace.

Sources: src/flext_target_ldif/writer.py:1-25, src/flext_target_ldif/utilities.py:1-25

Writer Architecture and Lifecycle

The FlextTargetLdifWriter operates as a buffered writer that collects records during the Singer STATE or RECORD messages and flushes them to a file upon closing. It supports context manager protocols (__enter__ and __exit__) to ensure proper resource handling and file closure.

The diagram shows the lifecycle of the writer from initialization through record buffering to the final conversion and file write operation during the closing phase.
Sources: src/flext_target_ldif/writer.py:34-118

Key Writer Components

Component	Description	Source
output_file	The destination path for the generated LDIF data.	writer.py:44
dn_template	A string template (e.g., uid={uid},ou=users,dc=com) used to generate the DN for each entry.	writer.py:46
attribute_mapping	A dictionary mapping Singer record keys to specific LDAP attribute names.	writer.py:47
line_length	Maximum length for LDIF lines before wrapping (default: 78).	writer.py:49-56
_ldif_api	Internal bridge to the flext-ldif project infrastructure.	writer.py:62

Record Transformation Logic

Extending the writer often requires custom transformation of record values (e.g., booleans to TRUE/FALSE, or formatting timestamps). The RecordTransformer class provides static methods for these standard LDAP conversions.

Transformation Pipeline

1. Field Mapping: Singer record keys are translated using attribute_mapping.
2. Normalization: Values are passed through specific transformers (Email, Phone, Timestamp, Boolean).
3. Required Attributes: Default attributes like objectclass are added if missing.

The flow illustrates how individual fields within a record are processed by the transformer utilities before being formatted for LDIF.
Sources: src/flext_target_ldif/utilities.py:214-325

Built-in Transformers

The writer supports several built-in transformers through FlextTargetLdifUtilities.TargetLdif.RecordTransformer.get_builtin_transformer.

Transformer	Target Attributes	Logic
transform_email	mail, email	Converts to lowercase and validates @ presence.
transform_phone	phone, mobile	Strips non-numeric characters except +- ().
transform_timestamp	createtimestamp, etc.	Converts Python datetime or ISO strings to LDAP format.
transform_boolean	*boolean, is*	Converts truthy values to TRUE and falsy to FALSE.

Sources: src/flext_target_ldif/utilities.py:257-285

LDIF Formatting and Encoding

The writer ensures that the output adheres to LDIF standards, specifically regarding line length and Base64 encoding for binary or non-ASCII data.

Base64 Encoding Rules

A value is automatically Base64 encoded if it meets any of the following criteria:

• It starts with a space ( ) or a colon (:).
• It contains non-ASCII characters.
• It contains newlines (\n) or carriage returns (\r).

# From src/flext_target_ldif/writer.py:165-173
def _needs_base64_encoding(self, value: str) -> bool:
    """Check if a value needs base64 encoding."""
    if value and value[0] in {" ", ":"}:
        return True
    try:
        value.encode("ascii")
    except UnicodeEncodeError:
        return True
    return "\n" in value or "\r" in value

Sources: src/flext_target_ldif/writer.py:165-173, tests/unit/test_writer.py:215-245

Line Folding

LDIF requires lines longer than a specified length (default 78) to be folded. The writer handles this by inserting a newline followed by a single space at the start of the continuation line.

# From src/flext_target_ldif/writer.py:230-244
def _write_line(self, line: str) -> None:
    """Write a line to the file handle, wrapping if necessary."""
    if len(line) <= self.line_length:
        self._file_handle.write(line + "\n")
    else:
        self._file_handle.write(line[: self.line_length] + "\n")
        remaining = line[self.line_length :]
        while remaining:
            chunk = remaining[: self.line_length - 1]
            self._file_handle.write(" " + chunk + "\n")
            remaining = remaining[self.line_length - 1 :]

Sources: src/flext_target_ldif/writer.py:230-244, src/flext_target_ldif/utilities.py:192-212

Data Models for Extension

The FlextTargetLdifModels class provides the structural definition for LDIF entries and files, which can be used to validate data before it reaches the writer.

Model	Purpose	Key Fields
LdifFormatOptions	Defines formatting constraints.	line_length, fold_lines, base64_encode
LdifEntry	Represents a single LDAP object.	distinguished_name, attributes, object_classes
LdifFile	Represents the output file metadata.	file_path, entry_count, is_compressed

Sources: src/flext_target_ldif/models.py:53-159

Validation Sequence

When extending the writer, the LdifEntry.validate_business_rules method can be used to ensure compliance:

1. DN Check: Validates that the DN contains attribute=value pairs.
2. Object Class Check: Ensures the entry has at least one object class.
3. Attribute Check: Verifies that the entry is not empty.

Sources: src/flext_target_ldif/models.py:96-125

Integration with Sinks

The Sink class in models.py acts as the orchestrator between the Singer target and the FlextTargetLdifWriter. It handles the initialization of the writer using target settings.

The sequence shows how the Sink manages the writer's lifecycle, ensuring it is only initialized when needed and closed properly at the end of the stream.
Sources: src/flext_target_ldif/models.py:171-267, src/flext_target_ldif/target.py:61-71

Conclusion

Extending the Writer in flext-target-ldif is achieved by utilizing the FlextTargetLdifWriter's configuration options and the FlextTargetLdifUtilities transformation methods. By modifying dn_template, attribute_mapping, and ldif_options, developers can control the structure and format of the LDAP output, while the underlying infrastructure handles standard-compliant tasks like line folding and Base64 encoding.