Singer Target Execution

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_target_ldif/target.py
• src/flext_target_ldif/models.py
• src/flext_target_ldif/writer.py
• src/flext_target_ldif/utilities.py
• src/flext_target_ldif/api.py
• src/flext_target_ldif/_utilities/service_runtime.py

Singer Target Execution in the flext-target-ldif project refers to the process by which structured data records from a Singer Tap are consumed, processed, and exported into the LDAP Data Interchange Format (LDIF). The system operates as a specialized Singer Target that maps incoming stream records to LDIF entries, adhering to LDAP Distinguished Name (DN) structures and attribute requirements.

The execution flow involves a facade-based architecture that leverages the FLEXT ecosystem (Meltano and Core modules) to handle Singer protocol compliance while delegating format-specific logic to internal writers and sinks.

Sources: src/flext_target_ldif/target.py:1-15, src/flext_target_ldif/api.py:1-25

Architecture and Component Flow

The execution architecture follows a layered approach where a service orchestrator (FlextTargetLdifService) manages the lifecycle of sinks, and each sink utilizes a specialized writer to produce LDIF files.

Execution Component Overview

Component	Responsibility
FlextTargetLdif	Main entry point that manages configuration and sink lifecycle.
FlextTargetLdifService	Orchestrator providing the thin facade for service execution.
FlextTargetLdifModels.TargetLdif.Sink	Handles batch and record processing for specific Singer streams.
FlextTargetLdifWriter	Buffers records and performs the final write to the file system.
FlextTargetLdifUtilities	Provides logic for building DNs and formatting LDIF attributes.

Sources: src/flext_target_ldif/target.py:20-75, src/flext_target_ldif/models.py:168-245, src/flext_target_ldif/writer.py:31-75

Data Flow Diagram

The following diagram illustrates the flow from a Singer record to the final LDIF output.

The flow ensures that records are validated and transformed into LDAP-compatible formats before being buffered for file I/O.
Sources: src/flext_target_ldif/models.py:206-225, src/flext_target_ldif/writer.py:100-115

Configuration and Validation

Before execution, the target validates the environment and settings. The configuration includes LDAP-specific parameters such as DN templates and attribute mappings.

Core Configuration Settings

Option	Type	Default Value	Description
output_path	str	./output	Directory where LDIF files are saved.
dn_template	str	uid={uid},ou=users,dc=example,dc=com	Template for generating the Distinguished Name.
file_naming_pattern	str	{stream_name}_{timestamp}.ldif	Pattern for output filenames.
line_length	int	78	Max characters per line (RFC 2849 compliance).
base64_encode	bool	False	Forces base64 encoding for attributes.

Sources: src/flext_target_ldif/target.py:33-40, src/flext_target_ldif/writer.py:46-55

Validation Logic

The validate_config method ensures required fields like output_file (in test contexts) or output_path are present and that domain rules (e.g., non-empty DN templates) are satisfied.

def validate_config(self) -> None:
    # ...
    allowed_fields: set[str] = {
        "output_file", "output_path", "file_naming_pattern",
        "dn_template", "attribute_mapping", "ldif_options"
    }
    # ...
    settings = FlextTargetLdifSettings.model_validate(filtered_config)
    settings.validate_domain_rules()

Sources: src/flext_target_ldif/target.py:80-108

Sink and Record Processing

The execution core lies within the Sink.process_record method. Each record is passed through a transformation layer that maps fields to LDAP attributes and generates a unique DN.

Sequence of Record Processing

The sequence below shows how a sink coordinates with the writer and utilities during a record intake.

Attribute Transformation

The RecordTransformer (part of FlextTargetLdifUtilities) handles domain-specific formatting:

• Booleans: Converted to "TRUE" or "FALSE".
• Timestamps: Normalized to ISO formats.
• DN Generation: Placeholders in the dn_template (e.g., {uid}) are replaced with values from the record.

Sources: src/flext_target_ldif/utilities.py:136-175

LDIF Generation and Compliance

The FlextTargetLdifWriter is responsible for the final serialization. It ensures compliance with the LDIF standard, specifically regarding line wrapping and encoding.

Serialization Logic

• Base64 Encoding: Automatically triggered for values starting with spaces, colons, or containing non-ASCII/newline characters.
• Line Wrapping: Lines exceeding the configured line_length (standard 78) are folded using a single space indentation for continuation lines.
• Version Header: Files begin with version: 1 as required by the specification.

Sources: src/flext_target_ldif/writer.py:139-165, src/flext_target_ldif/writer.py:192-210

Writer Class Structure

Summary of Execution Lifecycle

1. Initialization: FlextTargetLdif loads settings and ensures the output_path directory exists.
2. Sink Creation: For each stream in the Singer catalog, a TargetLdif.Sink is instantiated.
3. Processing: process_record is called repeatedly. Records are buffered in the FlextTargetLdifWriter.
4. Transformation: The RecordTransformer normalizes values (e.g., phone numbers, emails) and maps them to target LDAP attributes.
5. Finalization: When the Singer stream ends, clean_up() calls writer.close(), which flushes the buffer to the file system with proper LDIF headers, DNs, and folded attributes.

Sources: src/flext_target_ldif/target.py:65-75, src/flext_target_ldif/models.py:202-212, src/flext_target_ldif/writer.py:77-98