Core Utilities

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_tap_ldap/_utilities/client_support.py
• src/flext_tap_ldap/_utilities/_processor.py
• src/flext_tap_ldap/_utilities/__init__.py
• tests/utilities.py
• src/flext_tap_ldap/ldif_streams.py
• src/flext_tap_ldap/streams.py

The Core Utilities within flext-tap-ldap provide a centralized set of tools for LDAP data normalization, LDIF file processing, and client-side support. These utilities are architected as mixins and nested namespaces (primarily within the u.TapLdap and u.Ldap namespaces) to ensure consistency across various LDAP extraction flows, including standard directory searches and LDIF-based stream processing.

The utility system bridges the gap between raw LDAP/LDIF data structures and the Singer-compatible formats required by the tap. It includes specific logic for handling Oracle-specific LDAP attributes, normalizing search scopes, and performing complex LDIF transformations such as schema mapping and attribute filtering.

Sources: src/flext_tap_ldap/_utilities/_processor.py, src/flext_tap_ldap/_utilities/client_support.py

LDIF Processing Engine

The LDIF processing engine is primarily housed in the FlextTapLdapUtilitiesProcessorMixin and delegates its heavy lifting to the flext-ldif library. It provides a testing-convenience wrapper around LDAP entries and a robust file/content processor.

Processor Components

Component	Responsibility
Processor	Parses LDIF files or strings into entry objects; provides filtering by DN, object class, or attribute existence.
Entry	A wrapper for m.Ldif.Entry that provides methods for attribute manipulation (add_attribute, update_attribute, remove_attribute) and validation.
Validator	Uses flext-ldif to verify entry integrity and returns detailed error reports.
Transformer	Applies transformation rules including attribute mappings, schema normalization, and attribute removals.

Sources: src/flext_tap_ldap/_utilities/_processor.py:15-325

LDIF Transformation Flow

This diagram illustrates how raw LDIF data is processed and transformed into a Singer-compatible format.

Client Support Utilities

The FlextTapLdapUtilitiesClientSupport class provides helpers for normalizing data returned by the LDAP client. This includes URI construction, scope normalization, and specific handling for Oracle Internet Directory (OID) modes.

Key Normalization Functions

• build_server_uri: Constructs canonical ldap:// or ldaps:// URIs based on host, port, and SSL status.
• normalize_scope: Ensures search scopes match LDAP standards (BASE, ONELEVEL, SUBTREE), defaulting to SUBTREE if invalid.
• to_entry_mapping: Converts p.Ldif.Entry objects or raw mappings into a standard JsonMapping, flattening single-value attributes and preserving multi-value attributes as lists.

Sources: src/flext_tap_ldap/_utilities/client_support.py:17-61

Oracle-Specific Logic

For Oracle environments, the utilities provide automated enrichment to maintain compatibility with standard LDAP expectations.

Test and Infrastructure Utilities

The project extends core utilities for testing purposes, specifically within tests/utilities.py. This includes helpers for managing Docker-based LDAP test environments.

• admin_credentials: A static method that resolves LDAP administrative credentials by checking environment variables (FLEXT_LDAP_BIND_DN, FLEXT_LDAP_BIND_PASSWORD) followed by known default candidates.
• FileLock: Inherited from FlextTestsUtilities to manage concurrent access to shared resources during integration tests.

Sources: tests/utilities.py:32-75

LDIF Analysis and Stream Support

Utilities support the LdifAnalysisStream by providing aggregation logic for LDIF content. This is used to generate summaries of LDAP datasets without full extraction.

Feature	Description
_classify_entry_type	Uses heuristics to categorize entries as user, group, ou, or other based on objectClass.
_as_counter_map	Validates and converts JSON values into integer-based mapping for statistical tracking.
_discover_ldif_files	Recursively searches directories for files matching a pattern (defaulting to *.ldif).

Sources: src/flext_tap_ldap/ldif_streams.py:27-145

Code Example: LDIF Transformation

The following snippet demonstrates how the Transformer utility is used to normalize LDAP entry attributes based on configuration.

# src/flext_tap_ldap/_utilities/_processor.py:348-358
def apply_attribute_mappings(
    self,
    entry: FlextTapLdapUtilitiesProcessorMixin.TapLdap.Entry,
    mappings: t.StrMapping,
) -> FlextTapLdapUtilitiesProcessorMixin.TapLdap.Entry:
    """Apply attribute name mappings to entry."""
    new_attributes: t.MutableStrSequenceMapping = {}
    for attr_name, values in entry.attributes.items():
        new_name = mappings.get(attr_name, attr_name)
        new_attributes[new_name] = list(values)
    # ... returns a new Entry object with mapped attributes

Sources: src/flext_tap_ldap/_utilities/_processor.py:348-364

The Core Utilities serve as the backbone for both live LDAP connectivity and offline LDIF file processing, ensuring that regardless of the data source, the output remains consistent with the Singer protocol and project-specific normalization rules.