Getting Started
Relevant source files
The following files were used as context for generating this wiki page:
The flext-tap-ldif project is a Singer Tap designed for extracting data from Lightweight Data Interchange Format (LDIF) files. It is built using the FLEXT ecosystem patterns, leveraging flext-core, flext-ldif, and flext-meltano to provide a typed, high-performance integration for enterprise ETL pipelines.
Sources: README.md:7-12, pyproject.toml:21-28, src/flext_tap_ldif/tap.py:1-12
Project Overview
The tap functions by discovering LDIF files from a local directory or a specific file path, parsing the entries, and emitting them as Singer-formatted records. It supports schema discovery for entries, including distinguished names (DN), object classes, and attributes.
Key Components
| Component | Description |
|---|---|
FlextTapLdif | The main Singer Tap class inheriting from SingerTapBase. |
Processor | Utility that handles file discovery and parsing via flext-ldif API. |
EntriesStream | The Singer stream implementation for ldif_entries. |
LdifFile | Data model representing file metadata and processing status. |
Sources: src/flext_tap_ldif/tap.py:18-20, src/flext_tap_ldif/_utilities/processor.py:22-25, src/flext_tap_ldif/_utilities/entries_stream.py:19-21, src/flext_tap_ldif/_models/file_metadata.py:13-15
Installation and Environment
The project requires Python 3.13+. Dependencies are managed via hatchling as the build backend.
[project]
name = "flext-tap-ldif"
requires-python = ">=3.13,<3.14"
dependencies = [
"flext-core",
"flext-ldif",
"flext-meltano",
"pydantic>=2.13.3",
]
Sources: pyproject.toml:7-9, pyproject.toml:21-35
Configuration Reference
The tap is configured via a JSON structure or environment variables (prefixed as defined in pyproject.toml).
| Option | Type | Default | Description |
|---|---|---|---|
file_path | string | - | Path to a specific LDIF file. |
directory_path | string | - | Directory to scan for LDIF files. |
file_pattern | string | *.ldif | Glob pattern for file discovery. |
encoding | string | utf-8 | File encoding for reading. |
strict_parsing | boolean | true | If true, fails on parsing errors. |
max_file_size_mb | integer | 100 | Maximum size of files to process. |
Sources: src/flext_tap_ldif/tap.py:23-40, src/flext_tap_ldif/_utilities/processor.py:27-33
Execution Flow
The following diagram illustrates the internal process from discovery to record emission.
The flow starts with the EntriesStream calling the Processor to find files based on configuration. For each file, the flext-ldif library parses the raw content into structured entries, which are then validated against Pydantic models before being yielded as records.
Sources: src/flext_tap_ldif/_utilities/entries_stream.py:32-80, src/flext_tap_ldif/_utilities/processor.py:75-100
Data Structure: LDIF Entries
Records emitted by the ldif_entries stream follow a specific schema designed to capture all relevant LDAP/LDIF metadata.
{
"dn": "distinguishedName",
"object_class": ["top", "person"],
"attributes": {
"cn": ["John Doe"],
"sn": ["Doe"]
},
"change_type": "add",
"source_file": "/path/to/data.ldif",
"line_number": 0,
"entry_size": 1234
}
Sources: src/flext_tap_ldif/tap.py:53-65, src/flext_tap_ldif/_utilities/processor.py:102-114
Development Guidelines
The project follows "Universal Agent Law" for engineering conduct, emphasizing zero-tolerance for dead code and a fix-forward-only policy.
Quality Gates
Development is governed by make verbs:
make check: Run linters and type checkers.make test: Execute the test suite (usespytest).make gen: Regenerate auto-generated files (e.g., lazy exports).
Sources: README.md:40-43, AGENTS.md:12-25, pyproject.toml:234-250
Summary
flext-tap-ldif provides a robust mechanism for integrating LDIF data into Singer-compatible pipelines. By utilizing the Processor utility and EntriesStream, it abstracts the complexities of LDIF parsing and file management into a standardized interface. Developers should focus on providing correct file_path or directory_path configurations and adhering to the project's strict typing and quality gate requirements.