Examples & Usage Patterns
Relevant source files
The following files were used as context for generating this wiki page:
This page provides a comprehensive overview of common usage patterns, configuration examples, and architectural flows for the Flext Tap LDAP project. It covers standard Singer tap operations, specialized LDIF processing, and Oracle-specific enrichment patterns.
CLI Execution Patterns
The tap is primarily interacted with via a Singer-compatible CLI. It supports standard modes such as discovery, full sync, and incremental sync using state files.
Common CLI Commands
| Command Pattern | Description |
|---|---|
tap-ldap --config config.json --discover | Discovers available streams and outputs a Singer catalog. |
tap-ldap --config config.json --catalog catalog.json | Executes a sync for the streams selected in the catalog. |
tap-ldap --config config.json --catalog catalog.json --state state.json | Executes an incremental sync using state bookmarks. |
Sources: src/flext_tap_ldap/tap.py:228-245, tests/e2e/test_integration.py:84-142
Discovery and Sync Flow
The following diagram illustrates how the CLI processes configuration to either discover metadata or stream records.
Configuration Usage
The tap requires a JSON configuration file defining connection parameters and optional processing rules.
{
"host": "ldap.example.com",
"port": 389,
"bind_dn": "cn=admin,dc=example,dc=com",
"password": "secret_password",
"base_dn": "dc=example,dc=com",
"use_ssl": false,
"page_size": 1000,
"enable_ldif_streams": true
}
Sources: src/flext_tap_ldap/tap.py:65-115
Custom Stream Definitions
Users can define custom streams by providing a custom_streams array in the configuration. This allows for specific LDAP filters and schema definitions beyond the standard users and groups streams.
Sources: tests/e2e/test_integration.py:173-195, src/flext_tap_ldap/tap.py:263-285
LDIF Processing Patterns
The project includes specialized utilities for processing LDIF (LDAP Data Interchange Format) files, either as primary data sources or for transformation tasks.
LDIF Transformation Rules
The Transformer class supports complex attribute mappings, removals, and value translations.
transformation_rules = {
"attribute_mappings": {"CN": "cn", "sn": "surname"},
"attribute_value_mappings": {
"department": {"IT": "Information Technology"}
},
"remove_attributes": ["obsolete"],
"add_attributes": {"status": "active"}
}
Sources: src/flext_tap_ldap/_utilities/_processor.py:441-498, tests/unit/test_ldif_processor.py:53-73
LDIF Stream Architecture
When enable_ldif_streams is active, the tap can ingest data directly from flat LDIF files or entire directories.
Oracle OID Enrichment Pattern
For environments using Oracle Internet Directory (OID), the tap provides a specific "Oracle OID mode" that normalizes non-standard attributes into canonical LDAP forms.
Enrichment logic
When oracle_oid_mode is enabled:
- Attribute Extension: The requested attribute list is automatically extended to include
orclPasswordandorclPasswordAttribute. - Attribute Mapping:
orclPasswordvalues are mapped to the standarduserPasswordfield. - ObjectClass Normalization: If an entry has the
orclContainerclass but lacksorganizationalUnit, the latter is added for better compatibility with standard LDAP consumers.
Sources: src/flext_tap_ldap/_utilities/client_support.py:73-125
Data Flow for Oracle Results
The following sequence shows the enrichment process for Oracle search results.
Validation and Error Handling
The tap employs a strict validation layer for LDIF entries and configuration, allowing for configurable error tolerance.
| Component | Responsibility | Failure Behavior |
|---|---|---|
Validator | Uses flext-ldif to check entries against LDAP schemas. | Returns detailed error list including code and message. |
Processor | Handles file/string parsing. | ignore_errors=True (default) logs and continues; False raises ValueError. |
FlextTapLdapTap | Validates custom stream definitions. | Logs warnings and ignores invalid stream items. |
Sources: src/flext_tap_ldap/_utilities/_processor.py:382-410, src/flext_tap_ldap/_utilities/_processor.py:145-165, src/flext_tap_ldap/tap.py:210-218
Summary
The Flext Tap LDAP project provides a robust framework for extracting directory data via the Singer protocol. By combining standard LDAP client operations with advanced LDIF transformation and Oracle-specific normalization, it accommodates both modern directory services and legacy enterprise environments. Key usage patterns revolve around standard CLI execution, flexible LDIF directory ingestion, and automated attribute enrichment for specialized directory types.