Error Handling & Recovery
Relevant source files
The following files were used as context for generating this wiki page:
Error Handling & Recovery in the flext-tap-ldap project is built upon a structured approach that categorizes failures into connection, authentication, and operation-level errors. The system utilizes a specialized utility namespace to provide enhanced context for LDAP-specific failures, ensuring that issues occurring during Bind, Search, or Connection phases carry descriptive metadata such as host, port, and Distinguished Names (DNs).
The recovery logic is integrated into the core processing loop, specifically within the LDIF processor and LDAP stream handlers. It allows for configurable error tolerance—such as ignoring individual malformed LDIF entries—while maintaining strict validation for critical configuration paths. This ensures the tap can survive minor data inconsistencies without failing the entire extraction process.
Contextual Error Creation
The FlextTapLdapUtilitiesErrorHandling class serves as a factory for creating specialized exceptions with attached metadata. This allows the system to capture stateful information at the point of failure.
Key Factory Methods
| Method | Exception Type | Context Captured |
|---|---|---|
create_bind_error | AuthenticationError | bind_dn, plus arbitrary kwargs |
create_connection_error | ConnectionError | host, port, base_dn |
create_search_error | OperationError | base_dn, filter (truncated to 100 chars) |
Sources: src/flext_tap_ldap/_utilities/error_handling.py:10-53
The diagram above shows the flow of error categorization through the utility factory methods.
Sources: src/flext_tap_ldap/_utilities/error_handling.py:10-53
Processor Error Tolerance & Validation
The LDIF Processor manages failures during file or string parsing. It supports an ignore_errors mode, which allows the system to log failures to an internal errors sequence and increment skipped_entries instead of terminating.
Error Handling Policies in Parsing
- Ignore Errors Mode: If
ignore_errorsisTrue, the processor logs the error message and continues to the next entry. This is common when processing large LDIF files where a single malformed entry should not block the entire stream. - Strict Mode: If
ignore_errorsisFalse, the system raises aValueErrorimmediately upon encountering invalid LDIF content or parsing failures. - Max Errors: The processor tracks error counts against a
max_errorslimit (default 100) to prevent infinite failure loops.
Sources: src/flext_tap_ldap/_utilities/_processor.py:155-165, src/flext_tap_ldap/_utilities/_processor.py:270-285
Validation Logic
The Validator class uses the flext-ldif library to validate entries. It captures both errors (which invalidate the entry) and warnings (which provide feedback without failure).
Sources: src/flext_tap_ldap/_utilities/_processor.py:348-385
Stream Resilience and Connection Recovery
LDAP streams implement defensive programming to handle connection failures and missing schema information.
LDAP Client Initialization Recovery
During stream initialization, the LDAPBaseStream attempts to create an LDAP client. If initialization fails due to configuration issues (e.g., missing host), it logs a warning and sets the client to None rather than crashing, allowing discovery to report status rather than crashing the process.
Sources: src/flext_tap_ldap/streams.py:145-165
Schema Search Fallback
The SchemaStream employs a fallback mechanism when searching for LDAP schema information. It iterates through multiple potential Base DNs until a successful result is found.
The sequence shows the recursive fallback strategy for locating the LDAP schema across various common DN patterns.
Sources: src/flext_tap_ldap/streams.py:465-495
Tap Execution Safety
The FlextTapLdapTap class wraps its execution logic in Result objects, ensuring that failure statuses are propagated correctly to the Singer orchestrator without unhandled stack traces.
Stream Discovery Validation
During discovery, the tap validates custom stream definitions. Invalid definitions are logged as warnings and skipped, ensuring the tap remains functional for standard streams.
Sources: src/flext_tap_ldap/tap.py:180-195, src/flext_tap_ldap/tap.py:240-250
Connection Health Checks
The LDAPClient provides a health_check method that measures response time and validates connectivity. This is used to report an "unhealthy" status if the test_connection call fails or times out.
Sources: src/flext_tap_ldap/client.py:265-275
Conclusion
The Error Handling & Recovery architecture of flext-tap-ldap prioritizes continuity and visibility. By decoupling error creation through specialized utilities and implementing "ignore-on-failure" patterns in bulk data processors, the system provides a resilient bridge between LDAP directory services and Singer-based data pipelines. Errors are transformed into actionable metadata, while recovery strategies like DN-fallback and optional validation allow the tap to adapt to various directory environments.