LDIF Parsing Engine
Relevant source files
The following files were used as context for generating this wiki page:
The LDIF Parsing Engine is a core component of the FLEXT-LDIF library designed to process LDAP Data Interchange Format (LDIF) content in compliance with RFC 2849. It facilitates the transformation of raw LDIF strings or files into structured domain models, specifically m.Ldif.Entry objects, while supporting various server-specific extensions such as Oracle Internet Directory (OID) and Oracle Unified Directory (OUD).
The engine employs a "Railway Oriented Programming" pattern, utilizing p.Result (often aliased as r) to handle successes and failures without conventional exception-driven control flow. This ensures robust error recovery and clear metadata tracking for round-trip processing, preserving minimal differences between original and parsed content.
Sources: src/README.md:52-56, examples/01_basic_usage.py:22-30, src/flext_ldif/_utilities/parser.py:168-175
Architectural Overview
The parsing engine is structured into several layers following Clean Architecture principles. The Infrastructure Layer handles low-level parsing logic, while the Application Layer orchestrates these operations through a public facade.
The diagram above illustrates the sequential transformation of raw input into a structured domain entity.
Sources: src/flext_ldif/_utilities/parser.py:157-175, src/flext_ldif/_utilities/parser.py:246-270
Core Processing Logic
Line Unfolding and Record Splitting
LDIF content often contains lines folded for readability (RFC 2849 §3). The engine first unfolds these lines by merging lines that start with a single space or tab into the previous line. Once unfolded, the content is split into discrete record blocks separated by blank lines.
Sources: src/flext_ldif/_utilities/parser.py:274-295, src/flext_ldif/_utilities/parser.py:228-244
Record Parsing and Attribute Decoding
Individual records are parsed to extract the Distinguished Name (DN), attributes, and optional control or change records. The engine supports various value types:
- Plain Values: Standard text attributes.
- Base64 Values: Identified by
::and decoded using the configured default encoding. - URL/File Values: Identified by
:<for external resource references.
| Value Spec | Indicator | Processing Logic |
|---|---|---|
| Plain | : | Direct string capture |
| Base64 | :: | base64.b64decode + UTF-8 decode |
| URL | :< | Captured as ValueOrigin.URL |
| File | :< file:// | Captured as ValueOrigin.FILE |
Sources: src/flext_ldif/_utilities/parser.py:118-137, src/flext_ldif/_utilities/parser.py:192-225
Change Record Handling
For LDIF files used in update operations (rather than data exports), the engine parses changetype directives. It supports add, delete, modify, moddn, and modrdn operations. For modify, it builds a sequence of ChangeOperation objects, capturing the specific action (e.g., replace, increment) and the associated values.
Sources: src/flext_ldif/_utilities/parser.py:197-226, src/flext_ldif/servers/_base/entry.py:348-380
Schema and Metadata Extraction
The engine includes specialized utilities for extracting schema definitions from attribute lines, particularly for attributeTypes and objectClasses.
Schema Component Extraction
The FlextLdifUtilitiesSchemaExtract class uses regex patterns to isolate OIDs, names, descriptions, and syntax constraints.
Metadata Management
The engine tracks "Minimal Differences" to ensure that when an entry is written back to LDIF, it can optionally preserve the exact original formatting (spacing, case, and comments). This is managed via FlextLdifUtilitiesMetadata, which injects ServerMetadata into the Entry model during parsing.
Sources: src/flext_ldif/_utilities/metadata.py:165-200, src/flext_ldif/_utilities/metadata.py:228-235
Error Recovery and Resilience
The parsing engine is designed to be resilient against malformed input. Rather than crashing, it reports failures through the Result object, allowing the caller to decide whether to abort or continue with other records.
| Scenario | Engine Response |
|---|---|
| Missing DN | Fails the specific entry with No DN found in entry |
| Malformed OID | Returns a failure result for schema extraction |
| Invalid Base64 | Preserves the raw string as the value for manual recovery |
| Orphaned Continuation | Handled gracefully via the unfolding logic |
Sources: tests/integration/test_error_recovery.py:31-45, src/flext_ldif/_utilities/parser.py:128-132, src/flext_ldif/_utilities/parser.py:220-226
Implementation Details
Key Classes and Methods
| Class | Method | Purpose |
|---|---|---|
FlextLdifUtilitiesParser | parse_ldif_record | The primary logic for converting line blocks into Entry models. |
FlextLdifUtilitiesParser | decode_value | Decodes individual value-specs (Plain, Base64, URL). |
FlextLdifServersBaseEntry | _parse_content | Subclass-facing method for implementing server-specific parsing. |
FlextLdifUtilitiesSchemaExtract | extract_schema_basic_fields | Extracts OID, Name, and Desc from schema strings. |
Sources: src/flext_ldif/_utilities/parser.py:157, src/flext_ldif/_utilities/parser.py:118, src/flext_ldif/servers/_base/entry.py:168, src/flext_ldif/_utilities/schema_extract.py:73
Snippet: Record Parsing Loop
The following logic illustrates how the engine iterates through raw unfolded lines to build an attribute mapping.
for raw_line in lines:
line = raw_line.rstrip()
if not line or line.startswith("#"):
continue # Skip empty or comments
key, _, remainder = line.partition(":")
value, value_origin, raw_value = decode_value(remainder)
if key.lower() == "dn":
dn = value
else:
attrs.setdefault(key, []).append(value)
Sources: src/flext_ldif/_utilities/parser.py:186-215
Conclusion
The LDIF Parsing Engine provides a strictly typed, RFC-compliant foundation for LDAP data management within the FLEXT ecosystem. By separating record splitting, line unfolding, and value decoding into discrete utility functions, the engine maintains high testability and allows for specialized server implementations to extend core functionality without duplicating parsing logic.