Exception Handling Hierarchy
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Meltano Singer SDK employs a structured exception handling hierarchy designed to manage data extraction (Taps), transformation (Mappers), and loading (Targets) failures. All SDK-specific exceptions are centralized to provide a consistent recovery strategy across the framework.
The primary goal of this hierarchy is to distinguish between different types of failures—such as configuration errors, API transient issues, or data mapping flaws—allowing the SDK to determine whether a process should be retried, skipped, or immediately aborted.
Sources: CLAUDE.md:46-47, cookiecutter/target-template/{{cookiecutter.target_id}}/AGENTS.md:126-129
Core Exception Architecture
All exceptions within the SDK inherit from a single root class, SingerSDKError. Developers are instructed to never define public exceptions in other files and must always inherit from established intermediate base classes rather than standard Python exceptions.
Exception Classification by Recovery Strategy
The hierarchy is organized based on how the system should respond to the error:
| Situation | Exception Class | Recovery Strategy |
|---|---|---|
| HTTP/API error | FatalAPIError | Abort the synchronization immediately. |
| HTTP/API error | RetriableAPIError | Safe to retry the operation. |
| Expected HTTP error | IgnorableAPIError | Skip silently and continue. |
| Invalid Config | ConfigValidationError | Fatal error requiring user intervention. |
| Discovery/Catalog | DiscoveryError | Issues with stream metadata or catalog. |
| Stream Mapping | MappingError | Expression or configuration failure in mappers. |
Sources: CLAUDE.md:48-57, CLAUDE.md:60-64
Class Hierarchy Diagram
The following diagram illustrates the relationship between the base error class and its specialized descendants.
Sources: CLAUDE.md:48-64
Implementation in Plugin Components
Taps and REST Streams
For Taps and RESTStream components, the SDK provides built-in retry logic. Errors encountered during API calls are categorized to facilitate automatic backoff.
- Retries: Managed using
python-backoff(version >= 2.2.2). - Pagination Errors: Errors during page token extraction should be handled by the SDK's paginator classes rather than custom implementations to ensure consistent error surfacing.
Sources: cookiecutter/tap-template/{{cookiecutter.tap_id}}/AGENTS.md:188-193, pyproject.toml:42
Targets and Sinks
In Target implementations, the sink class must handle both retryable and fatal errors during data loading.
try:
# Load data
result = self.load_data(records)
except RetryableError as e:
# SDK will retry
raise e
except FatalError as e:
# Log and fail the sync
self.logger.error(f"Fatal error: {e}")
raise e
Sources: cookiecutter/target-template/{{cookiecutter.target_id}}/AGENTS.md:126-140
Mappers and Stream Maps
Mappers use MappingError and its subclasses to handle failures in inline stream transformations, such as invalid Python-like expressions or schema mismatches during "in-flight" processing.
Sources: CLAUDE.md:54, cookiecutter/mapper-template/{{cookiecutter.mapper_id}}/AGENTS.md:21-22
Deprecation Warnings
The SDK also includes a specialized hierarchy for warnings to manage API lifecycle changes.
SingerSDKDeprecationWarning: Used when a specific removal version is known.SingerSDKPendingDeprecationWarning: Used when no removal timeline is committed; these are silenced by default.SingerSDKPythonEOLWarning: Used for Python versions nearing end-of-life.
Sources: CLAUDE.md:70-76
Conclusion
The Exception Handling Hierarchy in the Meltano Singer SDK provides a robust framework for error categorization. By utilizing specialized base classes like FatalSyncError and RetriableSyncError, the SDK ensures that connectors (Taps, Targets, and Mappers) behave predictably during failures, facilitating automated recovery where possible and clear diagnostic information for fatal configuration or data issues.