Error Handling & Troubleshooting
Relevant source files
The following files were used as context for generating this wiki page:
- airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/output/ExceptionHandler.kt
- airbyte-cdk/java/airbyte-cdk/core/src/main/kotlin/io/airbyte/cdk/integrations/base/errors/messages/ErrorMessage.kt
- airbyte-integrations/developer-docs/destinations/preflight-checklist.md
- airbyte-integrations/developer-docs/destinations/step-by-step/0-introduction.md
- docusaurus/platform_versioned_docs/version-2.0/connector-development/ux-handbook.md
- docusaurus/platform_versioned_docs/version-2.0/contributing-to-airbyte/resources/qa-checks.md
Error handling in Airbyte is designed to ensure data integrity, security, and reliability across various connectors and platform components. The system prioritizes "failing fast" with actionable feedback to minimize debugging time for users, particularly during the initial configuration and sync phases. By classifying errors and providing specific troubleshooting documentation, the architecture aims for a 99.9% reliability rate in infrastructure operations.
Sources: docusaurus/platform_versioned_docs/version-2.0/connector-development/ux-handbook.md:46-77, airbyte-integrations/developer-docs/destinations/step-by-step/0-introduction.md:42-47
Architecture of Error Management
The error handling architecture is built around two primary concepts: automatic classification and actionable user communication. The system distinguishes between implementation details (which should be hidden) and configuration errors (which must be reported clearly).
Error Classification and Messaging
The ErrorMessage utility provides standardized methods for generating error messages that follow the project's UX guidelines. It ensures that technical details are masked while providing clear instructions for resolution.
The diagram shows the logic flow from an exception to the final message presented to the user or system logs.
Sources: airbyte-cdk/java/airbyte-cdk/core/src/main/kotlin/io/airbyte/cdk/integrations/base/errors/messages/ErrorMessage.kt
Database Client Error Handling
Destination connectors must implement specific logic to handle database-level errors. This involves monitoring SQL State codes and numeric error codes to identify specific failure modes like unique violations or permission issues.
| Error Category | Identification Method | Common Pattern |
|---|---|---|
| Unique Violation | SQL State 23505 | "duplicate key value" |
| Permission Denied | Message keywords / SQL State | "insufficient privileges" |
| Identifier Length | DB Specific (e.g., 63 chars) | "identifier too long" |
| Connectivity | SQLException / Timeout | "connection refused" |
Sources: airbyte-integrations/developer-docs/destinations/preflight-checklist.md:195-212, airbyte-integrations/developer-docs/destinations/preflight-checklist.md:239-242
Actionable Troubleshooting
Airbyte emphasizes "Fail Fast & Actionably." A user should not be able to successfully configure a connector with parameters that are guaranteed to fail later.
Validation Strategies
Connectors utilize JSON schema "pattern" keywords and manual validation during the --check phase to ensure inputs adhere to required formats. This prevents runtime errors related to malformed API keys, hostnames, or credentials.
The sequence illustrates the check operation failing early to provide feedback before a sync starts.
Sources: docusaurus/platform_versioned_docs/version-2.0/connector-development/ux-handbook.md:197-204, airbyte-integrations/developer-docs/destinations/step-by-step/0-introduction.md:112-115
Prerequisite Documentation
Troubleshooting often stems from missing assumptions in the source environment. Documentation must cover:
- Assumptions: Specific configurations like "Instagram Business account must be linked to Facebook."
- Setup Guides: Step-by-step instructions for prerequisite tasks like creating logical replication slots for CDC.
Sources: docusaurus/platform_versioned_docs/version-2.0/connector-development/ux-handbook.md:273-286, docusaurus/platform_versioned_docs/version-2.0/contributing-to-airbyte/resources/qa-checks.md:46-60
Common Troubleshooting Scenarios
Test Context Confusion
A frequent issue during development is the confusion between different test contexts (Component vs. Integration vs. Basic Functionality). This can lead to misleading error reports where a component passes locally but fails when wired into the full lifecycle.
Dependency Injection (DI) Errors
In Kotlin/Java-based connectors using Micronaut, "No bean found" errors are common when components are not properly registered in the TableCatalog or application context.
| Issue | Resolution Path |
|---|---|
| DI Registration | Verify @Singleton or manual registration in factory. |
| Schema Evolution | Compare discover vs compute results in SqlGenerator. |
| Permission Errors | Query information_schema.columns to verify introspection access. |
Sources: airbyte-integrations/developer-docs/destinations/step-by-step/0-introduction.md:215-225, airbyte-integrations/developer-docs/destinations/preflight-checklist.md:173-178
Summary
Error handling and troubleshooting in Airbyte serve as a bridge between complex infrastructure implementation and user-facing ease of use. By employing standardized messaging via ExceptionHandler and ErrorMessage, and enforcing rigorous pre-flight checks, the system ensures that infrastructure failures are caught early and described in terms that allow for rapid self-service resolution.