Auxiliary Tooling (Watchdogs)
Relevant source files
The following files were used as context for generating this wiki page:
Auxiliary tooling consists of specialized binaries that assist the core Infrahub backup and collection operations. These tools handle specific environmental constraints, such as performing database-level operations inside restricted containers or managing object storage uploads. The primary components include the Neo4j Watchdog and the S3 Uploader.
These tools are built as standalone Go binaries and are often embedded within the main application or container images to facilitate operations in both Docker Compose and Kubernetes environments.
Sources: AGENTS.md:120-131, specs/001-k8s-helm-backup/plan.md:58-62
Neo4j Watchdog
The Neo4j Watchdog is a specialized utility designed to manage Neo4j database instances during backup operations, particularly for the Community Edition. It ensures the database is in a consistent state and manages the lifecycle of the database process to allow for safe data extraction.
Architecture and Execution Flow
The watchdog operates by monitoring a target Neo4j process or performing administrative tasks required for backups that cannot be executed remotely via standard APIs. In Kubernetes deployments, this binary is embedded directly into the container to handle local pod execution.
The diagram above illustrates how the Backup Orchestrator utilizes the Neo4j Watchdog within a container to manage the database process and generate dumps.
Sources: tools/neo4jwatchdog/main.go:1-20, specs/001-k8s-helm-backup/plan.md:65-72, src/internal/app/watchdog_assets.go:1-15
Key Functions and Components
| Component | Description |
|---|---|
neo4j_watchdog | The compiled binary located in tools/neo4jwatchdog. |
watchdog_assets.go | Contains logic to embed the watchdog binary as an asset within the main application. |
| Community Edition Support | Required specifically for Neo4j Community Edition backups where standard online backup tools are unavailable. |
Sources: src/internal/app/watchdog_assets.go, specs/001-k8s-helm-backup/plan.md:270-272
S3 Uploader
The S3 Uploader is a utility responsible for transferring backup artifacts to S3-compatible storage. It abstracts the complexities of the AWS SDK to provide a streamlined interface for uploading large tar.gz archives generated by the backup process.
Storage Integration Logic
The uploader interacts with S3-compatible endpoints (such as AWS S3 or MinIO) using credentials provided through environment variables or Kubernetes Secrets.
The sequence diagram shows the communication flow between the main backup tool, the S3 Uploader, and the storage backend.
Sources: tools/s3-uploader/main.go:1-25, specs/001-k8s-helm-backup/plan.md:125-155
Configuration Parameters
The S3 Uploader utilizes several configuration flags and environment variables to define its destination and authentication.
| Parameter | Environment Variable | Description |
|---|---|---|
--s3-bucket | INFRAHUB_S3_BUCKET | The target S3 bucket name. |
--s3-region | INFRAHUB_S3_REGION | The AWS region for the bucket. |
--s3-endpoint | INFRAHUB_S3_ENDPOINT | Custom endpoint for MinIO or other S3-compatible providers. |
--s3-prefix | INFRAHUB_S3_PREFIX | Directory-like prefix within the bucket. |
Sources: specs/001-k8s-helm-backup/plan.md:38-42, specs/001-k8s-helm-backup/plan.md:162-178
Binary Distribution and Build Process
The auxiliary tools are managed within the project's multi-arch build system. They are compiled for both amd64 and arm64 architectures to ensure compatibility across diverse Kubernetes and Docker environments.
Multi-Arch Docker Implementation
The Dockerfile includes specific build stages for these auxiliary tools:
# Build neo4j watchdog binaries for target architecture
RUN CGO_ENABLED=0 GOOS=linux GOARCH=$TARGETARCH go build -ldflags "-s -w" \
-o /neo4j_watchdog ./tools/neo4jwatchdog
Example of the build command used in the Dockerfile to produce the watchdog binary.
Sources: specs/001-k8s-helm-backup/plan.md:58-65, specs/archive/003-collect-tool/opsmill-implement-report.md:20-25
Summary
Auxiliary tooling provides the necessary low-level hooks to interact with database processes and storage backends. By decoupling these functions into binaries like neo4j_watchdog and s3-uploader, the system maintains a clean separation of concerns while ensuring that backups can be performed safely in restricted containerized environments.
Sources: AGENTS.md:120-135, specs/001-k8s-helm-backup/plan.md:270-275