Prefect Task Maintenance
Relevant source files
The following files were used as context for generating this wiki page:
Prefect Task Maintenance manages the lifecycle and health of task execution records within the Infrahub platform. This system operates as part of the infrahub-taskmanager binary and focuses on cleaning up completed, failed, or stalled flow runs to maintain database performance and operational clarity.
The maintenance operations utilize a combination of Go-based orchestration and embedded Python scripts to interact directly with the Prefect API and its underlying PostgreSQL database.
Sources: AGENTS.md:38-40, AGENTS.md:104-108
System Architecture
The task maintenance architecture follows a split-binary, shared-core design. The infrahub-taskmanager entry point provides the CLI interface, while the core logic resides in the internal/app package.
The diagram shows the relationship between the CLI entry point, the shared application logic, the task-specific database, and the embedded scripts used for API interactions.
Sources: AGENTS.md:83-85, AGENTS.md:104-108
Key Maintenance Commands
The infrahub-taskmanager tool provides specific subcommands for different maintenance scenarios.
| Command | Action | Scope |
|---|---|---|
flush flow-runs | Removes historical flow runs | Completed, Failed, or Cancelled runs |
flush stale-runs | Cancels runs stuck in a running state | Active/Stalled runs |
environment detect | Identifies the deployment backend | Docker or Kubernetes |
version | Displays tool version info | Binary version |
Sources: src/cmd/infrahub-taskmanager/main.go:34-45, AGENTS.md:61-66
Cleanup Logic and Operations
Flow Run Deletion
The system cleans up old flow runs to prevent the task manager database from growing indefinitely. This process identifies tasks that have reached a terminal state (Completed, Failed, or Cancelled) and removes them based on age or status.
The maintenance logic utilizes an embedded Python script, clean_old_tasks.py, which executes the following steps:
- Connects to the Prefect API.
- Queries for flow runs based on state and age filters.
- Deletes identified records via the Prefect client.
Sources: src/internal/app/scripts/clean_old_tasks.py:1-25, src/internal/app/taskmanager.go:104-108
Stale Run Cancellation
Stale runs are active tasks that have stopped making progress but remain in a "Running" state. The maintenance tool identifies these runs and transitions them to a "Cancelled" state to free up worker resources and provide accurate status reporting.
This sequence diagram represents the workflow for cancelling stale runs through the embedded script bridge.
Sources: src/internal/app/taskmanager.go:104-108, src/cmd/infrahub-taskmanager/main.go:34-45
Database Management
The task manager relies on a PostgreSQL database, referred to as task-manager-db. The Go application logic handles the connection management to this database to perform maintenance tasks that cannot be handled via the API alone.
Components
- PostgreSQL: Stores the task manager metadata and run history.
- InfrahubOps Struct: The main application controller that manages database connection strings and execution contexts.
- CommandExecutor: Handles the execution of commands within the appropriate environment (e.g., executing scripts inside Docker containers).
Sources: AGENTS.md:104-108, AGENTS.md:154-155, src/internal/app/taskmanager.go:20-40
Implementation Details
Embedded Script Execution
Infrahub-taskmanager uses Go's embed package to include Python scripts directly in the binary. This ensures that maintenance logic is versioned with the tool and does not require external script files on the host system.
// Example of script usage (conceptual)
// Sources: src/internal/app/taskmanager.go:104-108
func (i *InfrahubOps) FlushFlowRuns() error {
// Logic to execute embedded scripts/clean_old_tasks.py
return nil
}
Sources: AGENTS.md:118-120, src/internal/app/taskmanager.go:104-108
Summary
Prefect Task Maintenance is a specialized component of the Infrahub toolset that ensures the stability of the task execution environment. By automating the cleanup of flow runs and the cancellation of stale tasks, it prevents database bloat and ensures that the Prefect orchestration layer remains responsive and accurate.