Sandbox & Execution Environments
Relevant source files
The following files were used as context for generating this wiki page:
The Sandbox subsystem in tldw_server provides a secure, isolated environment for executing code and managing transient workloads. It is designed as a broad internal control plane that abstracts various execution backends, including Docker, Firecracker, and Lima, to ensure that untrusted or resource-intensive tasks do not compromise the host system.
The environment is managed via a dedicated service layer that handles the entire lifecycle of a "run"—from enqueuing and admission control to execution, heartbeat monitoring, and artifact collection. This architecture prioritizes "fail-closed" behavior, ensuring that if a specific runtime is unavailable or a policy is unsupported, the system denies execution rather than falling back to a potentially less secure state.
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md, Docs/Plans/2026-03-08-sandbox-subsystem-review-design.md
Core Architecture and Components
The sandbox architecture is divided into a core control plane and an API layer. The control plane manages the internal invariants of execution, while the API layer exposes these capabilities to external clients.
Sandbox Control Plane
The control plane is organized into several key modules located under tldw_Server_API/app/core/Sandbox/:
- SandboxService: The primary entry point for managing the run lifecycle, including scaffolding, claim leasing, and cancellation.
- SandboxOrchestrator: Handles the queuing logic, claim semantics, and capacity management.
- SandboxStore: Provides persistence for run state across different backends (In-memory, SQLite, or Postgres).
- Policy Engine: Enforces network policies and runtime capabilities to ensure execution stays within defined boundaries.
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md, Docs/Plans/2026-03-08-sandbox-subsystem-review-design.md
The following diagram illustrates the high-level request flow through the Sandbox subsystem:
The Sandbox subsystem coordinates between the API, orchestration logic, and specialized runners to maintain isolated execution.
Execution Lifecycle and Claim Semantics
Execution in the sandbox follows a strict lifecycle managed by the SandboxOrchestrator. To prevent duplicate execution and ensure consistency in distributed or concurrent environments, the system uses "claim" and "lease" semantics.
Run Lifecycle Phases
- Enqueue: A run is added to the queue via
enqueue_run. - Admit: The orchestrator checks capacity via
try_admit_run_start. - Claim: A worker attempts to
try_claim_runto take ownership of the task. - Execute: The run starts under a claim lease (
_run_with_claim_lease). - Heartbeat: The worker must
renew_run_claimperiodically to maintain the lease. - Release/Prune: Claims are released upon completion or pruned via
_prune_queue_ttlif they expire.
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md
Capacity and Fencing
The system enforces concurrency caps to prevent resource exhaustion. If the queue is full, the API returns a 429 Too Many Requests status. Fencing semantics ensure that expired leases are correctly handled to avoid "zombie" executions or status corruption.
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-design.md, Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md
The sequence of enqueuing, claiming, and executing a task ensures strict ownership and resource management.
Runtime Runners and Policies
The sandbox supports multiple execution backends, each with specific "enforcer" logic to maintain isolation.
Supported Runtimes
| Runtime | Component | Purpose |
|---|---|---|
| Docker | docker_runner.py | Container-based isolation for standard workloads. |
| Firecracker | firecracker_runner.py | MicroVM-based isolation for high-security or multi-tenant tasks. |
| Lima | lima_runner.py | Linux virtual machines (typically on macOS) with lima_enforcer.py for strict admission. |
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md
Fail-Closed Behavior
A critical architectural invariant is the "fail-closed" policy. The system is designed to reject execution if:
- The requested runtime is unavailable (
runtime_unavailable). - The requested network policy is unsupported (
policy_unsupported). - The system is in "no-fallback" mode and the preferred backend fails.
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md, Docs/Plans/2026-03-08-sandbox-subsystem-review-design.md
Ownership and Artifact Safety
The sandbox enforces strict ownership boundaries for runs, sessions, and artifacts to prevent unauthorized data access.
Ownership Invariants
- Run Ownership: Only the creator of a run (or an admin) can view status or logs.
- Session Isolation: Workspaces, uploads, and snapshots are bounded to specific sessions.
- Artifact Guard: The
SandboxArtifactGuardRouteprevents path traversal attacks, ensuring files cannot escape the designated artifact root.
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md, Docs/Plans/2026-03-08-sandbox-subsystem-review-design.md
Key Data Structures and API Endpoints
| Component | Description |
|---|---|
SandboxArtifactGuardRoute | Middleware-style route guard for validating artifact path safety. |
_require_run_owner | Internal dependency for validating user ownership of a run. |
download_artifact | Endpoint for retrieving files generated during execution. |
stream_run_logs | Endpoint for real-time log streaming using WebSockets. |
Sources: Docs/Plans/2026-03-08-sandbox-subsystem-review-implementation-plan.md
Summary
The Sandbox & Execution Environments in tldw_server provide a robust framework for secure task execution. By combining a centralized orchestration layer with swappable runtime runners and strict "fail-closed" policies, the system ensures resource governance, data isolation, and operational safety. This infrastructure is essential for handling untrusted code and complex media processing tasks while maintaining host integrity.