Object Storage Integration
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Object Storage Integration in Daytona provides a scalable and persistent solution for managing binary data, sandbox snapshots, and file artifacts. This system enables the platform to decouple storage from the compute runtime, ensuring that sandbox data remains available even after environments are stopped or deleted.
The primary implementation utilizes S3-compatible storage, specifically leveraging MinIO for local and containerized deployments. This integration is accessible through multiple SDKs (TypeScript, Python, Ruby, and Go) and is a core component of the Daytona infrastructure, supporting features like file uploads, downloads, and persistent volume snapshots.
Sources: docker/README.md:15-30, libs/sdk-typescript/README.md:113-125, apps/runner/pkg/storage/minio_client.go
Architecture and Components
The architecture relies on a provider-based model where the Daytona Runner interacts with object storage backends. In development environments, this is handled by a dedicated MinIO service.
Infrastructure Layout
The following diagram illustrates how the Object Storage service fits into the broader Daytona Docker Compose environment.
The integration is centralized within the Storage_Services cluster, providing S3-compatible endpoints for the Runner.
Sources: docker/README.md:15-25, apps/runner/pkg/storage/minio_client.go
Key Storage Components
| Component | Description |
|---|---|
| MinIO | S3-compatible object storage server used for data persistence. |
| MinIO Client | The Go-based client used by the Runner to communicate with the storage backend. |
| ObjectStorage SDK | Language-specific classes (e.g., Daytona::ObjectStorage in Ruby) that provide high-level abstractions. |
| Docker Volumes | Persistent storage backing the MinIO containers. |
Sources: docker/README.md:27-35, libs/sdk-ruby/scripts/generate-docs.rb:22, apps/runner/pkg/storage/minio_client.go
SDK Integration and File Operations
Daytona exposes object storage capabilities through its File System (FS) and Object Storage modules within the SDKs. This allows programmatic management of files within the isolated sandbox environments.
File Operation Workflow
The SDKs facilitate a flow where files can be moved between local environments and the sandbox's persistent storage.
This flow shows the mediation of the Sandbox between the SDK and the underlying Object Storage.
Sources: libs/sdk-typescript/README.md:113-125, examples/typescript/file-operations/index.ts:47-60
Supported Operations
Developers can perform the following operations using the SDKs:
- Upload: Transfer local files or memory buffers to the sandbox storage.
- Download: Retrieve files from the sandbox as buffers or save them locally.
- Batch Operations:
uploadFilesanddownloadFilesfor handling multiple items in a single request. - Metadata Retrieval: Get file information including size, modified time, and permissions.
Sources: examples/typescript/file-operations/index.ts:47-100, libs/sdk-typescript/README.md:113-125
Implementation Details
MinIO Client Configuration
In the runner, the storage is initialized using standard S3/MinIO credentials. In default local setups, the credentials are set to minioadmin for both access and secret keys.
// Example of the underlying storage client structure in the Runner
type MinioClient struct {
endpoint string
accessKeyID string
secretAccessKey string
useSSL bool
}
Sources: apps/runner/pkg/storage/minio_client.go, docker/README.md:35
SDK Model: FileInfo
Object metadata is standardized across the platform using a FileInfo model, which provides technical details about stored objects.
| Field | Type | Description |
|---|---|---|
name | String | The name of the file or directory. |
isDir | Boolean | Indicates if the object is a directory. |
size | Float/Int | Total size of the object in bytes. |
modTime | String | Timestamp of the last modification. |
Sources: libs/api-client-ruby/lib/daytona_api_client/models/file_info.rb:21-45
Conclusion
Object Storage Integration is a fundamental pillar of the Daytona architecture, providing the necessary persistence layer for AI-generated code and sandbox environments. By utilizing S3-compatible MinIO, Daytona ensures compatibility with industry standards while maintaining high-performance file operations across its multi-language SDKs.
Sources: README.md:45-55, docker/README.md:15-25