Binary Data Management
Relevant source files
The following files were used as context for generating this wiki page:
- packages/core/src/binary-data/binary-data.service.ts
- packages/core/src/binary-data/file-system.manager.ts
- packages/core/src/binary-data/object-store.manager.ts
- packages/core/src/binary-data/object-store/object-store.service.ee.ts
- packages/core/src/execution-engine/node-execution-context/utils/binary-helper-functions.ts
- packages/frontend/editor-ui/src/components/RunDataBinary.vue
- packages/core/src/execution-engine/node-execution-context/utils/tests/binary-helper-functions.test.ts
The Binary Data Management system in n8n provides a unified framework for handling binary files within workflows. Its primary purpose is to abstract the storage and retrieval of binary data, allowing nodes to work with files without needing to know the underlying storage mechanism. This system is designed to be flexible, supporting multiple storage backends such as in-memory, local filesystem, and S3-compatible object stores.
The central component is the BinaryDataService, which acts as a facade for all binary data operations. It delegates tasks to a configured storage manager, which implements the BinaryData.Manager interface. This architecture allows for easy extension with new storage backends and provides a consistent API for nodes through a set of Binary Helper Functions.
Architecture
The system is built around the BinaryDataService and a set of pluggable Manager classes that handle the specifics of each storage backend.
The following diagram illustrates the relationship between the key components of the binary data management system.
Core Components
| Component | Description | Source File |
|---|---|---|
BinaryDataService | The main service and entry point for all binary data operations. It selects the appropriate manager based on the configured mode. | packages/core/src/binary-data/binary-data.service.ts |
BinaryData.Manager | An interface that defines the contract for storage backends. Implementations handle the actual reading and writing of data. | packages/core/src/binary-data/types.ts |
FileSystemManager | An implementation of BinaryData.Manager that stores files on the local filesystem. | packages/core/src/binary-data/file-system.manager.ts |
ObjectStoreManager | An implementation of BinaryData.Manager for S3-compatible object storage. It relies on ObjectStoreService. | packages/core/src/binary-data/object-store.manager.ts |
ObjectStoreService | An enterprise service (.ee.ts) that handles the low-level interaction with S3-compatible APIs. | packages/core/src/binary-data/object-store/object-store.service.ee.ts |
BinaryHelperFunctions | A set of utility functions provided to nodes for easy interaction with the BinaryDataService. | packages/core/src/execution-engine/node-execution-context/utils/binary-helper-functions.ts |
Storage Modes
The system supports several storage modes, configured via the BinaryDataConfig.
| Mode | Description |
|---|---|
default | Data is stored in-memory. The binary content is Base64-encoded and placed directly in the data property of the IBinaryData object. This is not suitable for large files or persistent storage. |
filesystem / filesystem-v2 | Data is stored on the local filesystem. The data property of IBinaryData holds the manager name (filesystem-v2), and the id property contains a unique file identifier. |
s3 | Data is stored in an S3-compatible object store. The data property holds the manager name (s3), and the id property contains the object key. |
Sources: packages/core/src/binary-data/binary-data.service.ts:20-39
Data Flow
Storing Binary Data
The process of storing a binary file involves a node calling a helper function, which then delegates to the BinaryDataService and the configured manager.
This sequence diagram shows the flow for storing binary data.
Retrieving Binary Data
Retrieving data follows a similar pattern, where a request for a buffer or stream is passed through the service to the correct manager.
This sequence diagram shows the flow for retrieving binary data.
Data Structures
IBinaryData Interface
This is the core data structure representing a binary file in a workflow. Its properties vary slightly depending on the storage mode.
| Property | Type | Description |
|---|---|---|
id | string | undefined | A unique identifier for the stored file, formatted as manager-name:file-id. It is undefined in default (in-memory) mode. |
data | string | In default mode, this contains the Base64-encoded binary data. In persistent modes (filesystem-v2, s3), it contains the name of the manager responsible for the file. |
mimeType | string | The MIME type of the file (e.g., image/png). |
fileName | string | undefined | The original name of the file. |
fileSize | string | undefined | A human-readable string representing the file size (e.g., "1.2 MB"). |
fileExtension | string | undefined | The file extension (e.g., png). |
fileType | string | undefined | A general file type category derived from the MIME type. |
directory | string | undefined | The original directory path of the file, if available. |
Sources: packages/core/src/binary-data/binary-data.service.ts:109-153, packages/frontend/editor-ui/src/components/RunDataBinary.vue:59-93
FileSystem Storage Structure
When using the filesystem-v2 mode, files are stored in a structured directory path to avoid conflicts and facilitate cleanup. A corresponding .metadata file is created for each binary file.
- Binary File Path:
<storagePath>/workflows/<workflowId>/executions/<executionId>/binary_data/<uuid> - Metadata File Path:
<storagePath>/workflows/<workflowId>/executions/<executionId>/binary_data/<uuid>.metadata
The metadata file is a JSON file containing properties like mimeType, fileName, and fileSize.
Sources: packages/core/src/binary-data/file-system.manager.ts:12-108
Binary Helper Functions
Nodes interact with the binary data system primarily through a set of helper functions provided during execution. These helpers abstract away the need to interact directly with the BinaryDataService.
| Function | Description |
|---|---|
prepareBinaryData(...) | Converts a Buffer or Readable stream into the IBinaryData format, storing it via the configured manager and returning the metadata object. |
getBinaryDataBuffer(...) | Retrieves the binary content for a given IBinaryData object or property name as a Buffer. |
assertBinaryData(...) | A guard function that ensures an item contains a specific binary property and returns the IBinaryData object, throwing a NodeOperationError if not found. |
setBinaryDataBuffer(...) | Stores a buffer for an existing IBinaryData object. |
binaryToString(...) | Converts a binary Buffer or Readable stream to a string, handling character encoding detection. |
createBinarySignedUrl(...) | Creates a short-lived, secure URL to download a binary file via the n8n REST API. This is only available for persistent storage modes. |
Sources: packages/core/src/execution-engine/node-execution-context/utils/binary-helper-functions.ts
Example: Asserting and Retrieving Binary Data in a Node
// From a node's execute method
const binaryPropertyName = this.getNodeParameter('binaryPropertyName', i);
// 1. Ensure the binary data exists on the input item
const binaryData = this.helpers.assertBinaryData(i, binaryPropertyName);
// 2. Retrieve the file content as a buffer
const buffer = await this.helpers.getBinaryDataBuffer(i, binaryPropertyName);
Sources: packages/nodes-base/nodes/MistralAI/test/GenericFunctions.test.ts:18-29, [packages/core/src/execution-engine/node-execution-context/utils/binary-helper-functions.ts:71-107]
Signed URLs for Downloads
The system provides a mechanism to generate secure, temporary download links for binary data stored persistently. This is used by the frontend to allow users to download files from the workflow execution view.
- Token Creation: The
BinaryDataService.createSignedTokenmethod usesjsonwebtokento create a JWT. The payload contains the binary dataid. The token is signed with a secret key (signingSecretfrom config). - URL Generation: The
createBinarySignedUrlhelper function constructs a URL pointing to the n8n REST API endpoint/rest/binary-data/signed, with the generated token as a query parameter. - Token Validation: When the endpoint is accessed, the
BinaryDataService.validateSignedTokenmethod verifies the JWT and extracts the binary dataid. - File Download: The service then retrieves the file using the ID and streams it to the user.
This flow ensures that binary data can be accessed securely without exposing permanent links or requiring direct access to the storage backend.
Sources: packages/core/src/binary-data/binary-data.service.ts:41-55, packages/core/src/execution-engine/node-execution-context/utils/binary-helper-functions.ts:277-281, [packages/frontend/editor-ui/src/components/RunDataBinary.vue:26-29]