File Storage
Relevant source files
The following files were used as context for generating this wiki page:Introduction
The File Storage system provides a standardized interface for uploading and retrieving files within the application's backend. It is implemented in server/storage.ts and acts as a client for an external, pre-configured storage proxy service, referred to as "Forge". This module abstracts the complexities of authentication and API interaction with the proxy, exposing simple storagePut and storageGet functions.
This system is integral to features like the "Document Analysis" screen, where users can upload documents for processing. The module handles the construction of API requests, data formatting, and error handling, ensuring a consistent method for managing file assets. The URLs returned by this service are subsequently used by other backend systems, such as the LLM service, to access file content.
Sources: server/storage.ts, app/(tabs)/analyze.tsx, server/_core/llm.ts
Architecture
The file storage architecture involves three main components: the mobile app client, the application backend, and the external Forge storage proxy. The application backend serves as an intermediary, preventing direct exposure of storage credentials to the client.
- Mobile App Client: The user interacts with the UI (e.g., "Analyze Document" screen) to select a file using native capabilities provided by the
expo-document-pickerlibrary. - Application Backend: The client sends the file to the application's backend. The
storage.tsmodule on the backend receives this request. - Forge Storage Proxy: The
storage.tsmodule then communicates with the external Forge storage proxy, authenticating with an API key and using the proxy's dedicated endpoints for upload and URL generation.
This diagram illustrates the high-level data flow for file uploads.
Configuration
The storage module's functionality is dependent on environment variables that must be configured for the backend server. These variables provide the URL of the storage proxy and the necessary authentication credentials.
| Variable | Description |
|---|---|
BUILT_IN_FORGE_API_URL | The base URL for the Forge storage proxy API. |
BUILT_IN_FORGE_API_KEY | The Bearer token used to authenticate requests to the storage proxy. |
The getStorageConfig function reads these variables. If either is missing, it will throw an error, preventing the application from starting or functioning correctly.
Sources: server/storage.ts:7-16
Core Functionality
The server/storage.ts module exposes two primary asynchronous functions for file management.
File Upload (storagePut)
The storagePut function handles the entire process of uploading file data to the storage proxy.
Parameters:
relKey(string): The relative path or key where the file will be stored (e.g.,documents/contract.pdf).data(Buffer | Uint8Array | string): The content of the file.contentType(string): The MIME type of the file, defaulting toapplication/octet-stream.
The following sequence diagram shows the detailed steps involved in an upload operation.
If the upload fails, the function throws an error with details from the proxy's response.
Sources: server/storage.ts:68-88
File URL Retrieval (storageGet)
The storageGet function is used to obtain a downloadable URL for a file that already exists in storage. It does not download the file content itself but rather asks the proxy to generate a URL.
Parameters:
relKey(string): The relative path or key of the file to retrieve.
The following sequence diagram illustrates the URL retrieval process.
This two-step process ensures that the storage proxy's internal location is not exposed, and it can generate time-limited or signed URLs if configured to do so.
Sources: server/storage.ts:24-33, server/storage.ts:90-97
Helper Functions
Several internal helper functions support the main operations.
| Function | Description | Source File |
|---|---|---|
getStorageConfig() | Retrieves and validates storage proxy URL and API key from environment variables. | server/storage.ts:7-16 |
buildUploadUrl() | Constructs the full URL for the file upload endpoint (v1/storage/upload). | server/storage.ts:18-22 |
buildDownloadUrl() | Asynchronously fetches a download URL from the proxy's v1/storage/downloadUrl endpoint. | server/storage.ts:24-33 |
normalizeKey() | Removes leading slashes from a file key to ensure a consistent path format. | server/storage.ts:39-41 |
toFormData() | Converts raw file data (Buffer, string, etc.) into a FormData object for POST requests. | server/storage.ts:43-53 |
buildAuthHeaders() | Creates the Authorization: Bearer <token> header object for authenticating requests. | server/storage.ts:55-57 |
Frontend Integration
The user-facing entry point for file uploads is the "Analyze Document" screen, which is part of the planned feature set. The UI design specifies an upload area and camera capture option.
The app/(tabs)/analyze.tsx component implements the UI for this feature. It uses the expo-document-picker package to allow users to select local files.
The user flow for selecting a document is as follows:
The handlePickDocument function orchestrates this process. While the current implementation in analyze.tsx uses a mock setTimeout to simulate analysis, a production version would take the file data from the state and send it to a backend API endpoint, which would then invoke storagePut.
Sources: app/(tabs)/analyze.tsx:39-48, design.md, todo.md, package.json:34
Usage with Other Services
The URLs generated by the storage service are consumed by other backend components. For instance, the LLM service module defines a FileContent type that includes a file_url property. This indicates that files are passed by reference (URL) to the LLM for processing, rather than embedding the entire file content in the API payload. This is an efficient pattern for handling large files like PDFs or audio.
// From server/_core/llm.ts
export type FileContent = {
type: "file_url";
file_url: {
url: string; // This URL would be supplied by storageGet or storagePut
mime_type?: "audio/mpeg" | "audio/wav" | "application/pdf" | "audio/mp4" | "video/mp4";
};
};
Sources: server/_core/llm.ts:16-22
Summary
The File Storage system is a critical backend module that provides a robust and abstracted interface for file operations. By centralizing interaction with an external storage proxy in server/storage.ts, it simplifies development for features requiring file uploads, such as document analysis. Its reliance on environment variables for configuration makes it adaptable to different deployment environments, and its design of passing file URLs to other services like the LLM promotes an efficient and scalable architecture.