S3 Tools & Operations
Relevant source files
The following files were used as context for generating this wiki page:
S3 Tools & Operations refers to the suite of capabilities within the aws-s3-mcp package that enables AI assistants to interact with Amazon S3 storage via the Model Context Protocol (MCP). This system acts as a secure intermediary, exposing S3's object storage functionalities—such as listing objects and retrieving content—as discrete, callable "tools" for AI models while keeping AWS credentials isolated on the server.
Sources: packages/aws-s3-mcp/README.md:44-50, packages/aws-s3-mcp/src/aws_s3_mcp/services/s3_service.py:1-6
The architecture is built on a foundation of asynchronous I/O using aioboto3, ensuring non-blocking operations suitable for high-concurrency AI applications. It features smart content detection to differentiate between text and binary files, providing automatic Base64 encoding for non-textual data.
Sources: packages/aws-s3-mcp/README.md:58-64, packages/aws-s3-mcp/src/aws_s3_mcp/services/s3_service.py:118-135
System Architecture
The system follows a tiered architecture where MCP tool definitions wrap a core service layer. The S3Service class handles direct interaction with the AWS API, while the S3Config class manages environment-based configuration and security filtering.
Service Flow Diagram
The following diagram illustrates the request flow from an AI model through the MCP layer to AWS S3.
Core Components
S3Service Class
The S3Service is the primary engine for S3 operations. It is designed to be stateless and implements exponential backoff for transient failures.
- Credential Validation: On initialization, the service verifies the presence of AWS credentials using
aioboto3.Session().get_credentials(). - Retry Logic: Implements
_get_object_with_retryusing exponential backoff (e.g.,2**attempt) for non-404 client errors. - Content Detection: Uses MIME type analysis and null-byte heuristics to determine if an object is text or binary.
Sources: packages/aws-s3-mcp/src/aws_s3_mcp/services/s3_service.py:34-46, packages/aws-s3-mcp/src/aws_s3_mcp/services/s3_service.py:159-180
Configuration & Security
The S3Config class enforces security boundaries by filtering accessible buckets based on environment variables.
| Variable | Description | Default |
|---|---|---|
AWS_REGION | The AWS region to connect to | us-east-1 |
S3_BUCKETS | Comma-separated list of allowed buckets | None (All accessible) |
S3_MAX_BUCKETS | Maximum number of buckets to list | 5 |
S3_OBJECT_MAX_KEYS | Hard limit on objects per list request | 1000 |
Sources: packages/aws-s3-mcp/src/aws_s3_mcp/config.py:19-33
Tool Operations
Object Listing (s3_list_objects)
This tool retrieves a list of objects from a bucket with optional prefix filtering. It enforces safety limits defined in the global configuration.
Input Parameters:
bucket_name(str): Required. Name of the target bucket.prefix(str): Optional. Filters keys that start with this string.max_keys(int): Optional. Defaults to 1000. Maximum results to return.
Output Schema:
{
"count": 2,
"objects": [
{
"key": "example.txt",
"last_modified": "2024-03-20T10:00:00Z",
"size": 1024,
"etag": "abc123"
}
]
}
Sources: packages/aws-s3-mcp/src/aws_s3_mcp/services/s3_service.py:48-83, tests/aws-s3-mcp/unit/tools/test_s3_tools.py:27-50
Content Retrieval
There are two primary ways to retrieve content, depending on the expected data type.
1. General Retrieval (s3_get_object_content)
Retrieves any object. If the system detects binary data (e.g., PDF, Image), it automatically encodes the content as Base64.
Sources: packages/aws-s3-mcp/src/aws_s3_mcp/services/s3_service.py:100-140
2. Text-Only Retrieval (s3_get_text_content)
Strictly for text-based files (Markdown, JSON, CSV). If binary data is detected, it returns an error with a suggestion to use the general retrieval tool.
Sources: packages/aws-s3-mcp/src/aws__s3_mcp/services/s3_service.py:228-260
Content Processing Logic
The system uses a sequence of checks to determine how to return data.
Error Handling Pattern
The S3 module follows a "fail-safe" pattern where technical exceptions (ClientError, UnicodeDecodeError) are caught and transformed into structured error dictionaries. This prevents the MCP server from crashing and provides the AI model with actionable error messages.
Example Error Structure:
{
"error": true,
"message": "Failed to get object 'data.pdf': Access Denied",
"details": {
"error_code": "AccessDenied",
"bucket_name": "my-bucket",
"key": "data.pdf"
}
}
Sources: packages/aws-s3-mcp/src/aws_s3_mcp/services/s3_service.py:141-157, tests/aws-s3-mcp/unit/tools/test_s3_tools.py:101-115
Summary
The S3 Tools & Operations system provides a robust, asynchronous interface for interacting with AWS storage. By centralizing logic in the S3Service and enforcing strict configuration through S3Config, the package ensures that AI-driven operations remain secure, performant, and reliable across varying content types.