Redirect Manager CLI
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Redirect Manager CLI is a Go-based utility designed to manage and deploy AWS S3 website redirects for the CircleCI documentation site. Its primary purpose is to ensure that legacy URLs are mapped to new content locations within the documentation hierarchy, maintaining SEO value and user experience when page structures change. The tool operates by creating empty objects in an S3 bucket with specific x-amz-website-redirect-location metadata, which S3 uses to perform 301 (Permanent Redirect) operations.
Sources: cmd/create-redirects/main.go:1-20, scripts/README-redirects.md:1-15
Architecture and Components
The system is built as a command-line tool that leverages the AWS SDK for Go v2. It follows a producer-consumer pattern where redirect mappings are loaded from a source file and processed concurrently using Go's concurrency primitives.
Logic Flow and Concurrency
The CLI uses an errgroup to manage multiple goroutines and a weighted semaphore to limit the number of simultaneous uploads to S3, ensuring it does not overwhelm the network or hit API rate limits. The default concurrency is set to 20 workers.
The diagram shows the initialization and execution flow of the redirect deployment process.
Sources: cmd/create-redirects/main.go:26-30, cmd/create-redirects/main.go:45-120
Core Data Structures
- options: Stores command-line flags including the target S3 bucket and index slicing (start/end) for batch processing.
- uploader: A struct containing the S3 client, target bucket name, and a semaphore for concurrency control.
- Redirect: A mapping object containing the
Oldpath andNewpath.
Sources: cmd/create-redirects/main.go:18-24, cmd/create-redirects/main.go:102-110
Implementation Details
S3 Object Creation
The CLI creates "shadow" objects at the old URL paths. These objects have a content length of 0 and a content type of text/html, but they contain a specific instruction for S3 to redirect users.
| Field | Type | Description |
|---|---|---|
| Bucket | string | The target S3 bucket name passed as a positional argument. |
| Key | string | The "Old" path, formatted as docs/{old_path}/index.html. |
| WebsiteRedirectLocation | string | The "New" target path, formatted as /docs/{new_path}. |
| ContentLength | int64 | Always 0, as the object itself contains no content. |
Sources: cmd/create-redirects/main.go:130-143, scripts/README-redirects.md:40-50
S3 Metadata Configuration
The core of the redirect functionality relies on the x-amz-website-redirect-location metadata. When an S3 bucket is configured for static website hosting, it detects this metadata and returns a 301 Moved Permanently status code to the client's browser.
The sequence diagram illustrates how S3 handles the metadata created by the CLI tool during a user request.
Sources: scripts/README-redirects.md:7-15, scripts/README-redirects.md:28-35
Usage and Operations
CLI Arguments
The tool accepts various flags to control which redirects are processed, allowing for resume-capability or partial deployments.
| Flag | Default | Description |
|---|---|---|
-start | 0 | The starting index in the redirect list (inclusive). |
-end | -1 | The ending index in the redirect list (inclusive). -1 processes until the end. |
| (Positional) | N/A | The name of the S3 bucket. |
Sources: cmd/create-redirects/main.go:40-44
Environment Integration
The redirect system is integrated into the project's overall CI/CD architecture. It is typically executed during the deploy-production job after the main site has been synced to S3.
Maintenance and Testing
Manual testing of the redirects is performed using curl -I to verify the HTTP status code and the Location header.
# Example manual validation
curl -I "https://circleci.com/docs/about-circleci/"
This command should return HTTP/1.1 301 Moved Permanently with a Location header pointing to the updated documentation path.
Sources: scripts/README-redirects.md:28-38, DEVELOPMENT.md:120-130
Conclusion
The Redirect Manager CLI is a critical component of the CircleCI documentation infrastructure, providing a scalable and concurrent method for managing legacy URL transitions. By utilizing S3's native website redirection features through Go-based automation, the project maintains consistent navigation across content reorganizations while minimizing administrative overhead during deployments.