AWS Infrastructure
Relevant source files
The following files were used as context for generating this wiki page:The AWS infrastructure for the Docker Docs project is responsible for hosting and serving the static documentation website. The site is built using the Hugo static site generator and deployed to a combination of AWS S3 for storage and AWS CloudFront for content delivery. A Lambda@Edge function is used to handle URL redirects. The entire process is automated through a CI/CD pipeline, likely using GitHub Actions, which builds, tests, and deploys the site artifacts to AWS.
Sources: hack/releaser/aws.go, hack/releaser/cloudfront-lambda-redirects.js, CONTRIBUTING.md:13-16
Architecture Overview
The documentation deployment architecture follows a standard static site hosting pattern on AWS. Code changes pushed to the GitHub repository trigger a CI/CD pipeline. This pipeline builds the Hugo site, and upon success, a release tool uploads the generated static files to an S3 bucket. A CloudFront distribution sits in front of the S3 bucket to cache and serve content globally to users. A Lambda@Edge function intercepts requests at the CloudFront edge to manage URL rewriting and redirects.
This diagram illustrates the high-level data flow from a code change to a user viewing the live documentation.
Sources: hack/releaser/aws.go, CONTRIBUTING.md:31-35, hugo.yaml
Build Process
The documentation site is built with the Hugo static site generator. The configuration in hugo.yaml defines the entire build process, including fetching content from multiple upstream repositories using Hugo Modules.
Sources: hugo.yaml:1-5, CONTRIBUTING.md:13-14
Hugo Modules and Vendored Content
A significant portion of the documentation, particularly CLI and API references, is not stored directly in the docker/docs repository. Instead, it is imported from upstream projects at build time. These projects are defined as modules in hugo.yaml.
Key upstream sources include:
- Moby Project (
moby/moby): For Docker Engine API reference. - BuildKit (
moby/buildkit): For Dockerfile reference. - Docker CLI (
docker/cli): For Docker command-line reference. - Docker Compose (
docker/compose/v5): For Compose CLI reference.
This modular approach ensures that reference documentation is always synchronized with its source code.
The diagram below shows how Hugo Modules are used to assemble the final site content.
CI/CD and Deployment
Changes to the documentation trigger a CI/CD pipeline, which is managed by GitHub Actions. The pipeline is responsible for building the site, running validation checks, and deploying the artifacts to AWS.
Sources: CONTRIBUTING.md:31-35, content/guides/nodejs/configure-github-actions.md:1-10
Validation
Before deployment, several validation steps are executed to ensure content quality and site integrity. These checks are run via docker buildx bake validate. The process includes markdown linting, HTML validation, and link checking.
Sources: CONTRIBUTING.md:52-56, agent.yml:119-122
Deployment to S3
The deployment to AWS is handled by a Go-based releaser tool found in hack/releaser/aws.go. This tool automates the process of setting up the S3 bucket and uploading the static site files.
The Upload function in aws.go performs the following steps:
- Creates a new S3 session.
- Checks if the specified S3 bucket exists.
- If the bucket doesn't exist, it creates it with public-read access and configures it for website hosting.
- It then iterates through the local directory of built files and uploads each one to the S3 bucket, setting the appropriate content type.
- After uploading, it invalidates the CloudFront cache to ensure the new content is served.
The flow of the S3 upload process is shown below.
Sources: hack/releaser/aws.go:34-124
S3 Bucket Configuration
The aws.go script configures the S3 bucket with specific settings for static website hosting.
| Parameter | Value/Description | Source File |
|---|---|---|
| ACL | s3.BucketCannedACLPublicRead | hack/releaser/aws.go:73 |
| WebsiteConfiguration | Enabled with index.html as the index document. | hack/releaser/aws.go:80-87 |
| Public Access Policy | A JSON policy allowing s3:GetObject for all principals on the bucket's objects. | hack/releaser/aws.go:94-111 |
CloudFront and Lambda@Edge
CloudFront serves as the Content Delivery Network (CDN), caching the static assets from the S3 bucket at edge locations around the world for low-latency access.
Sources: hack/releaser/aws.go:113-124
Redirect Handling with Lambda@Edge
A Node.js Lambda function, defined in hack/releaser/cloudfront-lambda-redirects.js, is attached to the CloudFront distribution's "viewer request" event. Its primary purpose is to handle pretty URLs by rewriting requests.
The function's logic is as follows:
- It intercepts an incoming request.
- It checks if the request URI path ends with a file extension or a trailing slash.
- If it does not, the function assumes it's a request for a directory index and appends
/index.htmlto the URI. - It then returns the modified request to CloudFront, which proceeds to fetch the correct object from the S3 origin.
This ensures that a request to /get-started/ is served the content from /get-started/index.html.
The following sequence diagram shows the interaction between a user, CloudFront, Lambda@Edge, and S3 for a request that needs rewriting.