Quick Start Deployment
Relevant source files
The following files were used as context for generating this wiki page:The Web Archive project is a self-hostable web archiving and sharing service. It is designed to be deployed on the Cloudflare ecosystem, leveraging Cloudflare Workers for the server logic, D1 for the database, and R2 for storage. The system consists of three main components: a browser plugin for saving webpages, a server for processing and storing the archives, and a web client for querying and displaying the saved content.
Deployment involves setting up the Cloudflare infrastructure for production or running a local development environment. The project provides scripts to facilitate both building the service for production and initializing a local setup for development purposes.
Sources: README.md, package.json, packages/server/wrangler.toml
Architecture Overview
The system is composed of a browser plugin, a server running on Cloudflare Workers, and a web client. The server component is the core of the deployment, interfacing with Cloudflare's D1 database and R2 storage buckets to manage archived pages.
This diagram illustrates the high-level architecture of the Web Archive service.
Project Scripts
The root package.json file defines several scripts for managing the development, build, and deployment processes across the different packages (web, server, plugin).
| Script | Description |
|---|---|
init:local | Runs the init:local script within the server package to set up a local development environment. |
dev:web | Starts the development server for the web client. |
dev:server | Starts the development server for the server (Cloudflare Worker). |
build:service | Builds both the server and web packages for production deployment and performs a cleanup step. |
build:plugin | Builds the browser plugin for Chrome. |
build:plugin-firefox | Builds the browser plugin for Firefox. |
deploy | A server-specific script to build the service and deploy it to Cloudflare Pages. |
Sources: package.json:14-23, packages/server/package.json:5-8
Production Deployment (Cloudflare)
Production deployment is managed through Cloudflare's Wrangler CLI. The configuration is defined in wrangler.toml.
Wrangler Configuration
The wrangler.toml file specifies the project name, build output directory, and compatibility settings. It also defines the necessary bindings to Cloudflare services.
name = "web-archive"
pages_build_output_dir = "../../dist/service"
compatibility_flags = [ "nodejs_compat_v2" ]
compatibility_date = "2024-10-21"
[[r2_buckets]]
binding = "BUCKET"
bucket_name = "web-archive"
[[d1_databases]]
binding = "DB"
database_name = "web-archive"
database_id = "ddc7b449-780b-43d9-9920-d7dcfde2407c"
migrations_dir = './src/sql/migrations'
[ai]
binding = "AI"
Sources: packages/server/wrangler.toml
Service Bindings
The server worker requires bindings to three key Cloudflare services, which are made available in the worker's environment.
| Binding | Service | Description |
|---|---|---|
BUCKET | R2 Storage | An R2 bucket named web-archive for storing archived files like HTML. |
DB | D1 Database | A D1 database named web-archive for storing metadata about the pages. |
AI | Workers AI | Used for features like automatic tag generation. |
Sources: packages/server/wrangler.toml:8-19, packages/server/wrangler.raw.toml:5-15
Build and Deploy Process
The build:service script first builds the server and web packages. The deploy script in the server package then uses Wrangler to deploy the built assets to Cloudflare Pages.
This diagram shows the production build and deployment flow.
Local Development Deployment
For local development, the project provides a script to initialize the required Cloudflare services locally using Wrangler.
Local Initialization
The init:local script, located at packages/server/scripts/init_local.sh, automates the creation of a local R2 bucket and a D1 database.
#!/bin/bash
wrangler r2 bucket create web-archive --local
wrangler d1 create web-archive --local
Sources: packages/server/scripts/init_local.sh
This flow chart visualizes the local initialization process.
After initialization, developers can run the server and web client in development mode using the dev:server and dev:web scripts, respectively. The local server will connect to the locally created D1 and R2 instances.
Sources: package.json:15-16