Local Development Setup

The web-archive project is a monorepo managed with pnpm workspaces. The local development environment consists of three primary components that run concurrently: a backend server, a web frontend, and a browser plugin. This guide outlines the steps and commands required to initialize the local environment and run these services for development.

The setup leverages Vite for fast development servers and Wrangler for emulating the Cloudflare environment locally. Understanding the interaction between these components is key to working on the project.

Project Structure

The project is organized as a monorepo, with its packages defined in pnpm-workspace.yaml. This structure allows for shared dependencies and streamlined development across the different parts of the application.

# pnpm-workspace.yaml
packages:
  - 'packages/*'

Sources: pnpm-workspace.yaml:1-2

The main packages are:

• packages/server: The Hono-based backend server that runs on Cloudflare Workers.
• packages/web: The React-based web client for viewing and managing archived pages.
• packages/plugin: The browser extension used to capture and upload web pages.

Prerequisites

The project uses pnpm as its package manager. Ensure you have pnpm installed to manage dependencies and run scripts. The required version is specified in the root package.json.

# package.json:3-3
"packageManager": "pnpm@9.4.0+sha512.f549b8a52c9d2b8536762f99c0722205efc5af913e77835dbccc3b0b0b2ca9e7dc8022b78062c17291c48e88749c70ce88eb5a74f1fa8c4bf5e18bb46c8bd83a"

Sources: package.json:3-3

Initialization

Before running the development servers, you must initialize the local Cloudflare environment. This involves creating a local D1 database and an R2 storage bucket. The project provides a script to automate this process.

Run the following command from the root of the repository:

pnpm init:local

This command is a shortcut defined in the root package.json that executes the init:local script within the server package.

Sources: package.json:15-15

Initialization Script Flow

The init_local.sh script uses the Wrangler CLI to set up the necessary local infrastructure.

The diagram below illustrates the steps performed by the initialization script.

The script executes these specific Wrangler commands:

1. wrangler d1 create web-archive-db --local: Creates the local D1 database.
2. wrangler r2 bucket create web-archive-bucket --local: Creates the local R2 bucket for file storage.
3. wrangler d1 migrations apply web-archive-db --local: Applies database schema migrations to the local D1 database.

Sources: packages/server/scripts/init_local.sh:3-5

Running Services

The three core components (server, web, plugin) must be run in separate terminal sessions. The root package.json provides convenient scripts to start each service in development mode.

Development Scripts

Sources: package.json:16-18, packages/server/package.json:4-4, packages/web/package.json:4-4, packages/plugin/package.json:11-11

The following diagram shows the overall architecture of the local development environment.

Server (dev:server)

The server runs on Vite, using @hono/vite-dev-server to provide a local development environment that emulates Cloudflare Pages. It handles API requests, interacts with the local D1 database, and serves files from the local R2 bucket.

Sources: packages/server/package.json:12-13

Web (dev:web)

The web frontend is a standard React application also running on Vite. It makes API calls to the backend server to fetch data and render pages. For example, when viewing an archived page, it fetches the page content from the /api/pages/content endpoint.

Sources: packages/web/package.json:4-4, packages/web/src/pages/(layout)/page.[slug].tsx:20-29

Plugin (dev:plugin)

The browser plugin is built using Vite with the @crxjs/vite-plugin. The dev script starts a watch process that rebuilds the extension on any file change. For development, it uses manifest.dev.json, which is configured for a local environment, likely with different permissions or settings than the production manifest.

Sources: packages/plugin/package.json:11-11, packages/plugin/manifest.dev.json

Development Workflow Example: Viewing a Page

When a user navigates to a saved page in the web application, a series of interactions occurs between the frontend, the server, and the local Cloudflare services.

The following sequence diagram illustrates the data flow for fetching and displaying an archived page's content.

This flow demonstrates how the decoupled frontend and backend services work together in the local development environment to serve content from the emulated R2 bucket.