Server API Overview
Relevant source files
The following files were used as context for generating this wiki page:The web-archive server is a backend service built with the Hono web framework, designed to run on Cloudflare Workers. Its primary role is to provide a RESTful API for managing archived web pages. This includes uploading new pages from the browser plugin, storing page content and metadata, and serving this data to the web client for display and management. The server interacts with Cloudflare D1 for database operations and Cloudflare R2 for file storage.
The API is consumed by two main clients: the browser plugin, which saves and uploads web pages, and the web frontend, which allows users to browse, search, and manage their archived content. The API endpoints are organized into logical modules, primarily for handling pages and public showcases.
Sources: README.md, packages/server/src/api/pages.ts, packages/server/src/api/showcase.ts
System Architecture
The server acts as the central hub between the clients (browser plugin, web UI) and the Cloudflare storage services (D1 Database, R2 Bucket). All interactions with archived data are mediated through the server's API.
This diagram illustrates the high-level architecture of the web-archive system.
Core Feature: Page Archiving Flow
The central feature is archiving a web page. This process is initiated by the browser plugin and involves scraping the page content, uploading it to the server, and storing the data and metadata.
The following sequence diagram shows the detailed steps involved in saving a page from the browser plugin.
API Endpoints
The server exposes several API endpoints grouped by functionality. All API routes are prefixed with /api.
Pages API (/api/pages)
This is the main API for managing individual archived pages. It handles creation, retrieval, updates, and deletion.
Sources: packages/server/src/api/pages.ts
| Method | Endpoint | Description |
|---|---|---|
POST | /upload_new_page | Uploads a new page. Expects multipart/form-data with title, URL, file, folder ID, and optional tags. |
POST | /query | Queries for pages with pagination, filtering by folder, tag, or keyword. |
GET | /detail | Retrieves the details (metadata) of a single page by its ID. |
GET | /content | Retrieves the raw HTML content of a page by its ID. Used for rendering the page in the web client. |
GET | /screenshot | Retrieves the screenshot image of a page by its screenshot ID. |
DELETE | /delete_page | Marks a page as deleted. This is a soft delete. |
PUT | /update_page | Updates a page's metadata, such as title, description, folder, and associated tags. |
POST | /query_deleted | Retrieves a list of all soft-deleted pages. |
POST | /restore_page | Restores a soft-deleted page. If the associated folder is also deleted, it restores the folder first. |
DELETE | /clear_deleted | Permanently deletes all soft-deleted pages and their corresponding files from the R2 bucket. |
GET | /recent_save | Retrieves a list of recently saved pages. |
PUT | /update_showcase | Updates the isShowcased flag for a page. |
POST | /query_all_page_ids | Retrieves all page IDs within a specific folder. |
POST | /query_by_url | Checks if a page with a given URL has already been archived. |
Page Upload Validation
When uploading a new page via POST /upload_new_page, the server performs validation on the incoming form data.
// packages/server/src/api/pages.ts:17-56
app.post(
'/upload_new_page',
validator('form', (value, c) => {
if (!value.title || typeof value.title !== 'string') {
return c.json(result.error(400, 'Title is required'))
}
// ... other validations for pageUrl, pageFile, folderId, etc.
if (isNotNil(value.bindTags)) {
if (typeof value.bindTags !== 'string')
return c.json(result.error(400, 'bindTags should be a string array'))
try {
const bindTags = JSON.parse(value.bindTags)
if (!Array.isArray(bindTags))
return c.json(result.error(400, 'bindTags should be a string array'))
}
catch (e) {
return c.json(result.error(400, 'bindTags should be a string array'))
}
}
// ...
}),
// ...
)
The hono/validator middleware is used to ensure that required fields like title, pageUrl, pageFile, and folderId are present and correctly typed. It also validates that bindTags is a JSON-parsable string representing an array.
Sources: packages/server/src/api/pages.ts:17-56
Page Content Retrieval Flow
When a user views an archived page, the web client fetches the content from the server.
This diagram shows the sequence of retrieving and displaying a page's content.
Showcase API (/api/showcase)
This API manages pages that are publicly visible in the "Showcase" section. These endpoints are generally not authenticated, allowing public access to featured content.
Sources: packages/server/src/api/showcase.ts
| Method | Endpoint | Description |
|---|---|---|
POST | /query | Retrieves a paginated list of all showcased pages, including their Base64-encoded screenshots. |
GET | /content | Retrieves the raw HTML content of a specific showcased page. Redirects to an error page if not found. |
GET | /detail | Retrieves the metadata for a single showcased page by its ID. |
GET | /next_id | Given a showcased page ID, finds the ID of the next showcased page, enabling sequential navigation. |
Authentication
The server uses a bearer token for authentication. The browser plugin and web client send an Authorization: Bearer <token> header with most requests. The server validates this token to authorize actions. Public-facing endpoints, like those in the Showcase API, do not require this header.
The background.ts file in the plugin is responsible for attaching this header to every API request.
// packages/plugin/background/background.ts:25-36
async function appendAuthHeader(options?: RequestInit) {
const { token } = await Browser.storage.local.get('token') ?? {}
if (!options) {
return {
headers: {
Authorization: `Bearer ${token}`,
},
}
}
options.headers = Object.assign(options?.headers ?? {}, {
Authorization: `Bearer ${token}`,
})
return options
}
If a request to a protected endpoint fails with a 401 Unauthorized status, the plugin updates its local login status.
Sources: packages/plugin/background/background.ts:25-54, packages/server/src/api/showcase.ts:41-61
Summary
The web-archive server API is a well-defined interface built on Hono and Cloudflare services. It effectively separates the concerns of data storage and business logic from the client-side applications. The API provides comprehensive functionality for archiving, managing, and retrieving web pages, with a clear distinction between private, authenticated actions and public, unauthenticated content showcasing. The use of multipart/form-data for uploads and JSON for metadata exchange is a standard and effective approach for this type of application.