Plugin Architecture
Relevant source files
The following files were used as context for generating this wiki page:- packages/plugin/manifest.json
- packages/plugin/background/background.ts
- packages/plugin/contentScripts/main.ts
- packages/plugin/popup/main.tsx
- packages/plugin/lib/single-file-extension-background.js
- packages/plugin/popup/components/PluginHomePage.tsx
- packages/plugin/popup/components/HistoryTaskList.tsx
The Web Archive browser plugin is a crucial component of the ecosystem, responsible for capturing web pages as single, self-contained HTML files and uploading them to a user's self-hosted server. The plugin is built as a standard WebExtension, composed of a background service worker, content scripts that interact with web pages, and a popup UI for user interaction. It heavily leverages the SingleFile library for the core page-saving functionality and webext-bridge for communication between its components.
The architecture is designed to be modular, with clear separation of concerns. The popup handles user input, the background script orchestrates the saving and uploading process, and content scripts perform the in-page scraping. This structure allows for a non-blocking user experience, where page saving occurs asynchronously in the background.
Core Components
The plugin consists of three main parts defined in the manifest.json: the background script, content scripts, and the popup UI.
This diagram shows the high-level interaction between the plugin's components and the external server.
Sources: packages/plugin/manifest.json
Manifest (manifest.json)
The manifest.json file is the entry point and configuration hub for the browser extension. It defines the core components, their entry files, and the permissions required for the plugin to function.
Sources: packages/plugin/manifest.json
Permissions
The plugin requests several permissions to perform its duties:
| Permission | Justification |
|---|---|
activeTab | Allows access to the currently active tab to initiate page saving. |
storage | Used to persist user settings, server URL, and authentication tokens. |
tabs | Needed to query tab information (URL, title) and create new tabs. |
scripting | Required to execute scripts (like the SingleFile library) in the page. |
<all_urls> | Host permission to allow content scripts to run on any page the user wants to save. |
Sources: packages/plugin/manifest.json:14-16, packages/plugin/manifest.json:71-76
Content Scripts
Multiple content scripts are injected into web pages to handle page scraping and interaction. The manifest defines several script injections, primarily from the bundled SingleFile library, which performs the heavy lifting of capturing the page.
| Scripts | run_at | all_frames | Purpose |
|---|---|---|---|
lib/browser-polyfill.min.js, lib/single-file-frames.js, lib/single-file-extension-frames.js | document_start | true | Injects the core SingleFile library and polyfills into all frames to handle frame content. |
lib/single-file-hooks-frames.js | document_start | true | Injects hooks into the main world ("world": "MAIN") of all frames for deeper integration. |
lib/browser-polyfill.min.js, lib/single-file-bootstrap.js | document_start | false | Bootstraps the SingleFile process in the top-level document. |
lib/browser-polyfill.min.js, contentScripts/main.js | document_end | false | Injects the main application logic for communication with the background script. document_end is the default. |
Sources: packages/plugin/manifest.json:17-69
Background Script
The background script (background/background.ts) is the central nervous system of the plugin. It runs as a service worker, persisting independently of any single web page. Its primary responsibilities include:
- Message Handling: It listens for messages from the popup and content scripts via
webext-bridgeto perform actions like logging in, starting a page save, or retrieving settings. - Task Management: It maintains a queue of page-saving tasks. This allows multiple pages to be queued for saving.
- Orchestration: It orchestrates the entire page-saving flow by commanding the content scripts (which use the SingleFile library) to scrape the page, receiving the scraped data, and then uploading it to the server.
- API Communication: It handles all HTTP requests to the backend Web Archive server, including authentication, fetching folders, and uploading the final HTML file.
- State Management: It manages global state such as the user's authentication token and server URL, stored using
browser.storage.
The core page processing is delegated to the SingleFile library, specifically functions within lib/single-file-extension-background.js. The background.ts script acts as a high-level controller for this library.
Sources: packages/plugin/background/background.ts, packages/plugin/lib/single-file-extension-background.js
Popup UI
The popup UI, with its entry point at popup/main.tsx, is a React application that provides the user interface for the extension.
The main component hierarchy is as follows:
main.tsx: The root of the React app, which handles routing between different pages of the popup (Login, Home, Settings, etc.).PluginHomePage.tsx: The main view after a user logs in. It provides the primary actions:- Save the current page.
- Navigate to history or settings.
- Open the web archive homepage.
- Log out.
HistoryTaskList.tsx: A component that displays the list of current and past page-saving tasks, polling the background script for status updates.SavedPageList.tsx: Shows a list of pages from the current URL that have already been archived.
Communication with the background script is handled exclusively through sendMessage from the webext-bridge library.
This diagram illustrates the component structure of the popup and its communication with the background script.
Sources: packages/plugin/popup/main.tsx, packages/plugin/popup/components/PluginHomePage.tsx, packages/plugin/popup/components/HistoryTaskList.tsx, packages/plugin/popup/components/SavedPageList.tsx
Data Flow: Saving a Page
The process of saving a page involves a coordinated effort between all components of the plugin and the remote server.
This sequence diagram details the step-by-step process of a user saving a web page.
- Initiation: The user clicks the "Save Page" button in the popup UI.
Sources: packages/plugin/popup/components/PluginHomePage.tsx:102-105 - Scraping Request: The popup sends a message to the background script to initiate the scraping process. This message includes metadata like the title, folder, and tags.
- Task Queuing: The background script adds a new task to its internal processing queue with a status of
init.
Sources: packages/plugin/background/background.ts - Content Scraping: The background script commands the content script (via the injected
SingleFilelibrary) to capture the current page. The content script processes the DOM, fetches resources, and packages everything into a single HTML string.
Sources: packages/plugin/lib/single-file-extension-background.js - Content Transfer: The resulting HTML content is sent back to the background script.
- Upload: The background script updates the task status to
uploadingand sends the HTML content, along with a screenshot and metadata, to the/api/pages/upload_new_pageendpoint on the Web Archive server as amultipart/form-datarequest.
Sources: packages/plugin/background/background.ts, packages/server/src/api/pages.ts:16-92 - Server Processing: The server saves the HTML file to an R2 storage bucket and records the page's metadata (title, URL, content URL, etc.) in the D1 database.
- Completion: Upon a successful response from the server, the background script marks the task as
done. TheHistoryTaskListcomponent in the popup will reflect this change on its next poll.
Sources: packages/plugin/popup/components/HistoryTaskList.tsx:19-25
Summary
The Web Archive plugin is a well-structured browser extension that effectively separates concerns between its UI, control logic, and page interaction layers. It builds upon the powerful, open-source SingleFile library for its core functionality and uses modern tools like React and webext-bridge to create a responsive and robust user experience. The asynchronous, task-based approach in the background script ensures that the complex process of saving and uploading large web pages does not interfere with the user's browsing activity.