Home

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• src/rendertron.ts
• src/renderer.ts
• package.json
• middleware/README.md
• middleware/src/middleware.ts
• src/test/app-test.ts
• src/index.html

Rendertron is a headless Chrome rendering solution built with Puppeteer, designed to render and serialize web pages on the fly. Its primary purpose is to enable Progressive Web Apps (PWAs) and other JavaScript-heavy sites to serve fully-rendered HTML content to bots and web crawlers that do not execute JavaScript. This improves Search Engine Optimization (SEO) and ensures correct content previews on social media platforms.

The system operates as a standalone HTTP server that receives a URL, renders it using a headless Chrome instance, and returns the serialized static HTML. To integrate Rendertron, a web application uses a middleware layer to inspect incoming requests. Based on the User-Agent header, the middleware decides whether to serve the standard application or proxy the request to the Rendertron service for pre-rendering.

Sources: README.md:5-15, middleware/README.md:6-10

Architecture

The Rendertron ecosystem consists of two main components: the Rendertron service and a middleware component integrated into the client application.

1. Rendertron Service: A Koa-based Node.js server that exposes HTTP endpoints to request page rendering or screenshots. It manages a pool of headless Chrome instances using Puppeteer to handle these requests.
2. Middleware: A layer in your application (e.g., an Express middleware) that intercepts incoming requests. It identifies requests from bots and proxies them to the Rendertron service. For regular users, it passes the request through to the standard application logic.

Sources: README.md:12-15, src/rendertron.ts:14, middleware/README.md:6-10

High-Level Request Flow

The following diagram illustrates how a request from a bot is intercepted by the middleware and handled by the Rendertron service.

Rendertron Service

The core of the project is the Rendertron service, a Koa application that orchestrates the rendering process.

Sources: src/rendertron.ts

Initialization

On startup, the Rendertron class initializes a Koa server, launches a Puppeteer-managed Chrome instance, and sets up the API routes.

1. Loads configuration from config.json or defaults.
2. Launches a headless Chrome browser instance via puppeteer.launch({args: ['--no-sandbox']}).
3. Initializes the Renderer class with the browser instance.
4. Sets up Koa middleware for logging, compression, and body parsing.
5. Defines API routes for /render, /screenshot, and a health check.
6. Starts listening on the configured port.

Sources: src/rendertron.ts:21-40

API Endpoints

The service exposes two main endpoints for rendering and screenshotting.

Method	Endpoint	Description	Query Parameters
GET	/render/<url>	Renders the target <url> and returns the serialized HTML content.	mobile: Set to true to emulate a mobile device.
GET	/screenshot/<url>	Takes a screenshot of the target <url>.	width, height, mobile
POST	/screenshot/<url>	Takes a screenshot with advanced options passed in the JSON body.	width, height, mobile

Sources: README.md:30-50, src/rendertron.ts:55-61

Rendering Process

The Renderer.serialize method contains the logic for rendering a page.

This sequence diagram shows the detailed steps involved when a request hits the /render endpoint.

Key steps in the rendering process include:

• Polyfill Injection: Before navigation, scripts are evaluated to force Shady DOM and Shady CSS polyfills for web components compatibility.
• Navigation: Puppeteer navigates to the URL and waits until the network is idle (networkidle0), ensuring most resources have been loaded.
• Script Stripping: After the page loads, a function is executed in the page context to remove all <script> and <link rel="import"> tags to prevent further execution. However, script tags for structured data like application/ld+json are preserved.
• Base Href Injection: A <base> tag is injected to ensure relative URLs in the serialized content resolve correctly.
• Status Code Handling: The original response status code is preserved. A 304 Not Modified is treated as a 200. The status code can be overridden by including a meta tag in the page: <meta name="render:status_code" content="404" />.

Sources: src/renderer.ts:41-68, 89-91, 101-102, 121-136, src/test/app-test.ts:90-93, 104-107

Middleware

To use Rendertron, applications must implement a differential serving layer. An official Express middleware is provided for this purpose.

Sources: middleware/README.md, middleware/src/middleware.ts

Middleware Logic

The middleware decides whether to proxy a request based on two main criteria:

1. User-Agent Matching: It tests the User-Agent header against a regular expression.
2. URL Exclusion: It tests the request path against a regular expression to exclude static assets.

Configuration

The middleware is configured via an options object passed to the makeMiddleware function.

Property	Type	Default	Description
proxyUrl	string	Required	Base URL of the running Rendertron service.
userAgentPattern	RegExp	A pattern matching common bots (Googlebot, bingbot, Slackbot, etc.).	RegExp for matching requests by User-Agent header.
excludeUrlPattern	RegExp	A pattern matching common static file extensions (.css, .js, .png, etc.).	RegExp for excluding requests by URL path.
injectShadyDom	boolean	false	If true, adds ?wc-inject-shadydom=true to the render URL to force web components polyfills.
timeout	number	11000	Millisecond timeout for the proxy request. If exceeded, the middleware falls through.

Sources: middleware/README.md:29-40, middleware/src/middleware.ts:65-83

Configuration

The Rendertron service can be configured by placing a config.json file in the project root.

Option	Type	Default	Description
datastoreCache	boolean	false	Set to true to enable caching with Google Cloud Datastore.
timeout	number	10000	The timeout in milliseconds for rendering a page.
port	number	3000	The port for the Rendertron service to listen on. Overridden by process.env.PORT.
width	number	1000	Default viewport width for rendering.
height	number	1000	Default viewport height for rendering.

Sources: README.md:118-125, src/rendertron.ts:16, 23-24