Project Goals & Philosophy

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• package.json
• src/rendertron.ts
• src/renderer.ts
• middleware/README.md
• src/test/app-test.ts
• CHANGELOG.md

Rendertron is a headless Chrome rendering solution designed to render and serialize web pages on the fly. Its primary goal is to enable Progressive Web Apps (PWAs) and other JavaScript-heavy applications to serve fully-rendered, static HTML to bots (like search engine crawlers and social media scrapers) that do not execute JavaScript. This approach significantly improves Search Engine Optimization (SEO) and ensures proper content display when links are shared.

The project is built with Puppeteer to control a headless Chrome instance and runs as a standalone HTTP server using the Koa.js framework. The intended integration pattern is through middleware within an application's existing backend, which selectively proxies requests from bots to the Rendertron service for rendering. This architecture keeps the rendering logic decoupled from the main application.

Sources: README.md:5-13, package.json:3-4, CHANGELOG.md:12-13, middleware/README.md:6-10

Core Philosophy: Prerendering for Bots

The central philosophy of Rendertron is to provide a differential serving layer. It distinguishes between regular users and bots based on the User-Agent header. While users receive the standard client-side rendered application, bot requests are routed to Rendertron to receive a pre-rendered, static HTML version of the page.

This ensures that crawlers can index the content of dynamic pages without needing to execute JavaScript, which is a common challenge for modern web applications.

Sources: README.md:9-13, middleware/README.md:8-10

Differential Serving Flow

The diagram below illustrates the typical request flow when using Rendertron with its middleware.

This diagram shows how the middleware intercepts all requests, sending bot requests to Rendertron and serving the standard PWA to human users.

Sources: middleware/README.md:8-10, middleware/src/test/middleware-test.ts:74-76

Architectural Principles

Rendertron's architecture is based on a few key principles: being a standalone service, leveraging powerful browser automation tools, and enabling flexible integration.

Standalone HTTP Server

Rendertron is designed to run as a separate, standalone HTTP server, not as a library within another application. This decouples the resource-intensive rendering process from the primary application server. The server is built on Koa.js, a modern and lightweight Node.js framework.

Key components of the server include:

• Koa App: The main application instance.
• Routing: Uses koa-route to define API endpoints like /render, /screenshot, and /_ah/health.
• Middleware: Employs koa-compress for gzipping responses and koa-bodyparser for handling POST request bodies.

Sources: src/rendertron.ts:16-18, 33-39, CHANGELOG.md:13

Powered by Puppeteer

The core rendering capability is provided by Puppeteer, a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol. This was a significant architectural shift from previous versions, chosen to provide a more robust and feature-rich browser automation engine.

The Renderer class encapsulates all interactions with Puppeteer.

This diagram shows the Rendertron class, which manages the server, using the Renderer class to perform Puppeteer-based operations.

Sources: README.md:6, src/renderer.ts:18-22, src/rendertron.ts:20, 27

Middleware-Based Integration

The project's philosophy promotes integration via middleware. This allows developers to easily plug Rendertron into their existing applications (e.g., Express.js, Firebase Functions) without significant refactoring. The middleware is responsible for identifying bot requests and proxying them to the Rendertron service.

Middleware Option	Description
proxyUrl	Required. The base URL of the running Rendertron service.
userAgentPattern	A RegExp to match user agents that should be prerendered.
excludeUrlPattern	A RegExp to exclude certain paths (e.g., static assets) from being prerendered.
injectShadyDom	A boolean to force web components polyfills to be loaded.
timeout	A millisecond timeout for the proxy request to Rendertron.

Sources: README.md:12-13, 22-26, middleware/README.md:12-20, 26-34

Key Features & Capabilities

Rendertron provides two main functionalities through its API: rendering and screenshotting.

Dynamic Rendering

The primary feature is accessed via the /render endpoint. It renders a given URL and returns the serialized HTML content.

Endpoint	Method	Description
/render/<url>	GET	Renders the specified URL and returns the serialized HTML.

Rendering Sequence

The rendering process follows a specific sequence to ensure dynamic content is captured correctly.

This sequence shows the request being handled by the Koa server, which calls the Renderer module. The renderer uses Puppeteer to load the page, wait for it to be idle, clean up scripts, and finally serialize the DOM.

Sources: src/rendertron.ts:93-104, src/renderer.ts:24-150

Special Handling

• Script Stripping: To prevent further resource loading and execution on the client, Rendertron strips most <script> tags and <link rel="import"> tags from the final output. However, it preserves <script type="application/ld+json"> tags to support structured data for SEO.
  Sources: src/renderer.ts:31-37, src/test/app-test.ts:98-105
• Status Codes: The service forwards the HTTP status code from the rendered page. A page can also specify a status code by including a meta tag: <meta name="render:status_code" content="404" />.
  Sources: README.md:92-94, src/renderer.ts:121-136, test-resources/http-meta-status-code.html:13
• Web Component Support: For pages using Web Components, Rendertron can inject the Shady DOM polyfill to ensure shadow DOM content is correctly serialized. This is typically enabled with a query parameter like ?wc-inject-shadydom=true.
  Sources: README.md:80-88, src/renderer.ts:77-79, src/test/app-test.ts:65-71

Page Screenshotting

As a utility for developers, Rendertron can capture a screenshot of the rendered page. This is useful for debugging and verifying that a page is rendering as expected in the headless environment.

Endpoint	Method	Description
/screenshot/<url>	GET / POST	Captures a screenshot of the URL. Viewport dimensions can be passed as query parameters. Additional Puppeteer screenshot options can be sent in a POST body.

Sources: README.md:46-61, src/rendertron.ts:106-135, src/test/app-test.ts:133-139

Configuration

Rendertron can be configured via a config.json file.

Option	Default	Description
datastoreCache	false	Enables caching responses in Google Cloud Datastore.
timeout	10000	The timeout in milliseconds for rendering a page.
port	3000	The port for the Rendertron service to listen on.
width	1000	The default viewport width for rendering.
height	1000	The default viewport height for rendering.

Sources: README.md:120-126