Render API (/render)

Relevant source files

The following files were used as context for generating this wiki page:

• src/rendertron.ts
• src/test/app-test.ts
• src/renderer.ts
• README.md
• middleware/src/middleware.ts
• src/test/datastore-cache-test.ts
• test-resources/http-meta-status-code.html
• test-resources/include-json-ld.html

The Render API is the core feature of Rendertron, a headless Chrome rendering solution built with Puppeteer. Its primary purpose is to render and serialize web pages on the fly, making it possible for single-page applications (SPAs) and progressive web apps (PWAs) to serve fully-rendered HTML to bots and web crawlers that do not execute JavaScript. This significantly improves SEO and content sharing on social platforms.

The API is exposed via a standalone HTTP server, implemented using Koa.js. It takes a target URL, renders it in a headless Chrome instance, waits for the page to become idle, and returns the serialized DOM. The service is highly configurable and includes other functionalities like the Screenshot API.

Sources: README.md, src/rendertron.ts:15-18

API Endpoint

The service exposes a single GET endpoint for rendering.

GET /render/<url>

The <url> path parameter must be the full, URI-encoded URL of the page to be rendered.

Sources: README.md:32-34, src/rendertron.ts:58-59

Query Parameters

The rendering process can be modified with the following query parameters.

Parameter	Type	Description
mobile	boolean	If present, renders the page with a mobile user agent and viewport. Defaults to false.
wc-inject-shadydom	boolean	If present, forces the web components polyfills (Shady DOM) to be loaded and enabled. This is required for web components v1 to render correctly. This is typically added by the Express middleware.

Sources: README.md:37-38, src/rendertron.ts:85, src/test/app-test.ts:53-55, middleware/src/middleware.ts:100-102

Request Handling Flow

The following diagram illustrates the lifecycle of a request to the /render endpoint.

This diagram shows the flow of a render request from receipt to response.
Sources: src/rendertron.ts:49-62, src/rendertron.ts:79-92

1. URL Validation: The server first checks if the requested URL is restricted using the restricted() function. It blocks non-HTTP/HTTPS protocols (e.g., file:///) to prevent security vulnerabilities. If restricted, it returns a 403 Forbidden status.
   Sources: src/rendertron.ts:70-78, src/rendertron.ts:83-84
2. Cache Check: If Datastore caching is enabled (datastoreCache: true in config), the service checks for a cached version of the response. If a valid cache entry is found, it is returned immediately with an x-rendertron-cached header.
   Sources: src/rendertron.ts:52-55, src/test/datastore-cache-test.ts:40-42
3. Render Execution: If the URL is valid and there's no cache hit, the handleRenderRequest function calls renderer.serialize(). This function orchestrates the headless rendering using Puppeteer.
   Sources: src/rendertron.ts:87-88
4. Response: The serialized HTML content and status code from the renderer are returned to the client. A custom header, x-renderer: rendertron, is added to identify the response as having been processed by Rendertron.
   Sources: src/rendertron.ts:89-92

Core Rendering Logic

The Renderer class encapsulates all interaction with Puppeteer for rendering a page.

Sources: src/renderer.ts

Puppeteer Page Setup

Before navigating to the target URL, a new Puppeteer page is configured based on the request parameters.

• Viewport & User Agent: The viewport is set to the configured width and height. If the mobile parameter is present, a mobile user agent string is set, and the viewport is configured for a mobile device.
• Web Components Polyfills: To ensure Web Components (especially Shadow DOM) can be serialized correctly, several polyfills are forcibly enabled by evaluating scripts on the new document before any page content loads.

  page.evaluateOnNewDocument('customElements.forcePolyfill = true');
  page.evaluateOnNewDocument('ShadyDOM = {force: true}');
  page.evaluateOnNewDocument('ShadyCSS = {shimcssproperties: true}');
  

  Sources: src/renderer.ts:65-67

Sources: src/renderer.ts:59-67

Page Navigation and Serialization

The serialization process involves navigating to the page, waiting for it to be ready, and then extracting the processed HTML.

This sequence diagram shows the interaction between the Rendertron server and the Puppeteer page to serialize content.
Sources: src/renderer.ts:82-132

1. Navigation: The server navigates to the URL using page.goto. It waits until the networkidle0 event, which fires when there have been no new network connections for 500ms. This is the primary mechanism for "auto-detecting" when a page has finished loading. A configurable timeout (default 10 seconds) prevents infinitely long renders.
   Sources: src/renderer.ts:82-83, README.md:65-67
2. Content Sanitization: After the page loads, the stripPage function is executed in the browser's context. This function removes most <script> tags and all <link rel="import"> tags to prevent the rendered page from executing further JavaScript or loading more resources. However, it specifically preserves script tags with type="application/ld+json" to support structured data for SEO.
   Sources: src/renderer.ts:28-35, src/test/app-test.ts:98-104, test-resources/include-json-ld.html
3. Base Href Injection: A <base> tag is injected into the <head> of the document. This ensures that any relative URLs for resources like images or CSS resolve correctly against the original domain.
   Sources: src/renderer.ts:41-56
4. Serialization: Finally, the outerHTML of the root <html> element is extracted and returned.
   Sources: src/renderer.ts:130

Status Code Handling

Rendertron preserves the HTTP status code from the original page request. This allows crawlers to correctly interpret redirects, not found pages, or server errors.

• The status code of the initial response from the target URL is captured.
• A 304 Not Modified status is converted to 200 OK, as the content is still successfully served.
• If the original status is 200 OK, the service will check for a special meta tag in the page's HTML to override the status. The first tag found is used.

  <meta name="render:status_code" content="404" />
  

• This override is ignored if the original status code was anything other than 200 or 304.

Sources: src/renderer.ts:112-126, src/test/app-test.ts:111-121, test-resources/http-meta-status-code.html, README.md:80-82

URL Security

To prevent misuse, the service implements several security checks:

• Protocol Restriction: The restricted() function ensures that only http and https protocols are allowed. This blocks requests for local files (file:///) or other potentially harmful schemes.
• Metadata Service Protection: It checks for the metadata-flavor: Google header in the response from the target URL. If this header is present, it indicates a request to the Google Cloud metadata service, which is blocked to prevent exposing sensitive instance information. A 403 Forbidden is returned in this case.

Sources: src/rendertron.ts:70-78, src/renderer.ts:106-110, src/test/app-test.ts:143-146

Caching

Rendertron can be configured to use a Google Cloud Datastore instance for caching rendered pages.

• Activation: Caching is enabled by setting datastoreCache: true in the config.json file.
• Mechanism: It's implemented as a Koa middleware that intercepts requests. On a cache miss, it allows the request to proceed to the renderer and then stores the successful (200 OK) result in the Datastore.
• Cache Key: The request URL is used as the cache key.
• Cached Response: On a cache hit, the stored content and headers are served directly, and the response includes an x-rendertron-cached header with a timestamp.

Sources: src/rendertron.ts:52-55, src/test/datastore-cache-test.ts, README.md:104-105