Automated Link Checking

The Datadog documentation repository employs an automated link checking system to ensure the integrity and quality of hyperlinks within the documentation. This process is integrated into the project's CI/CD pipeline, automatically scanning for broken or invalid links whenever changes are proposed. This prevents dead links from being published to the live documentation site, improving the user experience.

The core of this system is a JavaScript-based configuration that defines the rules for validating URLs, including which links to ignore, how to handle redirects, and what constitutes a valid response.

Sources: README.md, local/etc/link-check-config.js

CI/CD Integration

Automated link checking is a critical step in the continuous integration pipeline. When a contributor opens a pull request (PR), a pipeline is triggered that builds a preview of the documentation. This process includes running various quality checks, one of which is the link checker. This ensures that potential issues, such as broken links, are identified before the changes are merged into the master branch and deployed to production.

Sources: README.md, CLAUDE.md

The following diagram illustrates the CI process for a pull request.

Sources: README.md, CONTRIBUTING.md, CLAUDE.md

Configuration

The behavior of the link checker is controlled by the local/etc/link-check-config.js file. This file exports a configuration object that specifies how links should be discovered, filtered, and validated.

Sources: local/etc/link-check-config.js

Core Configuration Options

The main properties of the configuration are summarized in the table below.

| Property | Type | Description -
| aliveStatusCodes | Array<number> | An array of HTTP status codes that are considered successful. The checker accepts 200 (OK) and 203 (Non-Authoritative Information). -
| ignorePatterns | Array<Object> | A list of regular expression patterns to ignore certain URLs. -
| replacementPatterns | Array<Object> | A list of patterns to replace parts of a URL before checking it. This is useful for handling dynamic or environment-specific URLs. -
| retryOn429 | boolean | If true, the checker will retry requests that fail with a 429 Too Many Requests status code. -
| retryCount | number | The number of times to retry a failed request. Set to 2. -
| timeout | string | The timeout for each HTTP request, set to '20s'. -

Sources: local/etc/link-check-config.js:3-12

Link Processing Flow

The checker processes each discovered link through a series of steps defined by the configuration file.

Sources: local/etc/link-check-config.js

Ignore Patterns

The configuration specifies several patterns for URLs that should be excluded from the check. This is useful for links that are known to be valid in the build environment but not publicly accessible, or for placeholder links.

Key ignored patterns include:

• localhost URLs.
• Links to Datadog's internal Atlassian wiki (*.datadoghq.com/wiki).
• Specific documentation paths that may be conditionally available (/getting_started/service_catalog/).
• Example domains as defined by RFC 2606, such as example.com.
• Placeholders like <FOO> or https://<MY_SUBDOMAIN>.datadoghq.com.

Sources: local/etc/link-check-config.js:14-49, CONTRIBUTING.md

Replacement Patterns

Replacement patterns are used to modify URLs before they are checked. This is particularly useful for handling translated content or environment-specific paths. For example, links pointing to French (/fr/) or Korean (/ko/) versions of certain documentation pages are rewritten to point to the English (/en/) version for validation purposes. This ensures that the link structure is validated against the primary English content, which is the source of truth.

Sources: local/etc/link-check-config.js:50-61

Here is the implementation of the replacement pattern:

// local/etc/link-check-config.js:50-61
replacementPatterns: [
    {
        // For any page that's not on the docs, api, or marketplace domains,
        // replace /ja/ and /ko/ with /en/ to check against the english page
        pattern: '^((?!docs|api|marketplace).datadoghq.com\/)(ja|ko)',
        replacement: '$1en'
    },
    {
        // For any page that's not on the docs, api, or marketplace domains,
        // replace /fr/ with /en/ to check against the english page
        pattern: '^((?!docs|api|marketplace).datadoghq.com\/)(fr)',
        replacement: '$1en'
    }
],

This configuration ensures that links to translated content on non-core domains are validated against their English counterparts.

Sources: local/etc/link-check-config.js:50-61

Summary

The automated link checking system is an essential part of the documentation's quality assurance process. By integrating directly into the CI/CD pipeline and using a flexible JavaScript-based configuration, it proactively prevents broken links from being deployed. This system maintains the high quality of the documentation and ensures a reliable experience for users navigating the site.

Sources: README.md, local/etc/link-check-config.js, CLAUDE.md