Utility Scripts
Relevant source files
The following files were used as context for generating this wiki page:
Utility scripts within the CircleCI Docs Static Site project serve as essential helper tools for maintenance, content migration, and automated documentation generation. These scripts are categorized into maintenance tasks, such as fetching server version branches, and documentation-specific tasks, such as generating API code snippets and metadata.
The scripts are integrated into the broader Architecture of the project and are often invoked via Gulp tasks or directly through npm commands to streamline the development and build pipelines.
Sources: ARCHITECTURE.md:41-45, TECHNICAL_REFERENCE.md:144-156
API Documentation Generation
One of the most complex utility systems in the project is the API documentation pipeline. This system automates the fetching of OpenAPI specifications and the generation of interactive documentation with code samples.
API Snippet Generator
The scripts/generate-api-snippets.js utility uses the HTTPSnippet library to convert OpenAPI operations into executable code samples across multiple programming languages. It processes the specification by generating a HAR-format request and then converting it into specific targets like cURL, Node.js, and Python.
The generator performs several key functions:
- URL Construction: Replaces path parameters with example values derived from the schema.
- Header Management: Injects authentication placeholders (e.g.,
$CIRCLE_TOKEN) and Content-Type headers. - Example Generation: Recursively traverses JSON schemas to create representative request bodies.
- Post-processing: Specifically fixes cURL quotes for shell expansion and wraps Node.js fetch calls in async functions for CommonJS compatibility.
Sources: scripts/generate-api-snippets.js:1-15, scripts/generate-api-snippets.js:203-231, scripts/generate-api-snippets.js:284-306
Build Pipeline Integration
The script is a critical step in the buildApiV2 task located in gulp.d/tasks/build-api-docs.js. This task orchestrates a multi-step pipeline to produce the final API documentation.
The following diagram illustrates the flow of data through the API utility scripts during a build:
This diagram shows the sequential transformation of a raw API specification into a fully enriched HTML document.
Sources: gulp.d/tasks/build-api-docs.js:103-176
Content Migration and Maintenance
The project includes several specialized scripts for transforming content and maintaining repository health. These are primarily located in the scripts/ directory and handle tasks ranging from Git metadata generation to AsciiDoc syntax conversions.
Maintenance Script Reference
| Script | Purpose | Implementation Language |
|---|---|---|
fetch-server-branches.sh | Clones and updates server-4* branches for administration docs. | Shell |
gen-git-last-update-meta.sh | Generates "last updated" metadata for pages based on Git history. | Shell |
convert_frontmatter.py | Converts YAML frontmatter from legacy systems to AsciiDoc attributes. | Python |
convert_tabs.py | Transforms HTML-style tabs into the @asciidoctor/tabs extension format. | Python |
update-xrefs.py | Validates and updates cross-references across the component library. | Python |
Sources: TECHNICAL_REFERENCE.md:144-162, package.json:11
Server Branch Management
The fetch-server-branches.sh script is used to ensure the local development environment has access to versioned server documentation. It is invoked via npm run fetch-server-branches and is a prerequisite for a full documentation build.
Sources: DEVELOPMENT.md:38-40, package.json:11
Development Workflow Integration
Utility scripts are integrated into the daily developer workflow through package.json scripts. This allows developers to run complex automation without deep knowledge of the underlying script logic.
The sequence above demonstrates how utility scripts are triggered during a standard documentation build process.
Sources: package.json:6-12, gulp.d/tasks/build-api-docs.js:35-42
Environment Configuration
Many utility scripts, particularly those involved in the build process, rely on environment variables managed by dotenvx. Key variables include:
LIVERELOAD: Enables live reload for preview tasks.FORCE_NPM_INSTALL: Forces reinstallation of UI dependencies during the build.ALGOLIA_*: Used by search-related scripts to index content.
Sources: TECHNICAL_REFERENCE.md:167-175, package.json:6
Summary
Utility scripts in the CircleCI Docs project provide the automation necessary to maintain a large-scale, multi-repository documentation site. By handling complex tasks like API specification enrichment, Git metadata generation, and cross-reference validation, these tools ensure that the final static site remains accurate and feature-rich without requiring manual intervention for every content update.