Background Workers & Cron
Relevant source files
The following files were used as context for generating this wiki page:
Automation within the Fleet ecosystem is divided between static content generation pipelines and organizational recurring tasks known as "Rituals." These systems ensure that the documentation, query libraries, and departmental responsibilities are updated and communicated through automated triggers and scheduled processing.
The project utilizes a combination of Node.js scripts for intensive build-time automation and YAML-defined recurring tasks that integrate with GitHub's issue tracking system to maintain organizational momentum and data consistency.
Rituals and Scheduled Task Automation
Rituals represent recurring departmental tasks that are defined in YAML configuration files. These tasks are processed by background logic to track accountability and, in specific cases, automatically generate GitHub issues to notify the Directly Responsible Individual (DRI).
Ritual Configuration
Rituals are defined with specific metadata that determines their frequency and automation status. The system supports several frequencies for these recurring tasks.
| Frequency | Description |
|---|---|
| Daily | Tasks performed every business day (e.g., processing swag requests). |
| Weekly | Tasks performed once per week (e.g., updating KPIs). |
| Triweekly | Tasks synchronized with the three-week release cycle (e.g., publishing sprint demos). |
| Monthly | Tasks performed once a month (e.g., refreshing event calendars). |
| Quarterly | Tasks performed at the start of a quarter (e.g., settling event strategy). |
Sources: website/scripts/build-static-content.js:1018, handbook/marketing/marketing.rituals.yml:1-80
Automated Issue Generation (autoIssue)
The background automation script parses rituals.yml files and identifies tasks marked with the autoIssue key. When a ritual reaches its scheduled threshold, the system can automatically create a GitHub issue in a specified repository (e.g., fleet or confidential) with predefined labels.
The diagram shows the logic path used by the build script to transform YAML ritual definitions into actionable GitHub issues.
Sources: website/scripts/build-static-content.js:1018-1060
Static Content Build Pipeline
The build-static-content.js script acts as a heavy-duty background worker that generates the website's data structures and HTML partials from raw source files in the repository. This script is essential for keeping the Standard Query Library, osquery schema, and handbook in sync with the core codebase.
Core Build Functions
The build worker executes several asynchronous tasks simultaneously to process different content types:
- Query Library Parsing: Converts
docs/queries.ymlinto JSON, validates GitHub contributors, and determines line numbers for "Edit this page" links. - Markdown Compilation: Transforms Markdown files from
docs/,handbook/, andarticles/into EJS partials while rewriting relative links to absolute URLs. - Schema Extension: Merges the base osquery schema with Fleet-specific overrides to generate documentation for every supported table.
- Open Positions: Processes
handbook/company/open-positions.ymlto generate dynamic recruitment pages.
Sources: website/scripts/build-static-content.js:31-700
GitHub API Integration
During the build process, background workers interact with the GitHub API to:
- Validate contributor profiles for the query library.
- Retrieve the
lastModifiedAttimestamp for files to ensure accurate sitemap generation. - Check for the existence of labels used in ritual automation.
Sources: website/scripts/build-static-content.js:525-540, website/api/controllers/download-sitemap.js:80-90
GitOps Automation Assistant
Fleet employs an AI-driven automation layer, defined in system-prompt.js, which serves as a background assistant for managing environment configurations via Slack and GitHub.
Operational Modes
The automation assistant operates in two distinct modes based on user triggers:
- Information Retrieval: Uses Fleet tools to query live endpoint data, policy compliance, and vulnerability impact.
- Configuration Changes: Proposes changes to the GitOps repository. It reads existing YAML files (e.g.,
fleets/workstations.yml), modifies them, and submits the changes as a GitHub Pull Request.
The sequence shows how the background automation handles GitOps requests by interacting with the repository structure.
Sources: tools/fleet-slackbot/system-prompt.js:1-50
Data Consistency and Validation
To prevent broken builds or invalid configurations, the background workers perform extensive validation on incoming data before it is baked into the application configuration (.sailsrc).
- Slug Uniqueness: Ensures no two documentation pages or queries share the same URL path.
- Schema Validation: Validates that MDM commands in
docs/mdm-commands.ymlcontain required XML or plist data. - Path Correction: Background scripts automatically rewrite path-relative links like
./...and../...to absolutefleetdm.comequivalents during the static build. - Email Privacy: The build worker scans Markdown files for
@fleetdm.comemail addresses and throws errors to prevent accidental exposure of internal contact info.
Sources: website/scripts/build-static-content.js:235-250, website/scripts/build-static-content.js:315-325
Conclusion
Fleet's background workers and cron-like processes are integral to its "Infrastructure-as-Code" philosophy. By automating the transition from YAML configurations to live website content and GitHub tasks, the system ensures that the organization remains synchronized with the technical state of the project. This automation covers everything from high-level marketing rituals to granular GitOps changes, maintaining high technical accuracy with minimal manual intervention.