Architecture Overview

FluentDo Agent is a hardened, stable, and secure-by-default open-source (Apache-licensed) downstream distribution of Fluent Bit. Its architecture is centered around providing a production-ready data pipeline with predictable releases, long-term support (LTS), and enhanced features not found in the upstream project. The agent is designed for enterprise-grade security and performance, with a smaller footprint optimized for production deployments.

The core of the agent builds upon Fluent Bit, inheriting its core data processing capabilities. However, FluentDo Agent adds a strategic layer of hardening, custom features, and a rigorous release lifecycle. This includes native de-duplication, log sampling, AI-powered routing, and an enhanced storage layer. The project maintains a close but controlled relationship with its upstream counterpart, regularly syncing changes while preserving its unique value-adds.

Sources: README.md

Core Philosophy and Design Goals

The architecture of FluentDo Agent is driven by a set of principles aimed at making it a reliable and efficient choice for production environments. These goals differentiate it from the standard Fluent Bit project.

Sources: README.md

Relationship with Upstream (Fluent Bit)

FluentDo Agent is architected as a downstream distribution of the fluent/fluent-bit project. This means it incorporates the core functionality of Fluent Bit and regularly synchronizes with upstream changes to receive bug fixes and features. A dedicated script, create-sync-pr.sh, is used to manage this process.

The diagram below illustrates the workflow for synchronizing with the upstream Fluent Bit repository.

This diagram shows the automated process of creating a sync pull request.
Sources: scripts/create-sync-pr.sh, README.md

The synchronization process involves:

1. Identifying a commit range between the last synced version and the target version from the upstream repository.
2. Using the create-sync-pr.sh script to collect upstream patches and categorize them.
3. Generating a detailed pull request body that lists all technical fixes, test updates, and other changes.
4. Creating a feature branch and pushing it to the FluentDo Agent repository.
5. Opening a pull request against the main branch for review and merging.

This structured process ensures that FluentDo Agent stays current with upstream developments while allowing for careful integration that preserves its own customizations and security hardening.

Sources: scripts/create-sync-pr.sh:123-172

Release Lifecycle and Support

The project follows a structured, date-based release schedule to provide predictability for its users. It offers both Long-Term Support (LTS) and regular releases.

Version Support Matrix

Sources: README.md

Release Schedule

Sources: README.md

Build and Compilation Architecture

The project uses CMake as its build system, providing a flexible and cross-platform compilation process. This allows developers to build the agent from source, create packages, and run tests in a consistent manner. The architecture supports disabling features at compile time to create optimized, smaller binaries.

The general build flow is as follows:

This diagram illustrates the native compilation process using CMake.
Sources: README.md, source/lib/yyjson-0.12.0/doc/doxygen/html/building-and-testing.html

Key aspects of the build architecture include:

• CMake-Based: Provides a standardized way to manage dependencies and build configurations across different platforms (Linux, macOS, Windows).
• Source Compilation: A native build can be performed by running cmake .. and make from the source/build directory.
• Containerized Builds: The project provides a build.sh script for creating packages for specific Linux distributions (e.g., rockylinux/9) inside containers.
• Configurability: Features can be disabled via CMake flags (e.g., -DYYJSON_DISABLE_WRITER=ON) to tailor the build for specific needs, reducing the final binary size. This is a practice inherited from its dependencies like yyjson.

Sources: README.md, source/lib/yyjson-0.12.0/doc/doxygen/html/building-and-testing.html

Development and Contribution

The development process is governed by strict coding style guidelines and a structured workflow for contributions, particularly for upstream syncs.

Coding Style

The project's C code follows a style based on the Apache C style guidelines. This ensures consistency and readability across the codebase. Key rules include:

• Indentation: 4 spaces for tabs.
• Conditionals: Spaces around conditional expressions.
• Commit Messages: Must be prefixed with the relevant core section (e.g., HTTP: ...), have a subject line under 80 characters, and include a detailed, self-descriptive body.
• Signed-off-by: Commits must be signed off using the git commit -s flag.

Sources: source/lib/monkey/CONTRIBUTING.md, source/lib/librdkafka-2.10.1/CONTRIBUTING.md

Contribution Workflow

The scripts/create-sync-pr.sh script automates the creation of pull requests for upstream syncs. This enforces a consistent format and process.

The sequence diagram below shows the interaction when a developer runs the script.

This diagram outlines the automated steps performed by the create-sync-pr.sh script.
Sources: scripts/create-sync-pr.sh