FluentDo Agent vs. Fluent Bit
Relevant source files
The following files were used as context for generating this wiki page:
FluentDo Agent is a downstream distribution of the open-source project Fluent Bit. It is designed to be a hardened, stable, and secure version of Fluent Bit, packaged with additional features tailored for enterprise and production environments. While it builds upon the core of Fluent Bit, FluentDo Agent introduces several key differentiators in terms of security, performance, support, and feature set.
This document outlines the relationship between FluentDo Agent and its upstream counterpart, Fluent Bit, and details the specific enhancements and modifications that define the FluentDo Agent. The agent maintains synchronization with upstream Fluent Bit to incorporate bug fixes and core features, while layering its own customizations and advanced capabilities.
Sources: README.md:13-15, README.md:105-106, scripts/sync-upstream-patches.sh
Core Relationship and Synchronization
FluentDo Agent is fundamentally a fork of Fluent Bit, but it is managed as a downstream distribution. This means it periodically syncs with the upstream Fluent Bit repository to pull in changes, fixes, and features. This process is managed through a series of scripts that apply upstream commits as patches, ensuring that FluentDo Agent stays current with the core project while preserving its own unique modifications.
Sources: README.md:13, scripts/sync-upstream-patches.sh:5-11
The diagram below illustrates the relationship, showing that FluentDo Agent is built on top of Fluent Bit and adds its own layer of features and hardening.
Upstream Patch Sync Workflow
The synchronization with upstream Fluent Bit is a managed process using the sync-upstream-patches.sh script. This script fetches a range of commits from the upstream Fluent Bit repository, generates patch files for each commit, and categorizes them. Developers can then selectively apply these patches to the FluentDo Agent source tree. This selective process allows the team to include relevant bug fixes and features while excluding changes that conflict with FluentDo's architecture or are not relevant (e.g., upstream CI/CD workflow changes).
The workflow is as follows:
- Define a
fromandtoversion/commit range from the upstream Fluent Bit repository. - The script fetches the upstream repository and generates patch files for all commits in the specified range.
- Patches are categorized into
TECHNICAL,TESTS,PACKAGING,VERSION, andWORKFLOWS. - The script interactively or automatically applies patches, typically prioritizing
TECHNICALandTESTSpatches. - Applied patches are committed individually, with a reference to the original upstream commit hash.
Sources: scripts/sync-upstream-patches.sh:11-17, 140-155, 233-264
This diagram shows the patch synchronization process.
Key Differentiators
FluentDo Agent distinguishes itself from Fluent Bit in several key areas aimed at production readiness and enterprise needs.
Sources: README.md:47-81
Feature Comparison
| Feature Area | Fluent Bit (Upstream) | FluentDo Agent |
|---|---|---|
| Distribution Model | Open-source project | Hardened, downstream distribution |
| Release Cycle | Community-driven | Predictable LTS and regular releases |
| Support | Community support | 24-month LTS and commercial support |
| Security | Standard security practices | Hardened by default (FIPS, disabled interfaces) |
| Footprint | Includes all plugins | Optimized, smaller footprint with essential plugins |
| Advanced Features | Core feature set | Native deduplication, sampling, AI routing, etc. |
| Branding | Fluent Bit | FluentDo Agent |
Sources: README.md:13-21, 31-44, 49-81, scripts/setup-code.sh:127-133
Build Process and Customization
The build process for FluentDo Agent involves taking the synchronized source code and applying specific customizations. The scripts/setup-code.sh script is central to this process. It modifies the CMakeLists.txt file to change project branding, versioning, and package naming.
For example, the package name is changed from fluent-bit to fluentdo-agent, and the vendor is updated to FluentDo.
Sources: scripts/setup-code.sh:81-84, 127-133
# scripts/setup-code.sh:127-133
sed_wrapper -i "s/CPACK_PACKAGE_NAME \"fluent-bit\"/CPACK_PACKAGE_NAME \"fluentdo-agent${CPACK_PACKAGE_NAME_SUFFIX}\"/g" "$FLB_CMAKE"
sed_wrapper -i "s/CPACK_PACKAGE_VENDOR \"Fluent Bit\"/CPACK_PACKAGE_VENDOR \"FluentDo\"/g" "$FLB_CMAKE"
sed_wrapper -i "s/Eduardo Silva <eduardo.silva@chronosphere.io>/Fluent Do <info@fluent.do>/g" "$FLB_CMAKE"
sed_wrapper -i "s/Chronosphere Inc./FluentDo <https:\/\/fluent.do>/g" "$FLB_CMAKE"
This script also sets the agent's unique version number, which follows a Year.Month.Patch format, distinct from Fluent Bit's versioning scheme.
Sources: scripts/setup-code.sh:22, 90-93
Footprint and Optimization
FluentDo Agent is optimized for production deployments with a focus on a smaller footprint. This is achieved through:
- Selective Plugin Inclusion: Only production-essential plugins are included in the default build, reducing binary size.
- Compiler Optimizations: Builds utilize Interprocedural Optimization (IPO/LTO) and dead code elimination to further reduce size and improve performance.
Sources: README.md:50-53
Security Hardening
Security is a primary focus for FluentDo Agent, with several enhancements enabled by default that are not standard in upstream Fluent Bit:
- Compiler Fortifications: Builds are compiled with
FORTIFY_SOURCEand stack protection enabled. - Disabled Interfaces: All remote interfaces are disabled by default, and authentication is required to enable them.
- Disabled Plugins: 17 vendor-specific plugins are disabled by default to reduce the attack surface.
- FIPS-Compliant Builds: The agent can be built in a FIPS-compliant mode using OpenSSL.
Sources: README.md:56-61
Support and Lifecycle
FluentDo Agent offers a more predictable and extended support lifecycle compared to the community-driven support of Fluent Bit.
Release Schedule:
| Release Type | Frequency | Description |
|---|---|---|
| LTS Release | Twice yearly (April, October) | Long-term support for 24 months |
| Regular Release | Quarterly | 6-month support cycle |
| Security Updates | Weekly | CVE patches and critical fixes |
Sources: README.md:38-44
This structured release and support cycle provides stability and predictability for enterprise users, which is a major value-add over the standard Fluent Bit releases.
Sources: README.md:25-36, 63-66
Advanced Features
FluentDo Agent includes several built-in advanced features not present in the standard Fluent Bit distribution:
- Native Deduplication & Sampling: Processors to reduce log volume and associated costs.
- AI-based Filtering and Routing: Smart log routing capabilities using AI processors.
- Enhanced Storage Layer: More efficient buffering mechanisms.
- Native Flattening: For easier integration with OpenSearch/Elasticsearch.
- Type Safety: Automatic conflict resolution for data types.
Sources: README.md:17-21, 68-74
Conclusion
FluentDo Agent is not merely a rebuild of Fluent Bit; it is a carefully curated and enhanced distribution. It leverages the solid foundation of Fluent Bit and adds critical layers of security, optimization, and enterprise-grade features. The project maintains a disciplined synchronization process to benefit from upstream innovations while delivering a more stable, secure, and feature-rich product with a predictable lifecycle and long-term support. This makes FluentDo Agent a compelling choice for organizations requiring a robust and reliable data forwarder for production environments.
Sources: README.md:13-15, 49-81