Input Plugin: Tailing Files (in_tail)
Relevant source files
The following files were used as context for generating this wiki page:
This document provides a comprehensive overview of the Fluent Bit build process, configuration options, and contribution guidelines as derived from the project's source files. It is intended for developers who wish to build the agent from source, customize its features, or contribute to the project.
1. Introduction
Fluent Bit is a highly configurable and extensible log processor. Its build system, based on CMake, allows for fine-grained control over which features and plugins are included, enabling developers to create optimized builds for specific environments. The project encourages community contributions and has established clear guidelines for code style, pull requests, and testing to maintain code quality and stability.
This guide covers the main aspects of building the agent, including command-line options, dependency management, and the process for submitting changes.
2. Building Fluent Bit
Fluent Bit uses CMake as its build system. The main entry point for configuration is the source/src/fluent-bit.c file, which parses command-line arguments, and the source/lib/monkey/CMakeLists.txt file, which defines the build options and dependencies.
2.1. Command-Line Options
The fluent-bit executable accepts a variety of command-line options to control its runtime behavior and configure plugins on the fly. These options provide an alternative to using a static configuration file.
The main function in fluent-bit.c uses getopt_long to parse these arguments.
Sources: source/src/fluent-bit.c:360-577
The following diagram illustrates the flow of starting Fluent Bit with command-line arguments.
This diagram shows the initialization sequence of Fluent Bit, highlighting how command-line options and configuration files are processed.
Sources: source/src/fluent-bit.c:582-747
Key Command-Line Options:
| Option | Long Option | Description |
|---|---|---|
-c | --config=FILE | Specify an optional configuration file. |
-d | --daemon | Run Fluent Bit in the background (if FLB_HAVE_FORK is defined). |
-D | --dry-run | Perform a configuration test without starting the engine. |
-f | --flush=SECONDS | Set the flush timeout in seconds. |
-i | --input=INPUT | Define an input plugin instance. |
-o | --output=OUTPUT | Define an output plugin instance. |
-F | --filter=FILTER | Define a filter plugin instance. |
-p | --prop="A=B" | Set a configuration property for the last defined plugin. |
-e | --plugin=FILE | Load an external plugin from a shared library. |
-l | --log_file=FILE | Redirect log messages to a specified file. |
-v | --verbose | Increase logging verbosity. Can be used multiple times. |
-q | --quiet | Set logging to quiet mode (equivalent to FLB_LOG_OFF). |
-V | --version | Display the version number and exit. |
-h | --help | Print the help message and exit. |
-Y | --enable-hot-reload | Enable the hot-reloading feature. |
Sources: source/src/fluent-bit.c:116-163, source/src/fluent-bit.c:451-577
2.2. CMake Build Options
The build process is managed through CMakeLists.txt files. The file source/lib/monkey/CMakeLists.txt defines many of the core and plugin-related build options. These options allow for compiling a tailored version of Fluent Bit.
Core Build Options:
| Option | Default | Description |
|---|---|---|
MK_DEBUG | No | Build with debug symbols. |
MK_BACKTRACE | Yes | Enable backtrace functionality for debugging crashes. |
MK_HTTP2 | No | Enable experimental HTTP/2 support. |
MK_TESTS | No | Build the testing suite. |
MK_WITHOUT_BIN | No | Do not build the fluent-bit executable (library-only mode). |
MK_STATIC_LIB_MODE | No | Build as a static library. |
Sources: source/lib/monkey/CMakeLists.txt:28-56
Plugin Build Options:
The build system also allows for enabling or disabling specific plugins to reduce the final binary size.
| Option | Default | Description |
|---|---|---|
MK_PLUGIN_DIRLISTING | Yes | Build the Directory Listing plugin. |
MK_PLUGIN_LIANA | Yes | Build the Liana basic network layer plugin. |
MK_PLUGIN_MANDRIL | Yes | Build the Mandril security plugin. |
MK_PLUGIN_AUTH | No | Build the Basic Authentication plugin. |
MK_PLUGIN_CGI | No | Build the CGI plugin. |
MK_PLUGIN_TLS | No | Build the TLS/SSL support plugin. |
Sources: source/lib/monkey/CMakeLists.txt:59-71
3. Contribution Guidelines
The project maintains strict contribution guidelines to ensure code quality, consistency, and legal compliance. These are detailed in various CONTRIBUTING.md files across the repository and its submodules.
3.1. Pull Request Process
The general workflow for contributing involves forking the repository, creating a feature branch from dev, making changes, and submitting a pull request.
This diagram outlines the pull request workflow for the zstd submodule, which is representative of the general process.
Sources: source/lib/zstd-1.5.7/CONTRIBUTING.md
Key requirements for pull requests include:
- CLA: A Contributor License Agreement (CLA) must be signed.
- Branching: All work should be done on a feature branch created from the
devbranch. - Testing: New code should be accompanied by tests, and all existing tests must pass.
- Commit Messages: Commits should have a descriptive subject line prefixed with the affected area (e.g.,
cgrp: ...) and a detailed body. - Code Style: Code must adhere to the project's coding style.
Sources: source/lib/librdkafka-2.10.1/CONTRIBUTING.md, source/lib/zstd-1.5.7/CONTRIBUTING.md
3.2. Coding Style
The project enforces a consistent coding style to maintain readability.
- C Code: The C coding style is based on the Apache C style guidelines, with 4-space indentation. The
librdkafkasubmodule specifies an 8-space indent, similar to the Linux kernel. All code should comply with the C90 standard, with some specific extensions allowed. - Ruby Code: For parts of the standard library written in Ruby (within the mruby submodule), the code should comply with the Ruby standard (ISO/IEC 30170:2012).
- Formatting: Tools like
clang-formatare used to check and fix C/C++ style. Contributors are expected to runmake style-check-changedandmake style-fix-changedbefore committing.
Sources: source/lib/monkey/CONTRIBUTING.md, source/lib/librdkafka-2.10.1/CONTRIBUTING.md, source/lib/nghttp2-1.65.0/third-party/mruby/CONTRIBUTING.md
3.3. API and ABI Compatibility
The librdkafka submodule documentation emphasizes a strict API and ABI compatibility guarantee.
- Existing public APIs must never be changed. This includes function signatures and public struct layouts.
- New features should be introduced via new APIs.
- Deprecation should be handled through documentation (
@deprecate) rather than compiler hints to avoid breaking user builds.
Sources: source/lib/librdkafka-2.10.1/CONTRIBUTING.md
4. Security
Security is a primary concern for the FluentDo Agent. The README.md file outlines the project's security-hardened approach.
- Hardened Builds: Builds are created with
FORTIFY_SOURCEand stack protection enabled. - Secure Defaults: Remote interfaces are disabled by default, and authentication is required.
- FIPS Compliance: FIPS-compliant builds using OpenSSL in FIPS mode are available.
- Vulnerability Reporting: Security issues should be reported privately to
security@fluent.doand not through public GitHub issues.
Sources: README.md
5. Summary
The Fluent Bit build system is designed for flexibility, allowing developers to create customized and optimized binaries. The project's contribution process is well-defined, with a strong emphasis on testing, code style, and maintaining API/ABI stability. By following the established guidelines, contributors can effectively participate in the development of Fluent Bit, ensuring the continued quality and reliability of the software.