Extending with WebAssembly (Wasm)

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• source/README.md
• source/lib/yyjson-0.12.0/doc/doxygen/html/index.html
• source/lib/yyjson-0.12.0/doc/doxygen/html/building-and-testing.html
• source/lib/zstd-1.5.7/CONTRIBUTING.md
• source/lib/luajit-04dca791/dynasm/dynasm.lua
• source/lib/nghttp2-1.65.0/doc/mkapiref.py
• source/lib/monkey/deps/mbedtls-2.4.2/README.md

The provided source files do not contain specific implementation details regarding the use of WebAssembly (Wasm) for extending FluentDo Agent. The project's README.md file mentions "AI processors for routing" and "AI-based filtering and routing" as advanced features. While such features can be implemented using Wasm, there is no direct evidence or documentation in the given files to confirm this or describe the mechanism.

The analyzed files primarily consist of high-level project descriptions, build scripts, contribution guidelines, and documentation for the third-party libraries included in the agent.

Analysis of Provided Source Files

The available files offer insights into the overall project structure, its dependencies, and development practices, rather than specific extension mechanisms like Wasm.

Project Overview

The README.md files establish that FluentDo Agent is a hardened, open-source downstream distribution of Fluent Bit. It is designed for stability and security, with predictable releases and a 24-month Long-Term Support (LTS) cycle.

Key aspects mentioned include:

• Advanced Features: Native de-duplication, log sampling, AI-based filtering/routing, and an enhanced storage layer.
• Security Hardening: Built with FORTIFY_SOURCE, stack protection, and FIPS-compliant options. Remote interfaces are disabled by default.
• Build & Installation: Instructions are provided for running via Docker, installing pre-built packages, and building from source.

Sources: README.md, source/README.md

Third-Party Libraries

The agent bundles several core third-party libraries. The documentation for these libraries details their purpose, build process, and contribution models.

The following diagram illustrates the relationship between the FluentDo Agent and some of its key bundled libraries as inferred from the file structure.

yyjson (JSON Library)

yyjson is a high-performance C library for JSON processing.

• Features: It is compliant with RFC 8259, provides high performance, and supports accurate handling of int64, uint64, and double types.
• Building: The library can be built from source or integrated using CMake. It offers extensive compile-time options to customize the build by enabling or disabling features like the reader, writer, or non-standard JSON support.
• API: The documentation includes sample code for reading from and writing to JSON strings and files, handling errors, and iterating over objects and arrays.

yyjson Compile-time Option	Description
YYJSON_DISABLE_READER	Disables the JSON reader to reduce binary size.
YYJSON_DISABLE_WRITER	Disables the JSON writer to reduce binary size.
YYJSON_DISABLE_NON_STANDARD	Disables support for non-standard features like comments or NaN.
YYJSON_DISABLE_FAST_FP_CONV	Disables built-in fast floating-point conversion, falling back to strtod/snprintf.

Sources: source/lib/yyjson-0.12.0/doc/doxygen/html/index.html, source/lib/yyjson-0.12.0/doc/doxygen/html/building-and-testing.html

zstd (Compression Library)

zstd is a fast, lossless compression library.

• Development Process: Contributions are managed through a dev branch, with feature branches for new development. Pull requests must be made against the dev branch.
• Performance: The CONTRIBUTING.md file places a strong emphasis on performance. It provides detailed guidelines for benchmarking, including ensuring a stable machine, running multiple trials, and using profiling tools like perf and Instruments.
• Coding Style: The library follows a strict C90 standard with minimal dependencies to ensure portability.

Sources: source/lib/zstd-1.5.7/CONTRIBUTING.md

luajit / dynasm (Dynamic Assembler)

DynASM is a dynamic assembler for code generation engines, used by LuaJIT.

• Functionality: It processes .dasc files containing a mix of C code and assembly instructions, and outputs a C source file.
• Pseudo-Opcodes: The dynasm.lua script defines the core logic for parsing statements and handling pseudo-opcodes. Key pseudo-opcodes include:
  • .arch: Sets the target architecture (e.g., x86, arm, ppc).
  • .section: Defines code sections.
  • .define: Creates a substitution macro.
  • .macro: Defines a multi-line macro.
  • .if/.else/.endif: Enables conditional assembly.
• Architecture Modules: DynASM is extended with architecture-specific modules like dasm_ppc.lua and dasm_mips.lua, which provide the template strings and parsing logic for the instruction sets of those architectures.

Sources: source/lib/luajit-04dca791/dynasm/dynasm.lua, source/lib/luajit-04dca791/dynasm/dasm_ppc.lua, source/lib/luajit-04dca791/dynasm/dasm_mips.lua

nghttp2 (HTTP/2 Library)

nghttp2 is a C library for the HTTP/2 protocol.

• Documentation Generation: The file mkapiref.py is a Python script used to generate API reference documentation in reStructuredText (.rst) format. It parses specially formatted C-style comments (/** ... */) from source files.
• Directives: The script recognizes directives within comments to structure the documentation, such as:
  • @function
  • @functypedef
  • @enum
  • @struct
  • @union

Sources: source/lib/nghttp2-1.65.0/doc/mkapiref.py, source/lib/nghttp2-1.65.0/doc/README.rst

mbedtls (TLS/SSL Library)

mbedtls is a C library that implements cryptographic primitives, X.509 certificate manipulation, and the SSL/TLS and DTLS protocols.

• Building: It can be built using Make, CMake, or Microsoft Visual Studio. The build system creates three libraries: libmbedcrypto, libmbedx509, and libmbedtls.
• Configuration: Features are configured via include/mbedtls/config.h.

Sources: source/lib/monkey/deps/mbedtls-2.4.2/README.md

Conclusion

The provided source files offer a good overview of the FluentDo Agent's composition, its dependencies, and its development and build practices. However, they do not contain the necessary code or documentation to describe how to extend the agent using WebAssembly. To obtain this information, analysis of files related to the Wasm runtime (such as WAMR) and specific Wasm-based filter plugins would be required.