Machine Learning (AI/ML)
Relevant source files
The following files were used as context for generating this wiki page:
Machine Learning (AI/ML) within the Zephyr Project ecosystem is focused on providing a scalable and resource-optimized environment for deploying artificial intelligence on constrained devices. The project integrates industry-standard external modules such as TensorFlow Lite Micro (TFLM) and CMSIS-NN to enable efficient inference on a wide variety of hardware architectures, including ARM, RISC-V, and x86.
The integration follows the standard Zephyr External Components model, where ML libraries are treated as modules linked into the final monolithic binary. This approach ensures that AI/ML capabilities benefit from the core kernel services, such as execution protection, memory management, and thread separation, which are critical for secure and safe deployment in IoT and industrial environments.
Sources: README.rst:1-15, doc/security/security-overview.rst:45-55, modules/tflite-micro/CMakeLists.txt, modules/cmsis-nn/CMakeLists.txt
Architecture and Integration
The AI/ML stack in Zephyr is structured to bridge high-level neural network frameworks with hardware-specific optimizations. It primarily leverages the Zephyr module system to include specialized libraries.
Component Hierarchy
The architecture consists of three main layers:
- Framework Layer: High-level inference engines like TensorFlow Lite Micro that manage model execution and operator kernels.
- Optimization Layer: Libraries such as CMSIS-NN that provide optimized implementations of common neural network operators (e.g., convolutions, pooling) for specific hardware targets like ARM Cortex-M.
- Kernel/Hardware Layer: The underlying Zephyr RTOS providing scheduling, memory protection, and hardware abstraction.
The diagram shows the flow from application code through the ML framework to either generic or optimized hardware execution paths.
Sources: modules/tflite-micro/CMakeLists.txt, modules/cmsis-nn/CMakeLists.txt, doc/security/security-overview.rst:95-105
Key ML Modules
Zephyr integrates several key modules to support AI/ML workflows:
TensorFlow Lite Micro (TFLM)
TFLM is the primary inference engine used for running machine learning models on microcontrollers. It is designed to have a small footprint and does not require dynamic memory allocation, fitting the Monolithic Design of Zephyr.
Sources: modules/tflite-micro/CMakeLists.txt, doc/security/security-overview.rst:50-60
CMSIS-NN
CMSIS-NN provides optimized neural network kernels for ARM processors. When used alongside TFLM, it accelerates inference significantly by utilizing SIMD instructions and optimized memory access patterns.
Sources: modules/cmsis-nn/CMakeLists.txt
| Module | Purpose | Key Features |
|---|---|---|
| TFLite Micro | Inference Engine | Interprets .tflite models; support for custom operators. |
| CMSIS-NN | Hardware Acceleration | Optimized kernels for Conv, Depthwise Conv, Fully Connected. |
| PSA Crypto | Security | Used for secure model loading or data encryption via mbedTLS. |
| Sources: modules/tflite-micro/CMakeLists.txt, modules/cmsis-nn/CMakeLists.txt, doc/security/security-overview.rst:48-52 |
Safety and Security for ML
AI/ML deployments often occur in safety-critical or high-security environments. Zephyr applies its standard Safety Overview and Security Overview to ML modules.
Secure Execution
ML models in Zephyr are protected by several kernel-level mechanisms:
- Thread Separation: ML inference can be run in unprivileged threads to prevent the model or input data processing from affecting the rest of the system.
- Memory Protection: Stack guards and memory partitioning ensure that inference operations stay within defined boundaries, preventing buffer overflows during tensor processing.
- Monolithic Linking: Models are typically compiled into the static binary, removing the attack surface associated with dynamic model loading.
This diagram illustrates how Zephyr uses thread separation and the Memory Protection Unit (MPU) to isolate ML inference tasks.
Sources: doc/security/security-overview.rst:62-75, doc/safety/safety_overview.rst:110-125
Functional Safety (IEC 61508)
For ML applications requiring functional safety, Zephyr targets the IEC 61508 standard (SIL 3). This requires:
- 100% Test Coverage: ML-related code must meet structural, statement, and branch coverage requirements.
- Requirements Tracing: Tracing from system-level ML requirements down to the specific implementation in the module code.
Sources: doc/safety/safety_overview.rst:25-45, doc/safety/safety_overview.rst:145-160
Configuration and Build System
AI/ML features are enabled through Zephyr's Kconfig and CMake systems. This allows for modular inclusion of only the necessary kernels and operators to minimize the binary footprint.
Build Integration Example
The integration of TFLite Micro is handled via CMake logic that pulls in the necessary source files from the external module directory.
# Representative logic from modules/tflite-micro/CMakeLists.txt
if(CONFIG_TENSORFLOW_LITE_MICRO)
zephyr_library()
zephyr_library_sources(
tensorflow/lite/micro/all_ops_resolver.cc
tensorflow/lite/micro/micro_interpreter.cc
# Additional framework files...
)
endif()
Sources: modules/tflite-micro/CMakeLists.txt:1-15
Summary
Machine Learning in the Zephyr Project is achieved through the integration of lightweight, industry-standard modules like TFLite Micro and CMSIS-NN. By leveraging Zephyr’s monolithic architecture and robust execution protection mechanisms, developers can deploy AI models that are not only performant on resource-constrained hardware but also meet stringent security and safety requirements.
Sources: README.rst:10-20, doc/security/security-overview.rst:220-230