Testing Strategies

Testing in AtomVM is a multi-layered process designed to ensure the stability and correctness of the virtual machine across various hardware platforms and web environments. The strategy spans from low-level C unit tests and Erlang module verification to hardware-in-the-loop simulation and automated E2E browser testing.

The primary goal of these strategies is to validate the Bogdan Erlang Abstract Machine (BEAM) implementation, peripheral drivers (GPIO, I2C, SPI), and platform-specific integrations like Emscripten for WebAssembly.

Core Emulator and Module Testing

The foundation of AtomVM testing resides in the tests/ directory, which utilizes a C-based test runner to execute compiled Erlang modules. These tests compare the execution results of AtomVM against the standard Erlang/OTP BEAM to ensure behavioral parity.

C-Based Test Runner

The test-erlang executable (compiled from tests/test.c) is the primary driver for functional testing. It iterates through a suite of Erlang modules, loading them into a GlobalContext, executing a start/0 function, and asserting the returned value matches an expected integer.

Test Categories

The internal test suite covers several critical VM components:

• Arithmetic and Logic: Bitwise operations, big integers, and floating-point math.
• Process Management: Spawning, monitoring, links, and mailbox operations.
• Data Structures: Lists, tuples, maps, and binary pattern matching.
• BIFs and NIFs: Built-in and native functions including crypto and zlib.

Sources: tests/test.c:134-450

Hardware and Simulator Testing

For microcontrollers like the ESP32, AtomVM employs both physical hardware testing and simulated environments to validate peripheral interactions and system stability.

Wokwi CI Simulation

Wokwi CI allows for "on device" testing without physical hardware. It uses a diagram.json file to describe the virtual board layout, including pin connections and sensors.

Smoke Testing

Smoke tests are used to verify basic functionality on physical ESP32 devices. A dedicated set of scripts simplifies the environment setup, flash erasing, and binary deployment.

Sources: tools/test/smoke-test/README.md:38-65

WebAssembly and E2E Browser Testing

The Emscripten port of AtomVM is tested using Cypress for End-to-End (E2E) validation. This ensures that Erlang code running in a browser or WebWorker can correctly interact with the DOM and Web APIs.

Cypress Test Suites

• Platform Verification: Checks if the VM correctly identifies its platform as emscripten.
• HTML5 Events: Validates that Erlang processes can receive and handle DOM events (e.g., button clicks).
• Memory Integrity: Monitors for memory leaks or crashes when processes terminate in the browser environment.
• WebSockets: Tests network connectivity using echo servers in both local and CI environments.

Continuous Integration and Quality Standards

AtomVM enforces strict coding and contribution guidelines to maintain testability and code quality.

Build and Coverage

The project uses CMake to manage test targets. When the COVERAGE flag is enabled, the build system integrates lcov to generate code coverage reports for the core VM.
Sources: tests/CMakeLists.txt:104-120

Contribution Requirements

• Intermediate Commits: All intermediate commits in a Pull Request must build successfully.
• Coding Style: C code must adhere to the AtomVM C Coding Style (AVMCCS), enforced by clang-format-16. Erlang code is formatted via erlfmt.
• Documentation: Doxygen Javadoc-style comments are required for C functions to ensure API documentation remains accurate.
  Sources: CONTRIBUTING.md:25-30, CONTRIBUTING.md:46-55

Testing strategies in AtomVM ensure that the VM remains robust across diverse execution environments, from the bare-metal ESP32 to modern web browsers via WebAssembly. By combining low-level C assertions, hardware-in-the-loop simulation, and high-level E2E browser tests, the project maintains high reliability for IoT applications.