Testing Framework
Relevant source files
The following files were used as context for generating this wiki page:
The Parashell testing framework provides a multi-layered infrastructure for validating software quality, spanning unit tests, integration tests, and system-level validation. It is designed to ensure that contributions preserve existing functionality while allowing for experimental risk-taking within a controlled environment. The framework supports execution through multiple interfaces, including a Graphical User Interface (GUI), a Command Line Interface (CLI), and internal Python scripting within the application environment.
Testing is a mandatory component of the development lifecycle; all Pull Requests (PRs) must compile cleanly and pass project self-tests on all target platforms before they can be merged. The framework specifically targets regressions, which are prioritized as the highest category of bug fixes to prevent development from sliding backward.
Sources: CONTRIBUTING.md:45-55, CONTRIBUTING.md:100-105, src/Mod/CAM/Roadmap/README.md:110-120
Architecture and Execution Methods
The framework employs a flexible execution model that allows developers to run tests at various levels of granularity: entire modules, specific classes, or individual test methods.
Execution Interfaces
- Command Line Interface (CLI): Tests can be executed using the
FreeCADCmdorFreeCADexecutables with the--run-testflag. Passing0runs all tests, while specific strings target individual modules or classes. - Graphical User Interface (GUI): A dedicated "UnitTest" dialog allows users to select test names from a list, start the execution, and view progress, failures, and errors in real-time.
- Python Scripting: Within the application console, developers can use the
Testmodule to run tests from classes or modules dynamically.
Sources: src/Mod/Fem/femtest/test_information.md:85-115, src/Mod/Test/Gui/Resources/translations/Test.ts:5-40
Test Execution Logic
The following diagram illustrates the flow of test execution from user initiation to results.
This diagram shows how different entry points converge on the internal test runner to load and execute test cases.
Sources: src/Mod/Fem/femtest/test_information.md:30-120, src/Mod/Test/Gui/Resources/translations/Test.ts
Unit Testing Framework
Unit testing is primarily built upon the Python unittest standard library, extending it to handle application-specific objects and workbench metadata.
Metadata Validation
The framework includes specialized tests to validate the package.xml files used by workbenches. These tests ensure that metadata such as names, versions, descriptions, and dependencies are correctly formatted and accessible.
| Test Category | Description | Derived Source |
|---|---|---|
| Constructor Tests | Validates metadata object creation from XML files and handles bad root nodes or invalid versions. | src/Mod/Test/Metadata.py:35-60 |
| Tag Validation | Ensures top-level tags like Name, Version, Date, and Author are correctly parsed. | src/Mod/Test/Metadata.py:65-85 |
| Dependency Checking | Verifies that workbench dependencies (internal, addon, python) are accurately mapped. | src/Mod/Test/Metadata.py:125-155 |
| Content Types | Validates the parsing of nested content like macros and preference packs. | src/Mod/Test/Metadata.py:100-115 |
Workbench-Specific Testing (FEM)
The FEM (Finite Element Method) workbench utilizes the framework to validate solvers and object creation. It supports running tests with or without closing the document to allow for visual inspection of results.
import Test, femtest.app.test_object
# Run tests for a specific class
Test.runTestsFromClass(femtest.app.test_object.TestObjectCreate)
# Run all FEM App tests
import TestFemApp
Test.runTestsFromModule(TestFemApp)
Sources: src/Mod/Fem/femtest/test_information.md:35-80
Integration and Platform Testing
Integration testing ensures that various components work together within the OS environment. This is exemplified by the macOS QuickLook extension tests, which verify that the application bundle is correctly structured and that extensions are properly registered with the system.
Mac Integration Test Suite
The framework uses shell scripts to perform environmental sanity checks:
- Structure Verification: Checks for the existence of executables and
.appexplugin directories within the app bundle. - Bundle Identification: Validates
Info.plistCFBundleIdentifiers for thumbnails and previews. - Security and Signing: Verifies code signing status and Gatekeeper compliance.
- Registration: Uses
pluginkitto manually register and activate extensions during testing.
Sources: src/MacAppBundle/QuickLook/test_integration.sh:65-150, src/MacAppBundle/QuickLook/test_integration.sh:195-235
Testing Roadmaps and Refactoring
Modernization efforts in the CAM workbench demonstrate a shift toward "refactored" testing. Legacy postprocessors were often difficult to test due to code duplication. The refactored approach shares a common code base, allowing for:
- Shared Test Suites: Testing the core logic once for multiple machine dialects (e.g., Centroid, GRBL, LinuxCNC).
- Atomic Tests: Ensuring tests are efficient and do not require excessive disk I/O.
- Object-Oriented Customization: Using inheritance to test unique behaviors of specific controllers while relying on base class tests for standard G-code commands.
Sources: src/Mod/CAM/Roadmap/Epics/ImprovePostprocessors.md:40-75, src/Mod/CAM/Roadmap/Epics/ImprovePostprocessors.md:150-165
Conclusion
The Parashell testing framework is a robust system that integrates standard Python unit testing with specialized GUI dialogs and platform-specific integration scripts. By enforcing strict PR requirements and providing tools for workbench-specific validation, it maintains architectural integrity across a diverse and extensible codebase. The ongoing transition from legacy per-file testing to refactored, shared test bases reflects a commitment to developer efficiency and long-term maintainability.
Sources: CONTRIBUTING.md:100-110, src/Mod/CAM/Roadmap/Epics/ImprovePostprocessors.md:160-170