Overall System Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The Flyhard project implements a research experiment to train a neural model, built from a fruit fly's measured connectome, to operate a simulated body and drive a vehicle within the CARLA simulator. The architecture integrates biological data structures with mechanical simulation and a web-based interactive auction system.
Sources: README.md:1-5, pyproject.toml:7-9
Core Simulation Pipeline
The system operates through a causal loop where neural activity translates into physical limb movement, which interacts with vehicle controls to drive simulation updates.
The diagram shows the closed-loop control flow from sensory input to vehicle movement.
Sources: README.md:54-58
Neural Controller
The controller uses a model retaining 165,122 traced neurons and 25,563,197 measured neuron-pair connections from the MaleCNS dataset. It applies trained gains and rate dynamics to an engineering interpretation of biological connectivity.
Sources: README.md:11-13, THIRD_PARTY.md:10-14
Mechanical Interface
The simulation constructs a specific cockpit environment using MuJoCo for physics. Key components include:
- WheelRig: A class representing the steering setup, including a support hand and cockpit geometry.
- Actuators: Joint servos for the fly's legs and a passive steering wheel.
- Constraints: A disclosed forefoot grip constraint and a passive spring pedal with an engineered contact sole.
Sources: scripts/export_interior_assets.py:33-35, THIRD_PARTY.md:31-33
Software Components and Dependencies
The architecture is distributed across Python-based simulation/training tools and a Next.js web application.
Backend and Simulation Stack
| Component | Technology | Description |
|---|---|---|
| Physics Engine | MuJoCo 3.9.0 | Provides body and contact physics for the fly and cockpit. |
| Machine Learning | PyTorch 2.8.0 | Hosts the trained graph-based neural model. |
| Vehicle Simulator | CARLA 0.9.16 | Simulates the driving environment and vehicle dynamics. |
| Body Simulation | FlyGym 2.1.0 | Provides the NeuroMechFly body meshes and anatomy helpers. |
Sources: THIRD_PARTY.md:27-30, THIRD_PARTY.md:46-47, pyproject.toml:11-13
Web Architecture
The apps/website module provides an interactive interface for the experiment, including a 3D car viewer and a live auction for advertising spots on the vehicle livery.
The sequence shows the process of purchasing an advertisement spot and updating the system state.
Sources: apps/website/src/components/auction-site.tsx:112-140, apps/website/src/app/privacy/page.tsx:23-28
Data Modeling and Asset Pipeline
The project uses a structured pipeline to convert biological and mechanical data into displayable assets.
Asset Export Flow
Assets are processed from MuJoCo primitives into interchange geometry for use in Unreal Engine or web-based Three.js viewers.
- MuJoCo Mesh Extraction:
export_interior_assets.pyidentifies visible geoms (group-based) and extracts vertices and faces. - Coordinate Transformation: Vertices are scaled and transformed from rig units to car-relative meters.
- Livery Projection:
rebuild-layout.pyuses Blender's BVH tree to ray-cast and project 2D ad panels onto the curved 3D body of the Mini Cooper.
Sources: scripts/export_interior_assets.py:44-75, apps/mini-livery/rebuild-layout.py:30-45
Livery Data Structure
The advertising system identifies slots using a slot_id key. The inventory is stored in ad-spaces.json and includes metadata for the web viewer.
Sources: apps/mini-livery/README.md:25-28
| Field | Type | Description |
|---|---|---|
slot_id | String | Stable identifier for the ad placement. |
role | String | Defines mesh purpose (e.g., logo_surface or availability_marker). |
width_m / height_m | Float | Physical dimensions of the spot in meters. |
texture | Path | Reference to the advertiser-supplied artwork. |
Sources: apps/mini-livery/README.md:31-40, apps/mini-livery/rebuild-layout.py:145-148
Conclusion
The Flyhard architecture creates a bridge between high-fidelity biological connectome data and real-time vehicle simulation. It couples a PyTorch-driven neural controller with a MuJoCo-based physical interface, all while maintaining a public-facing web presence that allows community participation through a live-auction livery system.