Project Overview
Relevant source files
The following files were used as context for generating this wiki page:
Flyhard is a research experiment that trains a neural model derived from a fruit fly's measured connectome to drive a simulated vehicle. The system integrates biological brain data, physical limb simulation, and vehicle dynamics within the CARLA simulator. To support the experiment, the project includes an interactive website featuring a live auction where users purchase ad placements on the simulated vehicle's livery.
Sources: README.md:1-10, apps/website/src/components/auction-site.tsx:42-50
System Architecture
The project consists of three primary domains: the neural control system, the physical simulation environment, and the web-based funding platform.
Experiment Data Flow
The core experiment follows a closed-loop control cycle. Visual input from CARLA is processed by a connectome-based model, which actuate limbs that physically interact with steering and pedal controls. The resulting control positions then update the vehicle motion in the simulator.
Sources: README.md:46-51, THIRD_PARTY.md:46-53
Core Components
1. Neural Control System
The controller uses a model built from the MaleCNS v1.0 dataset, retaining 165,122 neurons and over 25 million neuron-pair connections. It translates neural activity into physical joint commands for the fly's body.
- Dataset: MaleCNS (HHMI Janelia, Google Research, et al.)
- Implementation: PyTorch 2.8.0+cu128
- Scope: Currently manages stationary steering and a 24-second instructed driving sequence.
Sources: README.md:16-25, THIRD_PARTY.md:7-15, THIRD_PARTY.md:58-61
2. Simulation & Physics
The project uses MuJoCo for body and contact physics and CARLA 0.9.16 for the driving environment. The fly body (NeuroMechFly) is integrated via FlyGym.
- Physics Rig: Includes a supported seat, joint servos, a passive steering wheel, and a spring pedal.
- Asset Processing:
export_interior_assets.pyconverts MuJoCo segmented fly meshes and recorded poses for use in Unreal Engine and web visualizers.
Sources: THIRD_PARTY.md:37-45, scripts/export_interior_assets.py:7-13
3. Advertising Livery System
The project features a "Mini advertising livery" with 59 unique slot groups. A live auction allows users to claim these spots.
- Mechanic: Every unclaimed spot starts at $1. New bids must exceed the current bid by at least $1.
- Persistence: Ads remain on the car until outbid; there is no fixed closing date.
- Tooling:
rebuild-layout.pyreprojects panels onto the vehicle body in Blender and exports a Draco-compressed GLB for the website.
Sources: apps/website/src/app/terms/page.tsx:12-18, apps/mini-livery/README.md:1-12, apps/mini-livery/rebuild-layout.py:7-12
Livery Technical Specifications
| Feature | Description | File Reference |
|---|---|---|
| Slot Count | 59 unique addressable ad spaces | apps/mini-livery/README.md |
| Mesh Role | logo_surface for artwork, availability_marker for dashed borders | apps/mini-livery/rebuild-layout.py |
| Coordinate System | glTF Y-up in metres (web) / Blender Z-up (source) | apps/mini-livery/README.md |
| Draco Compression | Used for compact web delivery of the GLB model | apps/mini-livery/README.md |
Sources: apps/mini-livery/README.md, apps/mini-livery/rebuild-layout.py:65-75
Asset Transformation Pipeline
The project maintains a pipeline from raw biological/physics data to web-ready 3D assets.
Experiment Milestones
The project utilizes a verifiable experiment plan. The first pilot (2026-09-09) achieved:
- Steering Accuracy: 100/100 held-out targets passed.
- Connectivity: 99% of steering response is lost if the foot grip is disconnected, confirming causal operation.
- Runtime: A packaged GPU runtime (image) is validated on Runpod A40/A6000 hardware.
Sources: README.md:16-32, reports/2026-09-09-runtime/build.json:1-10