iPhone LiDAR Capture
Relevant source files
The following files were used as context for generating this wiki page:
iPhone LiDAR Capture is an experimental integration that enables LiDAR-capable iOS devices to function as geometry sensors for the RuView platform. The system uses native iOS components to extract depth geometry and a browser-based visualization surface to render point clouds in real-time.
The integration serves as a high-fidelity spatial sensor that complements existing WiFi-based human sensing capabilities. By streaming compact millimeter wireframes, the iPhone provides precise environmental geometry that can be integrated into the future RuView Hardware Abstraction Layer (HAL) for sensor fusion.
Sources: integrations/iphone-lidar/README.md:1-12, harness/ruview/src/guidance.js:192-202
System Architecture
The capture system follows a multi-stage pipeline from native sensor data acquisition to web-based visualization.
The architecture separates the sensor capture layer (native Swift) from the receiver and visualization layer (Web). Mobile Safari does not expose ARKit scene depth directly, necessitating the native capture manager.
Sources: integrations/iphone-lidar/README.md:14-23
Native Capture Layer
The native path utilizes ARKit to capture spatial data. The application requires an iOS 17 deployment target and specific configuration in the Info.plist to access the camera and LiDAR scanner.
Configuration and Setup
- Target: iOS 17 or newer.
- Privacy Key:
NSCameraUsageDescriptionmust be defined to explain LiDAR geometry capture. - Hardware: Requires a physical LiDAR-capable iPhone or iPad; the simulator does not support LiDAR scene depth.
Sources: integrations/iphone-lidar/README.md:25-36
Data Extraction Logic
The LiDARCaptureManager class manages the ARWorldTrackingConfiguration to extract environmental data. The system specifically targets ARDepthData.depthMap and confidenceMap.
| Component | Purpose |
|---|---|
| sceneDepth | Provides depth information for the current frame. |
| confidenceMap | Indicates the accuracy level for each depth pixel. |
| Camera Intrinsics | Provides lens parameters for point cloud projection. |
| Device Pose | Tracks the position and orientation of the sensor in 3D space. |
The implementation explicitly excludes RGB camera frames to maintain privacy class labels as geometry-only.
Sources: integrations/iphone-lidar/README.md:38-40, integrations/iphone-lidar/README.md:65-68
Wire Format and Transmission
The system uses a custom schema (ruview.lidar.depth.v1) designed for efficient streaming over local networks.
Data Compression
To reduce bandwidth, depth data is downsampled by 2 in each dimension and streamed at a maximum of 15 FPS. Depth samples are encoded as little-endian UInt16 values representing millimeters.
| Data Type | Sizing | Description |
|---|---|---|
| Depth Sample | UInt16 (LE) | Millimeter distance values. |
| Confidence | UInt8 | Per-pixel accuracy value. |
| Payload Size | ~37 KB | Per 128 x 96 frame (after downsampling). |
| Throughput | ~0.75 MB/s | Estimated stream rate at 15 FPS after base64 overhead. |
Sources: integrations/iphone-lidar/README.md:52-63
Relay Protocol
Communication occurs over WebSockets. The development relay prints a random per-run access token for authentication.
Visualization and Fusion
The browser path serves as the primary receiver and visualization surface, rendering the point cloud using the metrics captured by the native sensor.
Web Interface Metrics
The index.html interface monitors real-time performance and sensor status:
- FPS: Frames per second of incoming LiDAR data.
- POINTS: Total number of geometry points in the current cloud.
- SEQ: Monotonic sequence number.
- LATENCY: Measurement in milliseconds (target is < 150ms p95 on local WiFi).
Sources: integrations/iphone-lidar/web/index.html:19-24, integrations/iphone-lidar/README.md:86-90
Sensor Fusion Integration
iPhone LiDAR geometry is intended for fusion with other RF modalities. This unified spatial model combines:
- Camera Depth (MiDaS)
- WiFi CSI (Human pose skeletons)
- mmWave Radar (Contactless vitals)
Sources: plugins/ruview/skills/ruview-applications/SKILL.md:27-29, harness/ruview/src/guidance.js:192-202
Validation and Acceptance
The integration includes a validation suite to ensure data integrity and privacy compliance.
- Native Source Validation: Type-checked against the iOS 17 SDK in CI with strict concurrency.
- Wire Decoding: Node.js tests cover decoding, malformed inputs, and relay authentication.
- Hardware Acceptance: Verified on physical devices by moving the phone through a room to ensure monotonic sequence increases and absence of RGB payloads in WebSocket frames.
Sources: integrations/iphone-lidar/README.md:73-90
Summary
iPhone LiDAR Capture provides RuView with high-fidelity geometry data while maintaining privacy through a geometry-only class. By downsampling data and using a compact UInt16 millimeter wire format, the system achieves efficient streaming suitable for local WiFi networks. This data integrates into the broader RuView ecosystem to enhance RF-based spatial perception with precise environmental modeling.