Streaming & OBS Composition
Relevant source files
The following files were used as context for generating this wiki page:
The Streaming & OBS Composition system provides a specialized architecture for integrating J.A.R.V.I.S. visual elements and hand-tracking interfaces into professional broadcast software like OBS. It relies on a "Split Role" architecture where a hidden tracker process manages heavy computation (like camera processing and physics) while a separate, lightweight render page handles high-fidelity visual output with true alpha transparency for seamless compositing.
Sources: backend/src/barehands/stage.html:1506-1512, backend/src/barehands/sample-notes/Field Guide/Streaming and Recording.md
Split-Role Architecture
To overcome technical limitations in broadcast software (such as OBS browser sources being unable to access cameras on certain operating systems), the system implements a two-page synchronization model.
- The Tracker (
role=stage): This is the primary control interface. It opens the user's camera, runs the MediaPipe hand-tracking landmarker, and calculates object physics. It streams normalized scene state to a local endpoint at approximately 45Hz. - The Render (
role=render): This page is intended to be loaded as an OBS Browser Source. It is "camera-free" and purely mirrors the state provided by the tracker. It uses CSS transparency to provide a true alpha channel for overlays.
Sources: backend/src/barehands/stage.html:1506-1519, backend/src/barehands/stage.html:1571-1577
Data Synchronization Flow
The synchronization between the tracker and the OBS render source occurs via window-fractional coordinates, ensuring visual consistency regardless of the differences in window sizes between the two roles.
Visual Composition & "The Glass Era"
The broadcast aesthetic, referred to as the "Glass Era," focuses on high-transparency UI elements that remain readable over live video through specific graphical techniques.
High-Fidelity Rendering Techniques
- True Transparency: Render pages use
background: transparentandon-airCSS classes to strip solid backdrops, relying on high-contrast "halo" text shadows and luminous edges for legibility. - 2X Supersampling (
ss=2): To prevent pixelation in OBS (which often rasterizes DOM elements once at layout size), the system supports a body zoom multiplier. A 3840x2160 browser source withss=2ensures that cards and glass gradients remain sharp even when scaled. - Mirror Doctrine: While the tracker operates in a mirrored "selfie" view for the presenter, the OBS render page automatically flips the X-axis (
RFLIP) to match the standard un-mirrored broadcast perspective.
Sources: backend/src/barehands/stage.html:43-50, backend/src/barehands/stage.html:1571-1582, src/renderer.css:1010-1018
Key Elements of the OBS Overlay
| Element | Description | Broadcast Treatment |
|---|---|---|
| The Ring | Core assistant presence indicator. | Pulses white-hot based on live voice waveform; static ticks. |
| Glass Orbs | Tap targets for navigation. | Floating satellites with constant clearance from the Ring's edge. |
| 3D Holograms | GLB/glTF models. | Rendered in local Three.js scenes on both pages; supports fresnel rim shaders. |
| Browser Panes | Indented file trees. | High-alpha pulse animation on tap for clear visual feedback. |
| Cursors | Hand position indicators. | Optional green rings; hidden via cursors=0 for "maximum sorcery" effect. |
Sources: backend/src/barehands/stage.html:232-243, backend/src/barehands/stage.html:430-440, backend/src/barehands/stage.html:1571-1582
Integration Parameters
The system supports several URL query parameters to tune the broadcast output for specific streaming requirements.
| Parameter | Type | Default | Description |
|---|---|---|---|
mode | string | mirror | Set to overlay for transparency or key for solid chroma colors. |
key | string | magenta | Defines the chroma key color (magenta, green, blue, or hex). |
role | string | stage | Use render for the camera-free OBS browser source. |
ss | number | 1 | Supersampling scale for high-resolution displays. |
mirror | boolean | 0 | Set to 1 to disable the automatic broadcast X-flip. |
cursors | boolean | 1 | Set to 0 to hide tracking rings in the stream. |
Sources: backend/src/barehands/stage.html:164-184, backend/src/barehands/stage.html:1571-1582
3D Hologram Processing
The system handles 3D models specifically for streaming through "Glow Lift" and "Adaptive Resolution" to ensure visibility against varied video backgrounds.
Conclusion
The Streaming & OBS Composition feature transforms J.A.R.V.I.S. from a local assistant into a powerful broadcast tool. By separating the tracking logic from the rendering process, the project provides a high-performance, low-latency overlay system that maintains professional visual quality through supersampling and advanced 3D shaders, specifically optimized for the constraints of modern streaming environments.