Voice Notes Integration
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Voice Notes Integration in SweepBot serves as a multi-modal input system designed to enhance user productivity and session documentation. Its primary purpose is twofold: capturing session-specific auditory annotations for gameplay record-keeping and providing a hands-free interface for the NLP Flow Engine. By leveraging the browser's MediaRecorder API, users can record voice notes that are subsequently processed as either session metadata or executable automation commands.
The integration spans the browser extension and the web dashboard, allowing for seamless capture during active sweepstakes play or retrospective session review. Within the broader ecosystem, voice notes contribute to the "Data Moat" by providing qualitative context to quantitative session data like RTP and volatility.
Sources: AGENTS.md:121-125, DATABASE_LAYER_COMPLETE.md:107-111, QUICK_WINS_CHECKLIST.md:126-128
Architecture and Data Flow
The architecture follows a distributed capture model where recording logic is implemented in both the browser extension (for real-time input) and the web application (for session management).
Voice Capture Components
| Component | Location | Responsibility |
|---|---|---|
voice-recorder.ts | apps/extension/src/lib/ | Provides low-level voice input for flow commands. |
VoiceNoteRecorder.tsx | apps/web/src/components/sessions/ | UI component for recording notes during session review. |
| MediaRecorder API | Browser | Standardized API used to capture audio streams from the user's microphone. |
Sources: AGENTS.md:121-123, DATABASE_LAYER_COMPLETE.md:83-84, QUICK_WINS_CHECKLIST.md:126-128
Capture and Storage Flow
The following diagram illustrates the sequence from audio capture to persistent storage in the database.
Voice-Driven Automation (Flows)
A critical sub-module of the voice integration is the translation of audio input into executable automation scripts via the NLP Flow Engine. This allows users to "speak" their automation strategies, which the EntityRecognizer then parses for platforms, games, and schedules.
The voice integration specifically feeds into the FlowInterpreter pipeline to convert natural language into a 9-node Abstract Syntax Tree (AST).
Sources: packages/flows/README.md:1-12, AGENTS.md:121-125
Data Model and Persistence
Voice notes are persisted in the public schema within the session_notes and related tables. The schema supports linking audio data to specific gameplay sessions and user profiles.
Database Schema: Session Voice Notes
| Field | Type | Description |
|---|---|---|
id | UUID | Primary key for the voice note. |
session_id | UUID | Foreign key linking to the sessions table. |
user_id | UUID | Foreign key linking to the profiles table. |
audio_url | Text | Storage URL for the audio file (Supabase Storage). |
transcript | Text | Textual representation of the voice note. |
metadata | JSONB | Inferred metadata including duration and file format. |
Sources: DATABASE_LAYER_COMPLETE.md:12-25, DATABASE_LAYER_COMPLETE.md:65-68
Implementation Status
The voice notes feature is currently part of the "Quick Wins" implementation phase, with specific components integrated into the API and Extension.
- Extension Support:
voice-recorder.tsexists but is currently undocumented/internal for flow commands. - API Routes: Endpoints are defined for
POST /sessions/:id/voice-notesandGET /sessions/:id/voice-notes. - Frontend UI: The
VoiceNoteRecordercomponent is utilized in the session review UI to allow manual note-taking.
Sources: AGENTS.md:121-125, DATABASE_LAYER_COMPLETE.md:83-84, QUICK_WINS_CHECKLIST.md:126-128
Conclusion
Voice Notes Integration bridges the gap between manual session documentation and automated flow creation. By providing a unified interface for audio capture in both the extension and web apps, SweepBot enables users to maintain detailed records and create complex automations through simple voice commands. This integration is vital for the platform's goal of reducing the friction associated with tracking multi-platform sweepstakes gameplay.
Sources: AGENTS.md:12-18, packages/flows/README.md:1-5