High-Level Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The Nudge (OpenLive) project is designed as a multi-tier learning assistant platform that leverages on-device AI for voice interactions and delegated agentic loops for complex tasks like web research. The system is structured to preserve student ownership by providing step-by-step guidance rather than direct answers, utilizing a "Signal Layer" to detect struggle and a "Policy Layer" to manage interventions.
The architecture is implemented as a monorepo containing a desktop shell (Electron), a web-based user interface (Next.js), an agent service for model orchestration, and various shared packages for data persistence and model hardware abstraction.
Sources: CONTRIBUTING.md:21-28, website/src/renderer.js:154-171
System Component Overview
The project is divided into specialized modules that handle specific concerns from low-level model communication to high-level UI rendering.
| Component | Responsibility |
|---|---|
| apps/desktop | Electron shell managing local servers, system permissions, and auto-updates. |
| apps/web | Next.js frontend, including the on-device voice engine (VAD, STT, TTS). |
| services/agent | WebSocket server managing the /live session and model tool execution. |
| packages/harness | Model adapters for Anthropic and OpenAI, including prompt caching logic. |
| packages/shared | Wire protocol definitions and shared TypeScript types. |
| packages/db | JSON-based storage for settings, API keys, and conversation history. |
Sources: CONTRIBUTING.md:21-28, apps/desktop/electron-builder.yml:14-19
Technical Topology
The following diagram illustrates the relationship between the desktop shell, the local services, and external AI providers.
The desktop application encapsulates both the UI and the agent services, communicating with external LLM providers through the harness package.
Sources: CONTRIBUTING.md:21-28, apps/web/next.config.ts:16-20
Voice Interaction and Tool Execution
The core experience revolves around a "live" voice loop that includes Voice Activity Detection (VAD), Speech-to-Text (STT), and Text-to-Speech (TTS). The system uses a delegation model where a "Main Agent" handles the voice conversation and a "Worker Subagent" performs intensive web-based tasks to minimize latency for the user.
Delegation Logic
When a complex task is required (e.g., searching the web), the main agent delegates to a worker subagent. This allows the main agent to maintain the conversation while the worker executes tools like web_search and fetch_url.
The sequence demonstrates how the system maintains responsiveness by providing verbal feedback ("Let me look that up") while the background worker processes data.
Sources: services/agent/src/live/worker.ts:47-75, services/agent/src/tools.ts:107-118
Tool Ecosystem
Tools are defined using Zod schemas to ensure type safety and consistent JSON Schema generation for LLM consumption.
- Main Agent Tools: Includes
delegate,update_todos(UI checklist), andremember(persistent memory). - Worker Tools: Includes
web_search(via Exa) andfetch_url(with SSRF protection and HTML-to-text conversion).
Sources: services/agent/src/tools.ts:60-105, services/agent/src/tools.ts:121-127
Data Flow and Model Adaptation
The packages/harness module acts as a translation layer between the project's internal message format and provider-specific APIs (Anthropic, OpenAI).
Anthropic Provider Implementation
The Anthropic adapter (anthropic.ts) implements aggressive prompt caching to reduce latency and costs. It marks the system prompt, tool definitions, and the most recent message prefix with cache_control: { type: "ephemeral" }.
// Example of Anthropic cache breakpoint implementation
if (!noCacheControl) {
const last = out[out.length - 1];
const content = last?.content;
if (Array.isArray(content) && content.length) {
const block = content[content.length - 1] as Record<string, unknown>;
block.cache_control = { type: "ephemeral" };
}
}
Sources: packages/harness/src/anthropic.ts:50-59, packages/harness/src/anthropic.ts:88-93
Security and Network Configuration
The application implements a strict Content Security Policy (CSP) to manage on-device AI model downloads and WebSocket connections.
- Model Hosts: Weights are downloaded from
huggingface.coandhf.co. - Wasm Execution: The
onnxruntime-webruntime requireswasm-unsafe-evalfor WebAssembly instantiation. - SSRF Protection: The
fetch_urltool includes a guard (hostIsPrivate) that blocks loopback, private, and cloud metadata IP addresses (e.g.,127.0.0.1,169.254.169.254).
Sources: apps/web/next.config.ts:13-33, services/agent/src/tools.ts:43-56
UI and Frontend Architecture
The user interface, built with Next.js and Tailwind CSS, focuses on a "calm" experience. A key component is the TranscriptPanel, which renders the conversation history, reasoning blocks, and tool execution status.
Transcript Rendering
The transcript groups consecutive reasoning and tool parts into "work" blocks. These blocks are interactive, allowing users to see the underlying logic and tool summaries (e.g., "Searching for...", "Reading website...").
Summary
The Nudge architecture is a sophisticated blend of local desktop capabilities and cloud-based AI orchestration. By offloading complex research to a background worker and utilizing aggressive caching at the harness layer, the system achieves the low-latency response times necessary for a natural voice interface while maintaining a secure environment through SSRF guards and strict CSPs.