Customizing Tools
Relevant source files
The following files were used as context for generating this wiki page:
Tools in this project are modular components that allow the AI agent to interact with external systems, perform research, manage state, and update the user interface. The system distinguishes between "Main Agent" tools, which handle high-level interaction and delegation, and "Worker" tools, which perform specialized technical tasks like web searching and URL fetching.
The customization of tools involves defining their metadata, parameter schemas using Zod, and execution logic that often involves streaming progress events (SSE) back to the client. This architecture ensures the main voice agent remains responsive while a delegated subagent performs heavier computational or network-bound tasks.
Sources: services/agent/src/tools.ts:1-15, services/agent/src/live/worker.ts:40-45
Tool Architecture and Types
All tools implement a standard interface that defines their identity and behavior. The system uses Zod to define parameter shapes, which are then converted to JSON Schema for model consumption.
Core Interfaces
| Property | Type | Description |
|---|---|---|
name | string | The unique identifier for the tool. |
description | string | Instructions for the LLM on when and how to use the tool. |
parameters | Record<string, unknown> | JSON Schema describing the expected arguments. |
execute | (args: any) => Promise<ToolResult> | The asynchronous logic that performs the tool's action. |
Sources: services/agent/src/tools.ts:14-23
Tool Execution Flow
The following diagram illustrates how the LiveTurnRunner interacts with the tool registry and execution environment.
The LiveTurnRunner manages the loop of sending tool results back to the model until a final text response is generated or the maximum step limit (MAX_STEPS = 6) is reached.
Sources: services/agent/src/live/turn-runner.ts:192-220
Categorization: Main vs. Worker Tools
The system separates tools based on the agent's role. This prevents the main voice agent's context from becoming bloated with raw technical data.
Main Agent Tools
The main agent focuses on conversation and high-level task management.
- delegate: Hands off complex tasks to a subagent.
- update_todos: Manages a UI-visible checklist.
- remember: Persists facts to a JSON-based memory store.
Worker Subagent Tools
Workers handle the "grind" of technical execution.
- web_search: Uses the Exa API to find information.
- fetch_url: Retrieves and cleans HTML content from public URLs.
Sources: services/agent/src/tools.ts:133-145, services/agent/src/live/worker.ts:40-45
Defining New Tools
To customize or add a tool, developers create factory functions that accept an Emit callback. This allows the tool to report status directly to the UI during long-running operations.
Parameter Schema Definition
Parameters are defined using zod-to-json-schema. The project utilizes a helper function to strip unnecessary metadata for compatibility with various model providers.
function params(shape: ZodRawShape): Record<string, unknown> {
const js = zodToJsonSchema(z.object(shape), { $refStrategy: "none" }) as Record<string, unknown>;
delete js.$schema;
return js;
}
Sources: services/agent/src/tools.ts:43-47
Example: Tool Implementation Pattern
The remember tool demonstrates how to implement persistent state updates within a tool.
function makeRemember(emit: Emit): TaktTool {
return {
name: "remember",
description: "Save a short fact worth keeping across turns...",
parameters: params({ note: z.string().describe("The fact to remember") }),
execute: async (args) => {
const note = String(args.note ?? "").trim().slice(0, 240);
const id = randomUUID();
await emit({ type: "tool_start", id, tool: "remember", summary: note });
// Logic for saving to @openlive/db
await emit({ type: "tool_done", id, detail: "saved" });
return { output: "Got it — I'll remember that." };
},
};
}
Sources: services/agent/src/tools.ts:106-123
Tool Visualization and Metadata
For a tool to be properly represented in the frontend transcript and status lines, it must be registered in the TOOL_META configuration. This maps tool names to human-readable labels, active state descriptions, and icons.
| Tool Name | Label | Active Description | Icon |
|---|---|---|---|
web_search | Searched the web | Searching the web | Search |
fetch_url | Read a page | Reading a page | Globe |
remember | Saved a note | Saving a note | Bookmark |
update_todos | Updated the plan | Planning | ListTodo |
look | Took a look | Looking | Eye |
Sources: apps/web/src/lib/live/toolMeta.ts:4-15
Delegation and Worker Coordination
The delegate tool is unique as it triggers a nested agentic loop. The runWorker function manages a fresh subagent that uses WORKER_PROMPT and has access to web tools.
If a worker tool takes longer than NARRATE_AFTER_MS (1500ms), the system automatically generates natural narration deltas (e.g., "Reading through [hostname] now") so the voice interface does not remain silent during background tasks.
Sources: services/agent/src/live/worker.ts:12-40, services/agent/src/live/turn-runner.ts:133-145
Provider-Specific Handling
Tool definitions are transformed based on the model provider's specific API requirements.
- Anthropic: Tools are sent in the
toolsarray withinput_schema. The project utilizes prompt caching by marking the last tool in the list withcache_control: { type: "ephemeral" }. - OpenAI Responses: Tools are flattened into a
functiontype within thetoolsarray, requiringname,description, andparameters. - Ollama/Vision Models: Certain models (specifically vision/VL models) are excluded from tool usage as they may reject the
toolsparameter with HTTP 400 errors.
Sources: packages/harness/src/anthropic.ts:63-71, packages/harness/src/openai-responses.ts:34-37, services/agent/src/live/turn-runner.ts:25-30