AI Agent Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The AI Agent Architecture in TalkCRM is designed around a voice-first, AI-native assistant that allows users to interact with Salesforce through natural language. Instead of mapping rigid commands to specific functions, the system employs a unified AI assistant pattern. This assistant, powered by Anthropic's Claude, interprets the user's intent from conversational input and orchestrates the necessary operations against the Salesforce API.
This architecture is built to be channel-agnostic, supporting interactions via voice (phone calls), text (SMS/WhatsApp), and web interfaces. The core logic is centralized in a Convex backend, which handles external API calls, real-time data synchronization, and serves as the intelligent layer between the user interface and Salesforce.
Sources: PROJECT_DESCRIPTION.md, README.md
High-Level System Architecture
The system processes user input through a series of services, starting with a voice or text interface, moving to an AI agent for interpretation, and finally executing actions against Salesforce. The Convex backend acts as the central hub for this entire process.
This diagram illustrates the flow of information from the sales representative to Salesforce and back, including the real-time dashboard updates.
Agent Configuration
The behavior of each AI agent is defined by a JSON configuration file. These files specify everything from the agent's personality and voice to the tools it can use. The system supports multiple agent configurations for different roles, such as a sales agent or a mindfulness coach.
Sources: agent_configs/TalkCRM.json, agent_configs/SIMPLE-Co-Sales-Agent.json, agent_configs/Mindfulness-coach.json
Conversation Configuration
The conversation_config object contains settings for the key components of the voice interaction.
| Section | Key | Description | Example Value |
|---|---|---|---|
| ASR | provider | The speech-to-text provider. | elevenlabs |
quality | The quality of transcription. | high | |
| TTS | voice_id | The specific ElevenLabs voice to use. | mHV5m7DLaQM0bIAP6BTK |
model_id | The text-to-speech model. | eleven_turbo_v2 | |
speed | The speaking speed of the agent. | 1.0 | |
| Turn | turn_timeout | Seconds to wait for user input before responding. | 10 |
turn_eagerness | How quickly the agent tries to take its turn. | normal | |
| Agent | first_message | The initial greeting spoken by the agent. | "Hey! I'm TalkCRM..." |
language | The primary language of the agent. | en |
Sources: agent_configs/TalkCRM.json:3-40, agent_configs/SIMPLE-Co-Sales-Agent.json:3-84
Agent Prompt and Personality
The core of the agent's behavior is dictated by the prompt within the agent configuration. This system prompt defines the agent's personality, goals, available tools, and rules of engagement.
For example, the TalkCRM.json agent is defined as a "high-energy voice assistant" and "hype partner" for sales reps. Its prompt includes:
- Mindset: "You are their hype partner AND their CRM assistant."
- Tools: A list of available Salesforce tools like
search_salesforceandupdate_record. - Common Flows: Examples of how to handle typical user requests, like "What's in my pipeline?".
- Affirmations: Phrases to weave into the conversation, such as "You've got this."
- Error Handling: Instructions on how to respond to failed searches or updates.
// agent_configs/TalkCRM.json:44-46
"prompt": {
"prompt": " You are TalkCRM, a high-energy voice assistant that empowers sales reps to dominate their Salesforce CRM hands-free. You believe in your rep. Every call is an opportunity. Every deal is winnable.\n\n Your Mindset...",
"llm": "glm-45-air-fp8",
Sources: agent_configs/TalkCRM.json:44-89
The architecture also supports channel-specific behavior. The SIMPLE-Co-Sales-Agent.json configuration includes a Channel Awareness section in its prompt, providing different instructions for voice calls versus WhatsApp messages.
Sources: agent_configs/SIMPLE-Co-Sales-Agent.json:20-38
The AI Brain: Claude Integration
The central intelligence of the agent is the askSalesforce Convex action, which uses Anthropic's Claude AI to interpret natural language and decide on a course of action.
Sources: convex/ai.ts:121-163
Intent Interpretation Flow
When a user speaks, the transcribed message is sent to the /tools/assistant HTTP endpoint, which triggers the askSalesforce action. This action then orchestrates the interpretation and execution process.
This sequence diagram shows the flow for a user request like "Find the Acme account".
Sources: convex/http.ts:90-148, convex/ai.ts:121-200
Salesforce Schema Context
To enable the AI to generate accurate Salesforce operations (especially SOQL queries), it is provided with a detailed system prompt containing schema information. The SALESFORCE_SCHEMA_CONTEXT constant in convex/ai.ts defines standard Salesforce objects, their fields, and rules for querying.
More importantly, the system dynamically fetches custom object metadata for the user's specific Salesforce org. The askSalesforce action calls ctx.runQuery(internal.orgMetadata.getAvailableObjects, ...) to retrieve custom objects, their fields, picklist values, and relationship names. This rich, dynamic context allows the AI to work with custom Salesforce configurations without being explicitly pre-programmed for them.
// convex/ai.ts:11-118
const SALESFORCE_SCHEMA_CONTEXT = `
You are a Salesforce assistant helping a sales rep via voice. You have access to these Salesforce objects and their key fields:
STANDARD OBJECTS:
- Account: Id, Name, Type, Industry, AnnualRevenue, Phone, Website, BillingCity, BillingState, OwnerId
- Contact: Id, Name, FirstName, LastName, Email, Phone, AccountId, Title, OwnerId
...
`;
// convex/ai.ts:173-176
const objectDescriptions = orgMetadata.customObjects.map((obj: any) => {
let desc = `\n### ${obj.name} (${obj.label})`;
// ... builds a detailed description of the custom object and its fields
Sources: convex/ai.ts:11-118, 166-248, README.md
Intent Parsing and Execution
The interpretWithClaude function sends the user's message and conversation history to the Claude API. The system prompt instructs Claude to respond with a JSON object matching the ParsedIntent interface.
// convex/ai.ts:111-119
interface ParsedIntent {
action: "search" | "query" | "create" | "update" | "log_call" | "clarify";
objectType?: string;
recordId?: string;
searchTerm?: string;
soql?: string;
fields?: Record<string, any>;
clarificationQuestion?: string;
response: string;
followUp?: string; // Suggested next action for the user
}
Sources: convex/ai.ts:111-119
The askSalesforce action then uses a switch statement to execute the appropriate Salesforce action based on the action field returned by Claude.
| Action | Description |
|---|---|
search | Finds records by name or keyword. Uses api.salesforce.searchRecords. |
query | Runs a specific SOQL query. Also uses api.salesforce.searchRecords. |
create | Creates a new record (e.g., Task, Lead). Uses api.salesforce.createRecord. |
update | Updates fields on an existing record. Uses api.salesforce.updateRecord. |
log_call | Logs a call activity. Uses api.salesforce.logCall. |
clarify | Asks the user a clarifying question if the intent is ambiguous. |
Sources: convex/ai.ts:112, convex/ai.ts:203-317
Tooling and Channel Integration
The AI agent is designed to be invoked from multiple channels, with all requests funneling through the centralized askSalesforce action in the Convex backend.
Tool Definitions
The agent's capabilities are exposed as "tools". In the agent_configs/TalkCRM.json file, a single tool is defined which points to the primary AI assistant endpoint.
// agent_configs/TalkCRM.json:93-95
"tool_ids": [
"tool_3601kc84mz3rehct87bkqyb4x4b4"
]
The README.md and PROJECT_DESCRIPTION.md files list the conceptual tools available to the agent, which are implemented as different action types within the askSalesforce logic.
| Tool / Action | Description |
|---|---|
search_salesforce | Find accounts, contacts, opportunities, leads |
get_record | Get full details of a specific record |
create_record | Create new accounts, contacts, leads, tasks |
update_record | Update record fields (stage, status, amount) |
log_call | Log call activity to Salesforce |
get_my_tasks | View open tasks and to-dos |
get_my_pipeline | View opportunities and pipeline summary |
Sources: README.md, agent_configs/TalkCRM.json:93-95
Multi-Channel Support
The architecture is designed to handle input from various sources, each routing to the same core AI logic.
This diagram shows how different channels all trigger HTTP actions in the Convex backend, which then invoke the same askSalesforce AI core.
- Voice: The
/webhooks/twilio/incomingendpoint receives calls and connects them to ElevenLabs. ElevenLabs then calls the/tools/assistantendpoint. - Text (SMS): The
/webhooks/sendblue/incomingendpoint receives text messages and triggersinternal.sendblue.processIncomingText, which eventually callsapi.ai.askSalesforce. - Slack: The
/webhooks/slack/commandsendpoint handles/crmslash commands, and the/webhooks/slack/eventsendpoint handles@TalkCRMmentions. Both route tointernal.slackCommands.handleAIQuery, which callsapi.ai.askSalesforce.
Sources: convex/http.ts:803-868, 1269-1364, convex/slackCommands.ts:317-331, README.md
Summary
The TalkCRM AI Agent Architecture is a robust and flexible system centered around a single, intelligent action (askSalesforce). By combining detailed, dynamic system prompts with a powerful LLM (Claude), it can interpret natural language and perform a wide range of Salesforce operations. Its channel-agnostic design, with specific webhooks routing to a common backend logic, allows it to serve users across voice, text, and chat platforms, providing a consistent and powerful interface to their CRM data.