LLM Service and Quix Agent

Relevant source files

The following files were used as context for generating this wiki page:

• src/llm/llm.service.ts
• src/llm/quix-agent.ts
• src/llm/llm.provider.ts
• src/llm/schema.ts
• src/lib/constants.ts
• src/slack/slack-events-handler.service.ts
• src/llm/agent-evals/jira-agent/index.spec.ts

The LLM Service and the QuixAgent are the core components of the AI-powered functionality within the Quix Slack agent. They work together to understand user queries from Slack, determine the appropriate actions, interact with integrated business tools (like Jira, GitHub, HubSpot), and generate a final response. This system leverages Large Language Models (LLMs) to orchestrate a series of tool calls based on a dynamically generated plan.

The overall process is initiated by a user message in Slack, which is handled by the SlackEventsHandlerService. This service then invokes the LlmService to begin the AI-driven processing. The QuixAgent contains the primary logic for tool selection, planning, and execution, making it the central "brain" of the agent.

Sources: README.md, src/slack/slack-events-handler.service.ts:132-144, src/llm/quix-agent.ts

Architecture and Data Flow

The system follows a multi-step process to handle a user's request. It begins with a Slack event, flows through the LlmService and QuixAgent, and results in a response posted back to Slack.

The following sequence diagram illustrates the high-level interaction between the key components when a user sends a message.

This diagram shows the flow from a user message to the final response, highlighting the distinct phases of tool selection, planning, and execution managed by the QuixAgent.

Sources: src/slack/slack-events-handler.service.ts:189-211, src/llm/quix-agent.ts:25-40

LlmService

The LlmService acts as the primary entry point for handling incoming messages that require LLM processing. It is responsible for preparing the necessary context and invoking the QuixAgent.

Its main method, processMessage, is called by the SlackEventsHandlerService when a new message or app mention is received. This method orchestrates the agent's execution by:

1. Retrieving the appropriate LLM provider (e.g., OpenAI).
2. Gathering all available and configured tools for the workspace.
3. Invoking the QuixAgent.processWithTools method with the user's message and the prepared context.
4. Returning the final, user-facing response generated by the agent.

Sources: src/slack/slack-events-handler.service.ts:132-144, src/llm/llm.service.ts

QuixAgent

The QuixAgent class contains the core logic for interpreting and acting upon user requests. It uses a ReAct (Reasoning and Acting) framework, facilitated by LangGraph, to break down a query into a series of thoughts and actions.

Sources: src/llm/quix-agent.ts:19

processWithTools Method

This is the main public method of the QuixAgent. It orchestrates the entire agentic workflow, which consists of three main stages: Tool Selection, Plan Generation, and Agent Execution.

The following flowchart illustrates the logic within the processWithTools method.

1. Tool Selection

Before executing any actions, the agent first determines which categories of tools are relevant to the user's query. This is handled by the toolSelection method.

1. Prompt Construction: A system prompt is built using QuixPrompts.basePrompt and supplemented with specific toolSelection prompts from each available tool package (e.g., Jira, Slack).
2. Function Calling: The agent uses a DynamicStructuredTool named selectTool. The LLM is instructed to call this function, which forces it to output a structured response containing the selected toolCategories and a reason.
3. Output: The method returns the list of selected tool categories (e.g., ['jira', 'slack']) or 'none'. If no tools are needed, the reason field is expected to contain a direct answer to the user's query.

The schema for the selectTool function is defined as follows:

// From src/llm/quix-agent.ts:150-162
new DynamicStructuredTool({
  name: 'selectTool',
  description: QuixPrompts.baseToolSelection,
  schema: z.object({
    toolCategories: z.array(z.enum(availableCategories as [string, ...string[]])),
    reason: z
      .string()
      .describe(
        "An explanation of why the selected tool categories were chosen. If no tools were selected, this must include a direct answer to the user's query using general knowledge."
      )
  }),
  // ...
});

If toolSelection returns 'none', the agent execution stops, and the content from the reason field is returned to the user.

Sources: src/llm/quix-agent.ts:105-189, src/lib/constants.ts:178-184

2. Plan Generation

Once the necessary tools are identified, the generatePlan method creates a multi-step execution plan.

1. Prompting: The LLM is prompted with QuixPrompts.PLANNER_PROMPT, which instructs it to break down the user's request into an ordered list of tool calls. This prompt includes the schemas of all available functions from the selected tool categories.
2. Structured Output: The LLM is constrained to produce a JSON output that matches the PlanStepSchema. This ensures the plan is machine-readable and directly executable.
3. Output: The method returns an array of plan steps.

The structure of the plan is defined by PlanStepSchema:

Field	Type	Description
steps	array	An array of objects, where each object is a single step.
type	string	The type of step, currently supporting "tool".
tool	string	The name of the tool to call (e.g., jira_create_issue).
args	object	The arguments to pass to the tool.

Sources: src/llm/quix-agent.ts:191-235, src/llm/schema.ts:4-13, src/lib/constants.ts:185-220

3. Agent Execution

With a plan in hand, the agent proceeds to execute it.

1. Agent Creation: A ReAct agent is created using createReactAgent from @langchain/langgraph/prebuilt. This agent is provided with the LLM, the available tools, and a custom prompt (QuixPrompts.multiStepBasePrompt) that includes the generated plan.
2. Invocation: The agent is invoked with the conversation history and the user's query.
3. Execution Loop: The LangGraph agent enters a loop where it:
   • Decides which tool to call next based on the plan.
   • Executes the tool.
   • Observes the result.
   • Repeats until the plan is complete or it decides to stop.
4. Callback Management: A QuixCallBackManager is used to capture all tool calls made during the execution, which can be used for logging and debugging.

The final output of the agent execution is then returned as the result of the processWithTools method.

Sources: src/llm/quix-agent.ts:83-102, src/lib/constants.ts:149-176

Key Prompts

The agent's behavior is heavily guided by a set of predefined prompts stored in src/lib/constants.ts.

Prompt Name	Description
basePrompt	Sets the agent's persona as "Quix" and provides general instructions, such as how to handle support requests and bug reports. It also emphasizes using tools and not making up information.
multiStepBasePrompt	Used during agent execution. It instructs the agent to follow a specific plan and any custom instructions provided by the tool configurations.
baseToolSelection	A description for the selectTool function, guiding the LLM on how to select tool categories and provide a reason.
PLANNER_PROMPT	Instructs the LLM to act as a planner and break down a request into a structured JSON plan of tool calls. It emphasizes following custom instructions and including all required parameters for tools.

Sources: src/lib/constants.ts:133-220

Testing

The QuixAgent is tested using a combination of a real LLM and mocked tools. The tests, found in files like src/llm/agent-evals/jira-agent/index.spec.ts, follow a consistent pattern:

1. Define a series of test cases, each with a user query, chat history, and expected tool calls.
2. Initialize the QuixAgent and a set of mocked tools that return predefined data.
3. Invoke agent.processWithTools for each test case.
4. Use createTrajectoryMatchEvaluator from the agentevals library to compare the actual tool calls made by the agent against the reference tool calls.
5. Assert that the agent's trajectory matches the expected behavior.

This approach validates that the agent can correctly interpret user intent, select the right tools, and call them with the correct arguments.

Sources: src/llm/agent-evals/jira-agent/index.spec.ts, src/llm/agent-evals/slack-agent/index.spec.ts