Tool Calling & Structured Output
Relevant source files
The following files were used as context for generating this wiki page:
Tool Calling and Structured Output provide mechanisms for Large Language Models (LLMs) to interact with external tools and return data in specific formats. These features reside in the C++ commons core (core/), ensuring consistent behavior across all five platform SDKs. The system manages the lifecycle of tool requests, argument validation, and JSON schema enforcement to bridge the gap between unstructured natural language and structured programmatic execution.
Sources: AGENTS.md:73-77, core/src/features/llm/tool_calling.cpp:9-15
Architecture and Data Flow
The architecture centralizes logic in the C++ core to prevent duplication across platform SDKs. A logical service contract, defined via Protobuf IDL, governs the parsing, prompt formatting, and validation of tool calls and structured outputs.
The diagram shows the iterative loop where LLM output is parsed, validated, and used to drive external execution or final synthesis.
Sources: idl/tool_calling.proto:237-241, core/src/features/llm/tool_calling.cpp:1645-1670
Tool Calling Mechanisms
Tool calling identifies when an LLM intends to use a specific function and extracts the required arguments. The system supports multiple tool call formats based on the model family.
Supported Formats
The parser automatically detects or forces specific formats to match model expectations:
| Format | Identifier | Description |
|---|---|---|
| DEFAULT | RAC_TOOL_FORMAT_DEFAULT | Standard JSON-in-tag format: <tool_call>{json}</tool_call> |
| LFM2 | RAC_TOOL_FORMAT_LFM2 | Liquid AI format: `< |
| Pythonic | RAC_TOOL_FORMAT_PYTHONIC | Bare bracketed format: [name(args)] used by grammar-constrained engines. |
Sources: core/src/features/llm/tool_calling.cpp:33-43, core/src/features/llm/tool_calling.cpp:89-105
Parsing Logic
The system extracts tool names and arguments from LLM output using several strategies. It attempts to find standard keys like "tool", "name", or "function" and wraps scalar or array results into a JSON object if necessary.
The flowchart represents the multi-stage parsing logic that handles both tagged JSON and bare Pythonic function calls.
Sources: core/src/features/llm/tool_calling.cpp:115-160, core/src/features/llm/tool_calling.cpp:737-780
Validation and Grounding
The system validates tool calls against a provided schema to ensure arguments match expected types (string, number, boolean, object, array). To prevent models from hallucinating data, the system appends a grounding rule to tool results: "State only values literally present in the tool result... never estimate, invent, or recall a plausible number."
Sources: core/src/features/llm/tool_calling.cpp:55-60, core/src/features/llm/tool_calling.cpp:1600-1640
Structured Output
Structured Output ensures the LLM returns data strictly conforming to a JSON schema. This is used for extracting entities or generating programmatic responses without natural language filler.
Data Structures
The system uses several Protobuf messages to manage schema enforcement and results:
StructuredOutputRequest: Contains the raw text to parse and the target JSON schema.StructuredOutputResult: Carries the parsed JSON, an indication of success, and any validation errors.StructuredOutputOptions: Configures strictness and formatting preferences.
Sources: idl/structured_output.proto:45-75
Schema Enforcement
The system supports both standard JSON schema validation and grammar-constrained generation. For grammar backends (like QHexRT), the system forces specific output shapes during the generation phase to guarantee validity.
Sources: core/src/features/llm/structured_output.cpp:120-150, idl/structured_output.proto:80-85
Configuration Options
Tool calling and structured output are governed by the ToolCallingOptions and StructuredOutputOptions messages.
| Option | Type | Description |
|---|---|---|
auto_execute | bool | If true, the SDK automatically runs the registered executor. |
tool_choice | enum | Controls if tool use is AUTO, REQUIRED, NONE, or SPECIFIC. |
parallel_tool_calls | bool | Allows a single turn to emit multiple tool call envelopes. |
strict | bool | Enforces exact schema adherence for structured outputs. |
max_tool_calls | int32 | Caps the number of calls in one turn (default is 5). |
Sources: idl/tool_calling.proto:170-220, idl/structured_output.proto:55-65
Implementation Details
The implementation includes utilities for manual JSON parsing and normalization to handle common LLM errors, such as unquoted keys or missing closing tags.
// Example of unquoted key repair in tool_calling.cpp
if (j < len && json_str[j] != '"' && json_str[j] != '{' && json_str[j] != '[') {
size_t key_start = j;
while (j < len && is_key_char(json_str[j])) { j++; }
if (j < len && json_str[j] == ':') {
result += '"';
result.append(json_str + key_start, key_end - key_start);
result += '"';
}
}
Sources: core/src/features/llm/tool_calling.cpp:640-665
Tool Calling and Structured Output serve as critical components for integrating LLMs into programmatic workflows. By centralizing detection, parsing, and validation in the C++ core, the system provides a robust and consistent interface across all platforms while minimizing model hallucinations through strict grounding and schema enforcement.