LLM Integration (AI SDK)
Relevant source files
The following files were used as context for generating this wiki page:
The LLM Integration in Bolt leverages the Vercel AI SDK to provide a seamless interface between the user interface and Large Language Models. Its primary purpose is to facilitate the generation of full-stack web applications by orchestrating prompts, managing streaming responses, and parsing specialized XML-like tags (Artifacts and Actions) that trigger system-level operations like file creation or terminal commands.
Currently, the integration is optimized for Anthropic's Claude 3.5 Sonnet model, though the underlying architecture uses the provider-agnostic AI SDK to allow for potential expansion. The system handles complex developer workflows by including local file modifications in the LLM context and executing generated code within a sandboxed WebContainer environment.
Sources: CONTRIBUTING.md:1-20, README.md:1-25, package.json:52-60
Core Architecture and Data Flow
The integration follows a reactive flow where user input is enriched, sent to an API endpoint, and the resulting stream is processed in real-time.
LLM Communication Loop
- Input Collection: User prompts are captured via the
BaseChatcomponent. - Context Augmentation: Before sending, the system checks for unsaved files or manual modifications. These are converted into a unified diff format and appended to the prompt to ensure the LLM has the latest project state.
- Request Execution: The
useChathook fromai/reactsends the message history to the/api/chatendpoint. - Streaming & Parsing: As the LLM streams the response, a specialized
StreamingMessageParserscans for<boltArtifact>and<boltAction>tags. - Side Effect Execution: Parsed actions are dispatched to the
WorkbenchStore, which interacts with the WebContainer API to modify the filesystem or run shell commands.
Component Breakdown
Vercel AI SDK Integration
The project utilizes the ai package (v5.0.52) and the @ai-sdk/anthropic provider. The frontend logic is primarily contained within the ChatImpl component using the useChat hook.
| Dependency | Version | Role |
|---|---|---|
ai | ^5.0.52 | Core stream handling and React hooks |
@ai-sdk/anthropic | ^0.0.39 | Anthropic model provider |
useChat | N/A | Manages message state and API calls |
Sources: package.json:52-60, app/components/chat/Chat.client.tsx:75-85
Streaming Message Parser
The StreamingMessageParser is a critical stateful class that processes raw text from the LLM. It identifies and strips out control tags so they don't clutter the UI while triggering necessary callbacks.
- Artifacts: Represent a high-level task or container (e.g., "Installing Dependencies").
- Actions: Specific operations within an artifact, such as
file(writing code) orshell(running commands).
API and Data Models
Message States
The system maintains the state of each message as it is being parsed to handle chunks that might split a tag across multiple stream packets.
interface MessageState {
position: number; // Current index in the input string
insideArtifact: boolean; // Flag for active <boltArtifact>
insideAction: boolean; // Flag for active <boltAction>
currentArtifact?: BoltArtifactData;
currentAction: BoltActionData;
actionId: number; // Sequence counter for actions
}
Sources: app/lib/runtime/message-parser.ts:41-48
Prompt Enhancement
A usePromptEnhancer hook (referenced in chat logic) allows for refining user queries before they are sent to the LLM, ensuring higher-quality code generation.
Sources: app/components/chat/Chat.client.tsx:87, app/components/chat/BaseChat.tsx:145-165
Integration with Filesystem
The LLM integration is "filesystem-aware." When a user sends a message, the Chat.client.tsx logic executes workbenchStore.saveAllFiles() and computes modifications via fileModificationsToHTML. This ensures the LLM receives a context-rich prompt containing:
- The user's text instructions.
- A unified diff of all changes made since the last turn.
Sources: app/components/chat/Chat.client.tsx:156-175, app/utils/diff.ts:14-45
Conclusion
The LLM Integration via AI SDK transforms the chat interface into a powerful IDE agent. By combining the streaming capabilities of Claude 3.5 Sonnet with a robust XML-based action protocol, Bolt can perform complex orchestration of web development tasks directly in the browser environment.