Web UI Artifact Rendering

Relevant source files

The following files were used as context for generating this wiki page:

• packages/web-ui/src/tools/artifacts/artifacts.ts
• packages/web-ui/src/tools/artifacts/artifacts-tool-renderer.ts
• packages/web-ui/src/tools/artifacts/ImageArtifact.ts
• packages/web-ui/src/tools/artifacts/PdfArtifact.ts
• packages/web-ui/src/tools/artifacts/HtmlArtifact.ts
• packages/web-ui/src/tools/artifacts/TextArtifact.ts
• packages/web-ui/src/prompts/prompts.ts
• packages/web-ui/src/utils/test-sessions.ts

The Web UI Artifact Rendering system enables an AI agent to create, manage, and display persistent files, known as artifacts, within the chat interface. This feature allows the agent to generate rich content like HTML applications, Markdown documents, images, and text files that live alongside the conversation. The system is composed of an ArtifactsPanel that hosts the rendered artifacts, an AgentTool for the AI to interact with, and a series of specialized ArtifactElement components for rendering different file types.

This system provides a powerful way for the AI to present complex information, create interactive visualizations, and persist data across multiple turns in a conversation. A key feature is the ability to render sandboxed HTML artifacts that can programmatically access other artifacts and user attachments, enabling the creation of dynamic data dashboards and tools.

Architecture Overview

The artifact system consists of several key components that work together to manage and render artifacts. The ArtifactsPanel is the central orchestrator, managing artifact state and hosting the UI. The ArtifactsToolRenderer handles the display of tool calls within the chat log, while various ArtifactElement subclasses are responsible for rendering the content of specific file types.

The following diagram illustrates the high-level architecture of the artifact rendering system.

ArtifactsPanel Component

The artifacts-panel custom element is the core of the artifact UI. It manages the lifecycle of all artifacts, provides the AgentTool for the AI, and renders the tabbed interface for viewing artifact content.

Sources: packages/web-ui/src/tools/artifacts/artifacts.ts

Responsibilities

• State Management: Maintains a map of all current artifacts, with the filename as the key and an Artifact object as the value.
  Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:46-47
• Tool Provider: Exposes an AgentTool named artifacts that the AI agent can call to perform CRUD operations on artifacts.
  Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:316-332
• Command Execution: Implements the logic for the create, update, rewrite, get, delete, and logs commands.
  Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:413-432
• Rendering Logic: Determines which ArtifactElement to use for a given file based on its extension via the getFileType method.
  Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:162-218
• UI Management: Renders the tab bar for switching between active artifacts and handles the display of the selected artifact's content.
  Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:589-628
• State Reconstruction: Can reconstruct the entire artifact state from a list of chat messages, ensuring consistency when a session is loaded.
  Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:335-411

Artifacts Tool

The artifacts tool allows the AI to manage files. It is defined by a JSON schema and an execute function within the ArtifactsPanel.

Tool Parameters

Parameter	Type	Description
command	string	The operation to perform (create, update, rewrite, get, delete, logs).
filename	string	The name of the file, including its extension.
content	string	Optional. The file content for create or rewrite.
old_str	string	Optional. The string to be replaced for update.
new_str	string	Optional. The replacement string for update.

Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:39-45, packages/web-ui/src/prompts/prompts.ts:75-83

Creation Flow

The following sequence diagram shows the process when an agent creates a new HTML artifact.

Artifact Rendering

Rendering is handled by two main parts: the ArtifactsToolRenderer for the chat log entry and the specific ArtifactElement subclass for the content within the ArtifactsPanel.

ArtifactsToolRenderer

This class is responsible for rendering the visual representation of an artifacts tool call within the message history.

• It displays the status of the tool call (e.g., "Creating artifact", "Updated artifact", "Error").
• It provides a collapsible section to view the details of the operation, such as the code being created or the diff for an update.
• For HTML artifacts, it can also display a console-block with logs or errors from the sandboxed execution.
• It uses a getLanguageFromFilename helper to apply correct syntax highlighting to code blocks.

Sources: packages/web-ui/src/tools/artifacts/artifacts-tool-renderer.ts

ArtifactElement and Implementations

ArtifactElement is the abstract base class for all artifact renderers. The ArtifactsPanel uses the getFileType method to select the appropriate concrete implementation based on the file extension.

Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:221-279, packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:20, packages/web-ui/src/tools/artifacts/TextArtifact.ts:45

File Type to Renderer Mapping

| File Type(s) - html - svg - md, markdown - png, jpg, jpeg, gif, webp, bmp, ico - pdf - xlsx, xls - docx - txt, json, xml, yaml, yml, csv, js, ts, jsx, tsx, py, java, c, cpp, to h, css, scss, sass, less, sh | text | TextArtifact |
| All others - generic | GenericArtifact |

Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:162-218

Specialized Artifacts

HTML Artifacts

HTML artifacts are a powerful feature, allowing the agent to create interactive applications that run in a sandboxed <iframe>.

Key Features:

• Sandboxed Execution: HTML, CSS, and JavaScript are executed inside a sandboxed <iframe> for security.
  Sources: packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:110-129
• Data Access: The sandbox environment is injected with helper functions to read other artifacts and user attachments via ArtifactsRuntimeProvider and AttachmentsRuntimeProvider. This allows for creating data visualizations (e.g., using Chart.js to render data from a .json artifact).
  Sources: packages/web-ui/src/tools/artifacts/artifacts.ts:110-122, packages/web-ui/src/prompts/prompts.ts:121-131
• Console Logging: Any console.log or console.error calls within the HTML artifact are captured and can be displayed in the UI, which is useful for debugging. The AI can request these logs using the logs command.
  Sources: packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:118-127, packages/web-ui/src/tools/artifacts/artifacts.ts:430-432
• Dual View: Users can toggle between a live "preview" of the rendered HTML and a "code" view showing the source with syntax highlighting.
  Sources: packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:28, 140-151

Text-Based Artifacts

Plain text and code files are handled by the TextArtifact component.

• It uses highlight.js to provide syntax highlighting for a wide variety of programming languages, determined by the file extension.
• It provides standard "Copy" and "Download" buttons in its header.
• If the file extension is not recognized as code, it renders the content in a plain <pre> tag.

Sources: packages/web-ui/src/tools/artifacts/TextArtifact.ts

Other Artifact Types

The system includes dedicated renderers for other common file types:

• ImageArtifact: Renders common image formats (.png, .jpg, etc.) inside an <img> tag.
• PdfArtifact: Embeds and renders PDF files.
• MarkdownArtifact: Renders Markdown content as formatted HTML.
• GenericArtifact: A fallback for any unrecognized file type, providing a download link.

Sources: packages/web-ui/src/tools/artifacts/ImageArtifact.ts, packages/web-ui/src/tools/artifacts/PdfArtifact.ts, packages/web-ui/src/tools/artifacts/artifacts.ts:182-218

Summary

The Web UI Artifact Rendering system is a flexible and extensible feature that significantly enhances the AI agent's capabilities. It allows the agent to move beyond simple text responses and generate rich, persistent files that are viewable and interactive directly within the chat application. The sandboxed execution of HTML artifacts is particularly powerful, enabling the creation of bespoke, data-driven tools and visualizations on the fly, making the agent a more capable and versatile assistant.