Relevant source files
The following files were used as context for generating this wiki page: - [packages/web-ui/src/tools/artifacts/artifacts.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/tools/artifacts/artifacts.ts) - [packages/web-ui/src/tools/artifacts/artifacts-tool-renderer.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/tools/artifacts/artifacts-tool-renderer.ts) - [packages/web-ui/src/tools/artifacts/ImageArtifact.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/tools/artifacts/ImageArtifact.ts) - [packages/web-ui/src/tools/artifacts/PdfArtifact.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/tools/artifacts/PdfArtifact.ts) - [packages/web-ui/src/tools/artifacts/HtmlArtifact.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/tools/artifacts/HtmlArtifact.ts) - [packages/web-ui/src/tools/artifacts/TextArtifact.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/tools/artifacts/TextArtifact.ts) - [packages/web-ui/src/prompts/prompts.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/prompts/prompts.ts) - [packages/web-ui/src/utils/test-sessions.ts](https://github.com/odra28/pi-mono/blob/main/packages/web-ui/src/utils/test-sessions.ts)
# Web UI Artifact Rendering The Web UI Artifact Rendering system enables an AI agent to create, manage, and display persistent files, known as artifacts, within the chat interface. This feature allows the agent to generate rich content like HTML applications, Markdown documents, images, and text files that live alongside the conversation. The system is composed of an `ArtifactsPanel` that hosts the rendered artifacts, an `AgentTool` for the AI to interact with, and a series of specialized `ArtifactElement` components for rendering different file types. This system provides a powerful way for the AI to present complex information, create interactive visualizations, and persist data across multiple turns in a conversation. A key feature is the ability to render sandboxed HTML artifacts that can programmatically access other artifacts and user attachments, enabling the creation of dynamic data dashboards and tools. ## Architecture Overview The artifact system consists of several key components that work together to manage and render artifacts. The `ArtifactsPanel` is the central orchestrator, managing artifact state and hosting the UI. The `ArtifactsToolRenderer` handles the display of tool calls within the chat log, while various `ArtifactElement` subclasses are responsible for rendering the content of specific file types. The following diagram illustrates the high-level architecture of the artifact rendering system. ```mermaid graph TD subgraph Chat Interface A[Agent] -- "Calls 'artifacts' tool" --> B(ArtifactsPanel) C[Chat Log] -- "Displays tool call" --> D{ArtifactsToolRenderer} end subgraph Artifacts Panel B -- "Manages state" --> E[Artifact Map] B -- "Determines file type" --> F[getFileType()] F -- "Selects renderer" --> G((ArtifactElement)) end subgraph Artifact Renderers G -- "Base Class" --> H[ArtifactElement] H --> I[HtmlArtifact] H --> J[ImageArtifact] H --> K[TextArtifact] H --> L[PdfArtifact] H --> M[...and others] end D -- "Renders diff/code" --> C I -- "Renders in sandbox" --> B J -- "Renders " --> B K -- "Renders
" --> B

```
*Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts](), [packages/web-ui/src/tools/artifacts/artifacts-tool-renderer.ts]()*

## `ArtifactsPanel` Component

The `artifacts-panel` custom element is the core of the artifact UI. It manages the lifecycle of all artifacts, provides the `AgentTool` for the AI, and renders the tabbed interface for viewing artifact content.

*Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts]()*

### Responsibilities

-   **State Management**: Maintains a map of all current artifacts, with the filename as the key and an `Artifact` object as the value.
    *Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:46-47]()*
-   **Tool Provider**: Exposes an `AgentTool` named `artifacts` that the AI agent can call to perform CRUD operations on artifacts.
    *Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:316-332]()*
-   **Command Execution**: Implements the logic for the `create`, `update`, `rewrite`, `get`, `delete`, and `logs` commands.
    *Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:413-432]()*
-   **Rendering Logic**: Determines which `ArtifactElement` to use for a given file based on its extension via the `getFileType` method.
    *Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:162-218]()*
-   **UI Management**: Renders the tab bar for switching between active artifacts and handles the display of the selected artifact's content.
    *Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:589-628]()*
-   **State Reconstruction**: Can reconstruct the entire artifact state from a list of chat messages, ensuring consistency when a session is loaded.
    *Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:335-411]()*

### Artifacts Tool

The `artifacts` tool allows the AI to manage files. It is defined by a JSON schema and an `execute` function within the `ArtifactsPanel`.

#### Tool Parameters

| Parameter | Type     | Description                                     |
| :-------- | :------- | :---------------------------------------------- |
| `command` | `string` | The operation to perform (`create`, `update`, `rewrite`, `get`, `delete`, `logs`). |
| `filename`| `string` | The name of the file, including its extension.  |
| `content` | `string` | Optional. The file content for `create` or `rewrite`. |
| `old_str` | `string` | Optional. The string to be replaced for `update`. |
| `new_str` | `string` | Optional. The replacement string for `update`.    |

*Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:39-45](), [packages/web-ui/src/prompts/prompts.ts:75-83]()*

#### Creation Flow

The following sequence diagram shows the process when an agent creates a new HTML artifact.

```mermaid
sequenceDiagram
    participant Agent as "AI Agent"
    participant APanel as "ArtifactsPanel"
    participant ARenderer as "ArtifactsToolRenderer"
    participant HArtifact as "HtmlArtifact"
    participant SBox as "SandboxIframe"

    Agent->>APanel: Call tool: artifacts.create('index.html', '...')
    note right of Agent: Tool call with command and content
    APanel->>APanel: executeCommand('create', ...)
    APanel->>APanel: createArtifact(...)
    APanel->>HArtifact: new HtmlArtifact()
    APanel->>HArtifact: set content
    HArtifact->>SBox: loadContent(html, runtimeProviders)
    note right of HArtifact: Provides access to other artifacts and attachments
    SBox-->>HArtifact: Sandbox ready
    APanel-->>Agent: Return "Created file index.html"
    
    note over Agent, SBox: The ArtifactsToolRenderer displays the tool call in the chat log.
    ARenderer->>ARenderer: render(params, result)
```
*Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:327-332, 442-466](), [packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:110-129](), [packages/web-ui/src/tools/artifacts/artifacts-tool-renderer.ts:60-256]()*

## Artifact Rendering

Rendering is handled by two main parts: the `ArtifactsToolRenderer` for the chat log entry and the specific `ArtifactElement` subclass for the content within the `ArtifactsPanel`.

### `ArtifactsToolRenderer`

This class is responsible for rendering the visual representation of an `artifacts` tool call within the message history.

-   It displays the status of the tool call (e.g., "Creating artifact", "Updated artifact", "Error").
-   It provides a collapsible section to view the details of the operation, such as the code being created or the diff for an update.
-   For HTML artifacts, it can also display a `console-block` with logs or errors from the sandboxed execution.
-   It uses a `getLanguageFromFilename` helper to apply correct syntax highlighting to code blocks.

*Sources: [packages/web-ui/src/tools/artifacts/artifacts-tool-renderer.ts]()*

### `ArtifactElement` and Implementations

`ArtifactElement` is the abstract base class for all artifact renderers. The `ArtifactsPanel` uses the `getFileType` method to select the appropriate concrete implementation based on the file extension.

```mermaid
classDiagram
    direction TB
    class ArtifactElement {
        <>
        +filename: string
        +content: string
        +getHeaderButtons()
    }
    class HtmlArtifact {
        +runtimeProviders: SandboxRuntimeProvider[]
        +executeContent(html)
        +getLogs(): string
    }
    class TextArtifact {
        -isCode(): boolean
    }
    class MarkdownArtifact
    class ImageArtifact
    class PdfArtifact
    class SvgArtifact
    class DocxArtifact
    class ExcelArtifact
    class GenericArtifact

    ArtifactElement <|-- HtmlArtifact
    ArtifactElement <|-- TextArtifact
    ArtifactElement <|-- MarkdownArtifact
    ArtifactElement <|-- ImageArtifact
    ArtifactElement <|-- PdfArtifact
    ArtifactElement <|-- SvgArtifact
    ArtifactElement <|-- DocxArtifact
    ArtifactElement <|-- ExcelArtifact
    ArtifactElement <|-- GenericArtifact
```
*This diagram shows the inheritance structure for different artifact types.*
*Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:221-279](), [packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:20](), [packages/web-ui/src/tools/artifacts/TextArtifact.ts:45]()*

#### File Type to Renderer Mapping

| File Type(s)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       - `html`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  - `svg`                                                                                                                                                                                                                         - `md`, `markdown`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        - `png`, `jpg`, `jpeg`, `gif`, `webp`, `bmp`, `ico`                                                                                                                            - `pdf`                                                                                                                            - `xlsx`, `xls`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               - `docx`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               - `txt`, `json`, `xml`, `yaml`, `yml`, `csv`, `js`, `ts`, `jsx`, `tsx`, `py`, `java`, `c`, `cpp`, to `h`, `css`, `scss`, `sass`, `less`, `sh` | `text`      | `TextArtifact`      |
| All others                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            - `generic`   | `GenericArtifact`   |

*Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:162-218]()*

## Specialized Artifacts

### HTML Artifacts

HTML artifacts are a powerful feature, allowing the agent to create interactive applications that run in a sandboxed ``.

```mermaid
graph TD
    A[HtmlArtifact] --> B(sandbox-iframe)
    B --> C{RuntimeMessageRouter}
    C --> D(AttachmentsRuntimeProvider)
    C --> E(ArtifactsRuntimeProvider)
    
    subgraph Sandbox Environment
        D -- "Provides" --> F["listAttachments()\nreadTextAttachment()\nreadBinaryAttachment()"]
        E -- "Provides" --> G["listArtifacts()\ngetArtifact()"]
    end

    F -- "Access to" --> H[User Uploaded Files]
    G -- "Read-only access to" --> I[Other Artifacts]
```
*This diagram shows how `HtmlArtifact` uses a sandboxed iframe with runtime providers to access user attachments and other artifacts.*

**Key Features:**

-   **Sandboxed Execution**: HTML, CSS, and JavaScript are executed inside a sandboxed `<iframe>` for security.
    *Sources: [packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:110-129]()*
-   **Data Access**: The sandbox environment is injected with helper functions to read other artifacts and user attachments via `ArtifactsRuntimeProvider` and `AttachmentsRuntimeProvider`. This allows for creating data visualizations (e.g., using Chart.js to render data from a `.json` artifact).
    *Sources: [packages/web-ui/src/tools/artifacts/artifacts.ts:110-122](), [packages/web-ui/src/prompts/prompts.ts:121-131]()*
-   **Console Logging**: Any `console.log` or `console.error` calls within the HTML artifact are captured and can be displayed in the UI, which is useful for debugging. The AI can request these logs using the `logs` command.
    *Sources: [packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:118-127](), [packages/web-ui/src/tools/artifacts/artifacts.ts:430-432]()*
-   **Dual View**: Users can toggle between a live "preview" of the rendered HTML and a "code" view showing the source with syntax highlighting.
    *Sources: [packages/web-ui/src/tools/artifacts/HtmlArtifact.ts:28, 140-151]()*

### Text-Based Artifacts

Plain text and code files are handled by the `TextArtifact` component.

-   It uses `highlight.js` to provide syntax highlighting for a wide variety of programming languages, determined by the file extension.
-   It provides standard "Copy" and "Download" buttons in its header.
-   If the file extension is not recognized as code, it renders the content in a plain `<pre>` tag.

*Sources: [packages/web-ui/src/tools/artifacts/TextArtifact.ts]()*

### Other Artifact Types

The system includes dedicated renderers for other common file types:

-   **`ImageArtifact`**: Renders common image formats (`.png`, `.jpg`, etc.) inside an `<img>` tag.
-   **`PdfArtifact`**: Embeds and renders PDF files.
-   **`MarkdownArtifact`**: Renders Markdown content as formatted HTML.
-   **`GenericArtifact`**: A fallback for any unrecognized file type, providing a download link.

*Sources: [packages/web-ui/src/tools/artifacts/ImageArtifact.ts](), [packages/web-ui/src/tools/artifacts/PdfArtifact.ts](), [packages/web-ui/src/tools/artifacts/artifacts.ts:182-218]()*

## Summary

The Web UI Artifact Rendering system is a flexible and extensible feature that significantly enhances the AI agent's capabilities. It allows the agent to move beyond simple text responses and generate rich, persistent files that are viewable and interactive directly within the chat application. The sandboxed execution of HTML artifacts is particularly powerful, enabling the creation of bespoke, data-driven tools and visualizations on the fly, making the agent a more capable and versatile assistant.</article></body></html>