Prompt Studio & Chatbooks
Relevant source files
The following files were used as context for generating this wiki page:
- Docs/Development/Used_Prompts/UX/UX_Review_Prompt_for_Prompts_Playground.md
- Docs/Product/UX_REVIEW_PROMPT_PROMPTS_PAGE.md
- Docs/Product/Completed/Plans/IMPLEMENTATION_PLAN_prompts_page_10_missing_functionality_backend_gaps_2026_02_18.md
- Docs/Architecture.md
- AGENTS.md
- Docs/Product/WebUI/Workspace_Playground_Redesign.md
Introduction
Prompt Studio and Chatbooks represent the advanced prompt engineering and knowledge portability layers of the tldw_server project. Prompt Studio is a server-backed sub-app designed for advanced testing, versioning, and optimization of prompts, moving beyond simple local storage to provide a professional playground for developers and researchers. It integrates directly with the project's LLM provider layer to enable ad-hoc execution and systematic evaluation.
Chatbooks serve as the mechanism for exporting and importing content, handling background jobs, and facilitating the portability of analyzed media and chat sessions. Together, these systems bridge the gap between simple chat interactions and complex, iterative prompt development and knowledge management.
Sources: Docs/Product/UX_REVIEW_PROMPT_PROMPTS_PAGE.md, AGENTS.md:53-56
Prompt Studio Architecture
Prompt Studio is structured as a professional prompt management environment consisting of five primary functional areas: Projects, Prompts, Test Cases, Evaluations, and Optimizations. Unlike the "Custom" local prompt workspace which utilizes IndexedDB, Prompt Studio is entirely server-backed and requires an active server connection to function.
Core Components
- Project Management: Organizes prompts and test cases into logical containers with metadata including name and description.
- Versioning System: Tracks prompt iterations with version numbers, change descriptions, and "Revert" capabilities.
- Playground: Provides an ad-hoc execution environment for testing prompts with various providers and models, displaying token counts and execution times.
- Evaluation Engine: Enables batch execution of test cases against specific prompts to generate aggregate quality metrics.
Data Flow
The following diagram illustrates the lifecycle of a prompt within the Studio environment, from creation through testing and evaluation.
The flow represents an iterative cycle where evaluation results drive further optimizations and prompt edits.
Sources: Docs/Development/Used_Prompts/UX/UX_Review_Prompt_for_Prompts_Playground.md, Docs/Product/UX_REVIEW_PROMPT_PROMPTS_PAGE.md
Data Models
| Entity | Key Fields | Purpose |
|---|---|---|
| Project | id, name, description, prompt_count | High-level container for related prompts. |
| Prompt | project_id, system_prompt, user_prompt, few_shot_examples, version_number | The actual instructions and configuration for the LLM. |
| TestCase | project_id, inputs, expected_outputs, is_golden | Standardized inputs for consistent evaluation. |
| Evaluation | prompt_id, status, aggregate_metrics | Results of running test cases against a prompt version. |
Sources: Docs/Development/Used_Prompts/UX/UX_Review_Prompt_for_Prompts_Playground.md
Chatbooks and Knowledge Portability
Chatbooks facilitate the export and import of content within the tldw_server ecosystem. They are utilized to package media analysis, notes, and chat histories into portable formats. This system also handles background jobs for processing these exports and imports.
Functional Roles
- Exporting: Packages user data (summaries, reports, notes) into Chatbook formats for external use or backup.
- Importing: Allows users to bring previously exported Chatbooks back into the server, restoring the state of specific research sessions.
- Background Processing: Uses the project's job handling system to manage long-running export/import tasks without blocking the UI.
Sources: AGENTS.md:55-56, Docs/Architecture.md
Integration and Advanced Features
The Prompt Management system bridges the gap between local "Custom" prompts and the server-side "Studio" via a synchronization mechanism.
Sync and Collections
- Server Sync: Users can push local prompts to Studio, pull updates, or link/unlink local records with server-side versions. This includes conflict detection and resolution.
- Collections: The backend supports Prompt Collections API for creating named groups of prompts, although this is primarily a server-side capability used for higher-order organization.
- Optimization: Prompt Studio includes nine optimization strategies to help users refine their instructions based on evaluation performance.
Logic Flow: Prompt Execution
This sequence diagram shows the interaction between the user interface and the backend when executing a prompt within the Studio Playground.
Execution involves resolving variables, selecting the appropriate provider, and capturing performance data.
Sources: Docs/Product/UX_REVIEW_PROMPT_PROMPTS_PAGE.md, Docs/Product/Completed/Plans/IMPLEMENTATION_PLAN_prompts_page_10_missing_functionality_backend_gaps_2026_02_18.md
Summary
Prompt Studio and Chatbooks provide the sophisticated tools necessary for professional prompt engineering and data portability in tldw_server. By offering version control, structured evaluations, and robust import/export capabilities, these systems transform the platform from a simple media analyzer into a comprehensive research and development environment. They ensure that knowledge is not only captured but can be iteratively improved and moved across different environments.