System Prompts
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
System prompts in the Web-UI project serve as the foundational logic and behavioral guidelines for AI agents. They define the "persona," operational constraints, and response formats that LLMs must adhere to when interacting with web browsers or performing complex research tasks. By leveraging specialized prompts like the CustomSystemPrompt, the system ensures that agents maintain structured state awareness and follow rigorous execution rules during multi-step browser interactions.
These prompts are integrated into various modules, including the CustomAgent for standard browser navigation and the DeepResearch module for specialized information gathering and report writing. The architecture relies on these prompts to bridge the gap between high-level user instructions and low-level browser actions, ensuring consistent JSON-formatted outputs for reliable programmatic processing.
Agent System Prompt Architecture
The core of the agent's behavior is defined in CustomSystemPrompt, which inherits from the base SystemPrompt. This class encapsulates the rules for response formatting, element interaction, and error handling.
CustomSystemPrompt Components
The prompt is divided into several critical instruction sets that the LLM must follow during its execution loop.
| Component | Purpose |
|---|---|
| Response Format | Mandates a specific JSON structure containing current_state and an action sequence. |
| Common Action Sequences | Provides templates for frequent tasks like form filling or navigation. |
| Element Interaction | Restricts the agent to using valid numeric indexes for interactive HTML elements. |
| Task Completion | Defines the "Done" action criteria and verification requirements. |
| Visual Context | Instructs the agent on how to use screenshots and bounding boxes to verify page layout. |
Sources: src/agent/custom_prompts.py:12-92, src/agent/custom_views.py:22-31
Prompt Data Flow
The following diagram illustrates how the system prompt is integrated into the agent's message lifecycle via the CustomMessageManager.
Deep Research Prompts
The deep_research.py module utilizes three specialized system prompts to manage the lifecycle of a research project: planning, recording, and reporting.
Search and Recording Logic
The Deep Researcher prompt focuses on formulating research plans and generating specific search queries. The Expert Information Recorder prompt is designed to extract and summarize findings while maintaining source URLs and "thinking" logic for future report generation.
| Prompt Type | Role | Key Output Fields |
|---|---|---|
| Search System Prompt | Planner | plan, queries |
| Record System Prompt | Extractor | url, title, summary_content, thinking |
| Writer System Prompt | Author | Markdown Report, Reference List |
Sources: src/utils/deep_research.py:101-125, src/utils/deep_research.py:133-176
Deep Research Sequence
The research process involves a loop where these prompts are called sequentially.
Sources: src/utils/deep_research.py:186-267
Structured Response Format
A critical requirement of the system prompts is the strict JSON output format. This allows the CustomAgent to parse the LLM's "thoughts" and "actions" programmatically.
JSON Schema Requirements
The current_state block must include:
prev_action_evaluation: Analysis of whether the previous step succeeded.important_contents: Relevant data extracted from the current page.task_progress: A summary of completed sub-tasks.future_plans: The remaining steps to fulfill the user request.thought: The internal reasoning for the next operation.
Sources: src/agent/custom_prompts.py:16-25, src/agent/custom_views.py:22-31
Action Sequences
The system allows for multiple actions to be chained in a single response, such as inputting a username and password before clicking a button.
{
"current_state": { ... },
"action": [
{"input_text": {"index": 1, "text": "user"}},
{"click_element": {"index": 2}}
]
}
Sources: src/agent/custom_prompts.py:28-36, src/agent/custom_agent.py:197-205
Agent Message Prompts
While the System Prompt defines the rules, the CustomAgentMessagePrompt (User Message) provides the real-time context. It dynamically constructs the input for each step, including:
- Task: The original user instruction.
- Hints: Optional
add_infosprovided via the UI. - Memory: Cumulative information recorded in previous steps.
- State: Current URL, open tabs, and interactive elements.
- Previous Results: Evaluations and errors from the last action sequence.
Sources: src/agent/custom_prompts.py:118-175, src/agent/custom_agent.py:155-171
Conclusion
System prompts are the central nervous system of the Web-UI agents. They enforce the transition from unstructured natural language to the structured JSON required for reliable browser automation. By separating concerns into distinct prompts for searching, recording, and agent navigation, the system can handle complex, multi-stage research and automation workflows with high precision and error recovery capabilities.