Agent Views
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Agent Views represent the structured data models and schemas that define how the AI agent perceives its internal state, processes reasoning, and formats its output for the browser automation environment. These views act as the bridge between the raw Large Language Model (LLM) responses and the executable actions within the browser-use framework.
The primary purpose of the Agent Views system is to enforce a strict JSON schema for LLM outputs, ensuring that the agent not only selects actions but also provides structured "thinking" metadata. This metadata includes self-evaluations of previous actions, memory updates, and future planning, which are critical for the reliability of complex multi-step tasks.
Sources: src/agent/custom_views.py:1-52, src/agent/custom_prompts.py:18-43
Core Data Models
The system utilizes Pydantic and Python dataclasses to define the schema for the agent's internal "brain" and its externalized output.
CustomAgentBrain
This model encapsulates the cognitive state of the agent at any given step. It requires the LLM to reflect on its progress before deciding on the next action.
| Field | Type | Description |
|---|---|---|
prev_action_evaluation | str | Analysis of whether the previous step succeeded or failed based on visual/DOM evidence. |
important_contents | str | Data extracted from the current page relevant to the user's instruction. |
task_progress | str | A summary of completed items in the overall task. |
future_plans | str | A concise list of remaining steps needed to finish the instruction. |
thought | str | The reasoning process for the current step and reflection on failures. |
summary | str | A natural language description of the next intended operation. |
Sources: src/agent/custom_views.py:22-31, src/agent/custom_prompts.py:21-30
CustomAgentOutput
The CustomAgentOutput class extends the base AgentOutput to include the CustomAgentBrain. It is dynamically generated to support various action registries defined in the controller.
class CustomAgentOutput(AgentOutput):
model_config = ConfigDict(arbitrary_types_allowed=True)
current_state: CustomAgentBrain
action: list[ActionModel]
Sources: src/agent/custom_views.py:34-52
CustomAgentStepInfo
A dataclass used to track the environment state across agent iterations, carrying context like memory and task history.
Sources: src/agent/custom_views.py:9-19
System Architecture and Data Flow
The Agent Views system facilitates a feedback loop where the state of the browser is transformed into a prompt, processed into a structured view, and then executed.
View Interaction Logic
The following diagram shows how the CustomAgent uses these views to process an LLM response into executable actions.
The CustomAgent invokes the LLM and parses the response specifically into the AgentOutput model defined in the views. If the model_name includes "DeepSeek", the agent may adjust its token limits or handling logic while maintaining this view structure.
Sources: src/agent/custom_agent.py:165-188, src/agent/custom_agent.py:145-160
Relationship Hierarchy
The class hierarchy ensures that custom reasoning logic is bundled with standard browser-use actions.
Sources: src/agent/custom_views.py:22-52
Integration with Prompt Engineering
The CustomSystemPrompt explicitly defines the expected JSON format to match the CustomAgentBrain structure. This ensures the LLM populates all required fields for the UI to display thoughts and progress.
Prompt Requirements
- Response Format: Must always be valid JSON.
- Action Sequencing: Actions must follow the
ActionModeldefined in the controller. - Evaluation: The
prev_action_evaluationmust use the website as the ground truth, ignoring previous internal result codes if they contradict the visual state.
Sources: src/agent/custom_prompts.py:18-40
Usage in Web Interface
The WebUI (defined in webui.py) consumes the data provided by these views to update the user interface in real-time.
| UI Component | View Data Source |
|---|---|
| Model Thoughts | CustomAgentOutput.current_state.thought |
| Final Result | AgentHistory.final_result() (derived from views) |
| Task Progress | CustomAgentOutput.current_state.task_progress |
| Errors | ActionResult.error |
Sources: webui.py:615-635, src/agent/custom_agent.py:145-160
Conclusion
Agent Views provide the necessary structure for high-level reasoning and reliable automation in the web-ui project. By extending base action models with a dedicated "Brain" view, the system allows for sophisticated features like deep research, self-reflection, and persistent task tracking. This structured approach is essential for integrating advanced LLMs like DeepSeek-R1 and GPT-4o into a cohesive browser-based agent.
Sources: src/agent/custom_agent.py:1-50, src/utils/deep_research.py:230-250