Output Processing & Validation
Relevant source files
The following files were used as context for generating this wiki page:
Output Processing and Validation is a core mechanism in the browser-use library that ensures AI agents return structured, accurate, and validated data. By leveraging Pydantic models, the system can enforce specific schemas on the agent's final response, transition from unstructured web content to machine-readable JSON, and validate the success of executed actions against the intended goals.
This system bridges the gap between the Large Language Model (LLM) and the application layer by providing a robust framework for handling ActionResult objects and AgentOutput structures. It allows developers to define custom output formats, ensuring that the automation results are immediately usable in downstream processes.
Sources: browser_use/agent/views.py:1-40, examples/features/custom_output.py:20-40
Structured Output Models
The library uses Pydantic to define the structure of the agent's brain and its actions. The AgentOutput class serves as the primary container for the agent's state assessment and the sequence of actions it intends to perform.
Core Data Structures
| Class | Description | Key Fields |
|---|---|---|
AgentBrain | Represents the internal state and reasoning of the agent. | page_summary, evaluation_previous_goal, memory, next_goal |
AgentOutput | The base model for LLM responses, containing the brain and a list of actions. | current_state, action |
ActionResult | The outcome of a single action execution. | is_done, extracted_content, error, include_in_memory |
AgentHistory | Records the state, output, and result of a specific step. | model_output, result, state |
Sources: browser_use/agent/views.py:44-90
Custom Output Definition
Developers can override the default completion behavior by passing an output_model to the Controller. When a custom model is provided, the done action is dynamically re-registered to accept and validate the specific schema provided by the user.
Sources: browser_use/controller/service.py:44-55, examples/features/custom_output.py:23-35
The diagram shows the relationship between the primary output models used for agent reasoning and execution.
Sources: browser_use/agent/views.py:44-75
Validation Mechanism
Validation occurs at multiple levels: JSON schema validation of the LLM response and functional validation of the results.
Schema Validation
The AgentOutput can be extended with custom actions using create_model. This allows the Agent to validate that the LLM is calling registered tools with the correct parameters. If the LLM output does not match the expected schema, the AgentError class provides formatted validation error messages to be sent back to the model for correction.
Sources: browser_use/agent/views.py:77-86, browser_use/agent/views.py:270-285
Result Validation
When validate_output=True is set in the Agent configuration, the system checks the validity of the action results. This is particularly relevant for the done action, where the final extracted content must match the expected output format.
The flow shows the steps taken to process and validate output from the model.
Sources: browser_use/agent/message_manager/service.py:305-320, examples/features/validate_output.py:38-45
Result Processing Logic
The Controller and MessageManager handle the conversion of action outcomes into context for the next step.
Extraction and Memory
The ActionResult model includes an include_in_memory flag. When set to True, the extracted_content is added to the permanent message history, ensuring the agent "remembers" specific facts found on a page even if the page content itself is removed from the context window to save tokens.
Sources: browser_use/agent/message_manager/service.py:125-140, browser_use/controller/service.py:70-85
Multi-Action Execution
The multi_act method in the Controller processes a sequence of actions. It monitors for state changes (hashes of the element tree) to ensure that subsequent actions in a sequence are still valid based on the updated DOM. If a new element appears or the page changes unexpectedly, the sequence is interrupted to prevent errors.
Sources: browser_use/controller/service.py:280-315
Error Formatting
The AgentError class standardizes how exceptions are presented to the agent.
- Validation Errors: Instruct the model to follow the correct schema.
- Rate Limit Errors: Signals a retry delay.
- Stacktraces: Can be optionally included for debugging.
Sources: browser_use/agent/views.py:270-285
History Management
Processed outputs and results are stored in AgentHistoryList. This structure allows for post-execution analysis and serialization.
| Method | Description |
|---|---|
final_result() | Retrieves the extracted_content from the last successful action in history. |
is_done() | Checks if the terminal ActionResult has is_done=True. |
model_actions() | Extracts all executed actions along with the specific DOM elements they interacted with. |
save_to_file() | Serializes the entire history, including screenshots and state, to a JSON file. |
Sources: browser_use/agent/views.py:125-220
Summary
Output Processing & Validation in browser-use transforms raw LLM responses into structured AgentOutput objects. Through Pydantic models, it enforces strict schema adherence and allows for custom data extraction via the output_model configuration. By validating action results and managing a detailed execution history, the system provides a reliable bridge between browser interactions and structured data output.
Sources: browser_use/agent/views.py:44-90, browser_use/controller/service.py:44-60