High-Level Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The Browser-use framework provides a structured bridge between Large Language Models (LLMs) and web browsers. It enables AI agents to perceive web pages through a processed Document Object Model (DOM) and interact with them using a set of predefined or custom actions. The architecture is designed to handle the complexities of web automation, including state management, asynchronous execution, and multi-tab handling.
At its core, the system orchestrates a loop where the browser state is captured, converted into a format suitable for an LLM (including visual screenshots and interactive element lists), and then used by the agent to determine the next sequence of actions. These actions are executed through a controller that interfaces directly with Playwright.
Sources: browser_use/agent/prompts.py:84-110, browser_use/browser/context.py:21-40
Component Overview
The architecture is divided into three primary layers: the Browser Layer, the Controller Layer, and the Agent/Message Layer.
1. Browser Layer (BrowserContext)
This layer manages the lifecycle of the browser session using Playwright. It handles tab management, network stabilization, and state extraction. It maintains a BrowserSession containing the current page context and a cached_state.
- Key Responsibilities: Navigating URLs, taking screenshots, managing cookies, and maintaining a whitelist of allowed domains.
- State Extraction: Uses
DomServiceto execute JavaScript that builds a tree of interactive elements.
Sources: browser_use/browser/context.py:114-150, browser_use/browser/context.py:440-460
2. Controller Layer (Controller)
The Controller acts as the action registry and execution engine. it maps LLM-generated JSON commands to specific Python functions that interact with the BrowserContext.
- Registry: Stores default actions like
click_element,input_text, andgo_to_url. - Multi-Act: Allows the execution of multiple actions in a single step, with checks to see if the page state changed significantly between actions.
Sources: browser_use/controller/service.py:38-70, browser_use/controller/service.py:318-350
3. Agent & Message Layer (MessageManager)
This layer manages the conversation history with the LLM. It ensures that the system prompt, task instructions, and current browser state are correctly formatted and fit within the model's token limits.
- Token Management: Proactively trims message history or screenshots if the input exceeds
max_input_tokens. - Vision Integration: Formats the browser screenshot as part of the
HumanMessagefor multi-modal models.
Sources: browser_use/agent/message_manager/service.py:38-75, browser_use/agent/message_manager/service.py:230-260
System Data Flow
The following diagram illustrates the lifecycle of a single agent step, from state perception to action execution.
The flow ensures the LLM always acts on the most recent "stabilized" state of the browser.
Sources: browser_use/agent/message_manager/service.py:146-170, browser_use/controller/service.py:318-340
DOM Extraction Logic
The extraction of interactive elements is a critical part of the architecture. The system uses a specialized JavaScript file (buildDomTree.js) injected into the page to identify clickable, visible, and "top" elements.
- Interactivity Detection: Elements are considered interactive if they have specific tags (
<a>,<button>,<input>), ARIA roles, or event listeners (e.g.,onclick). - Coordinate Mapping: Each element is assigned a
highlight_indexand its coordinates (viewport and page) are recorded.
A visual representation of how elements are filtered before being presented to the AI.
Sources: browser_use/dom/buildDomTree.js:154-200, browser_use/dom/service.py:55-80
Core Data Structures
The following table summarizes the key data models used across the architecture to maintain consistency.
| Model | File Path | Description |
|---|---|---|
BrowserState | browser_use/browser/views.py | Contains the URL, title, element tree, selector map, and screenshot. |
AgentOutput | browser_use/agent/views.py | The expected JSON response from the LLM, containing current_state and action list. |
ActionResult | browser_use/agent/views.py | The result of an action, including errors or extracted content. |
BrowserContextConfig | browser_use/browser/context.py | Configuration for window size, timeouts, and security settings. |
Sources: browser_use/agent/views.py:53-75, browser_use/browser/context.py:44-100
Sequence Diagram: Action Execution
This diagram shows the interaction between the Controller and the BrowserContext during action execution.
The sequence highlights how the Controller translates abstract AI instructions into concrete browser operations.
Sources: browser_use/controller/service.py:406-440, browser_use/browser/context.py:650-680
Security and Navigation Control
The architecture includes several safety mechanisms to prevent agents from navigating to unintended locations or leaking sensitive data.
- URL Filtering: The
BrowserContextchecks every navigation against anallowed_domainslist. If a navigation to a non-allowed domain is detected, the browser automatically executes ago_back()operation and raises aURLNotAllowedError. - Sensitive Data Masking: The
MessageManagerincludes a filtering mechanism that replaces sensitive strings (defined in a dictionary) with placeholders like<secret>placeholder_name</secret>before sending messages to the LLM.
Sources: browser_use/browser/context.py:407-430, browser_use/agent/message_manager/service.py:207-225
Conclusion
The High-Level Architecture of Browser-use is built on a robust loop of perception and action. By abstracting the DOM into a "Selector Map" and providing a structured "Controller" for actions, it simplifies the integration of LLMs with complex web environments. The separation of concerns between state extraction (DomService), message orchestration (MessageManager), and execution (Controller) allows for high scalability and easier debugging of autonomous browser tasks.