Controller & Action Registry
Relevant source files
The following files were used as context for generating this wiki page:
The Controller & Action Registry system acts as the command execution engine for the browser-use framework. It provides a structured interface for AI agents to interact with web browsers by mapping LLM-generated intentions to concrete executable Python functions. The system manages a registry of available browser actions, handles parameter validation via Pydantic models, and executes sequences of commands within a specified browser context.
At its core, the Controller encapsulates the logic for navigating pages, interacting with DOM elements (clicking, typing, scrolling), and managing browser tabs. It bridges the gap between the high-level planning of an agent and the low-level automation provided by Playwright.
Core Architecture
The architecture is divided into the Controller service, which manages execution logic, and the Registry, which handles the registration and mapping of tools.
Controller Service
The Controller class is the primary interface for executing browser actions. It initializes with a set of default actions and allows for the registration of custom user-defined actions. It coordinates with the BrowserContext to perform operations on the live browser instance.
The flow shows how multi_act iterates through action sequences, checking for page stability and utilizing the registry to route calls.
Sources: browser_use/controller/service.py:321-366, browser_use/agent/views.py:27-35
Action Registry
The Registry (managed within the Controller) stores a mapping of action names to their corresponding function implementations and parameter schemas. This metadata is used to generate the system prompts that inform the LLM of its available capabilities.
Sources: browser_use/controller/service.py:36-40, browser_use/agent/prompts.py:114-125
Default Browser Actions
The Controller comes pre-configured with a suite of "Default Actions" that cover standard web navigation and interaction requirements.
Navigation and Tab Management
| Action Name | Parameter Model | Description |
|---|---|---|
search_google | SearchGoogleAction | Navigates to a Google search results page for a specific query. |
go_to_url | GoToUrlAction | Navigates the current tab to a specified URL. |
go_back | NoParamsAction | Navigates back in the browser history. |
open_tab | OpenTabAction | Creates a new browser tab at the specified URL. |
switch_tab | SwitchTabAction | Switches focus to a different tab by index. |
Sources: browser_use/controller/service.py:53-121
Element Interaction
| Action Name | Parameter Model | Description |
|---|---|---|
click_element | ClickElementAction | Clicks a DOM element identified by its numeric highlight index. |
input_text | InputTextAction | Types text into a specific input field. |
scroll_down | ScrollAction | Scrolls the page down by pixels or by one full page height. |
scroll_up | ScrollAction | Scrolls the page up. |
send_keys | SendKeysAction | Sends special keyboard keys or shortcuts (e.g., Control+o). |
Sources: browser_use/controller/service.py:124-219
Execution Logic
The Controller supports both single action execution (act) and sequential execution (multi_act).
Sequential Execution (multi_act)
When an agent provides a list of actions, the Controller executes them in order. A critical feature of multi_act is its state verification. If an action in the sequence results in new elements appearing on the page (detected via DOM branch path hashes), the Controller may interrupt the sequence to prevent the agent from using stale indexes.
Action Results
Every action returns an ActionResult object, which encapsulates:
is_done: Boolean indicating if the ultimate task is finished.extracted_content: String containing data retrieved from the page (e.g., text extraction).error: String containing error messages if the action failed.include_in_memory: Boolean determining if this result should be persisted in the agent's long-term memory.
Sources: browser_use/agent/views.py:27-35
Custom Action Registration
Developers can extend the Controller's capabilities by using the @controller.action decorator. This allows for domain-specific tools (e.g., save_jobs_to_csv) to be exposed to the AI agent.
# Example of custom action registration
@controller.action('Save jobs to file', param_model=JobModel)
def save_jobs(job: JobModel):
# Custom logic here
return ActionResult(extracted_content="Saved")
Sources: browser_use/controller/service.py:315-319, examples/use-cases/find_and_apply_to_jobs.py:44-49
Summary
The Controller & Action Registry provides a robust, type-safe framework for browser automation. By leveraging Pydantic for parameter validation and a structured registry for tool management, it ensures that AI agents can interact with complex web interfaces reliably. Its ability to handle sequential actions while monitoring for DOM state changes makes it highly effective for multi-step browser tasks.