Creating Custom Actions
Relevant source files
The following files were used as context for generating this wiki page:
Custom actions allow developers to extend the capabilities of the browser-use agent beyond standard browser interactions like clicking and typing. By registering custom functions with a Controller, you can enable the AI to perform domain-specific tasks, such as saving data to files, reading local PDFs, or interacting with external APIs, while providing the LLM with structured parameters and descriptions for accurate tool calling.
The system uses a Registry within the Controller to manage these actions. When a custom action is registered, it becomes part of the ActionModel that the agent uses to generate steps. These actions can return an ActionResult, which provides feedback (extracted content or errors) back to the agent's memory for subsequent decision-making.
Sources: browser_use/controller/service.py:16-30, browser_use/agent/service.py:126-140, examples/use-cases/find_and_apply_to_jobs.py:53-65
Action Registration Architecture
The Controller class acts as the primary interface for defining custom actions. It maintains a Registry that maps action names to their implementation functions and associated Pydantic models for parameter validation.
The Registration Flow
Custom actions are typically registered using the @controller.action decorator. This decorator requires a description, which is used in the system prompt to explain to the LLM when and how to use the tool.
The diagram shows the lifecycle of a custom action from definition to LLM selection.
Sources: browser_use/controller/service.py:340-345, browser_use/agent/service.py:180-185, browser_use/controller/registry/service.py
Key Components
| Component | Description |
|---|---|
Registry | Internal service that stores function mappings and generates JSON schemas for the LLM. |
ActionResult | The standard return type for actions, containing is_done, extracted_content, and error fields. |
ActionModel | A dynamically created Pydantic model that encompasses all registered actions as optional fields. |
param_model | A Pydantic BaseModel used to define and validate the arguments the LLM must provide. |
Sources: browser_use/agent/views.py:27-34, browser_use/controller/service.py:22-35, browser_use/agent/service.py:181-183
Implementing Custom Actions
To implement a custom action, you must define a function that accepts specific arguments. If the action needs to interact with the browser, you can include browser: BrowserContext as a parameter, and the controller will automatically inject the current session.
Interaction with Browser Context
Custom actions can access the underlying Playwright page or the browser-use DOM abstractions. For example, the upload_cv action retrieves a DOM element by index and then uses the BrowserContext to locate the actual Playwright ElementHandle.
@controller.action('Upload cv to element', param_model=UploadIndex)
async def upload_cv(params: UploadIndex, browser: BrowserContext):
path = str(CV.absolute())
dom_el = await browser.get_dom_element_by_index(params.index)
# ... logic to locate and upload file ...
return ActionResult(extracted_content=f'Uploaded to {params.index}')
Sources: examples/use-cases/find_and_apply_to_jobs.py:84-118, browser_use/controller/service.py:355-380
Handling Action Results
The ActionResult returned by a custom action determines the agent's next state:
include_in_memory: IfTrue, theextracted_contentis added to the chat history for the LLM to see in the next step.is_done: IfTrue, signals to the agent that the ultimate task is complete.error: If provided, the error string is passed to the LLM to allow for retries or alternative strategies.
Sources: browser_use/agent/views.py:27-34, browser_use/agent/service.py:245-255
Sequence of Action Execution
When the Agent decides to call a custom action, the Controller manages the execution through its act and multi_act methods.
This sequence illustrates how the Controller orchestrates calls between the Agent and custom registered functions.
Sources: browser_use/controller/service.py:347-380, browser_use/agent/service.py:236-244
Custom Output Models
Developers can define a global output_model for the Controller. When a task is marked as "Done", the controller can be configured to require the LLM to provide data matching a specific Pydantic schema rather than a simple string.
- Define a Pydantic model for the desired structured output.
- Initialize the
Controllerwithoutput_model=YourModel. - The
doneaction is automatically re-registered to use this model as itsparam_model.
Sources: examples/features/custom_output.py:23-38, browser_use/controller/service.py:41-55
Implementation Examples
Data Storage Action
This example demonstrates saving LLM-extracted data to a local CSV file.
class Job(BaseModel):
title: str
company: str
@controller.action('Save job to file', param_model=Job)
def save_job(job: Job):
with open('jobs.csv', 'a') as f:
# ... write logic ...
return 'Saved job to file'
Sources: examples/use-cases/find_and_apply_to_jobs.py:53-65
Information Extraction Action
Actions can be used to provide the LLM with context from local files, such as reading a PDF CV.
@controller.action('Read my cv')
def read_cv():
text = extract_pdf_text(CV_PATH)
return ActionResult(extracted_content=text, include_in_memory=True)
Sources: examples/use-cases/find_and_apply_to_jobs.py:75-81
Summary
Creating custom actions is the primary method for integrating domain-specific logic into browser-use. By utilizing the @controller.action decorator and structured Pydantic models, developers can extend the agent's capabilities while maintaining strict validation and providing clear instructions to the LLM. Custom actions leverage the BrowserContext for deep browser integration and return ActionResult objects to influence the agent's memory and task progression.