Web Interfaces (Gradio & Streamlit)
Relevant source files
The following files were used as context for generating this wiki page:
The web interfaces for browser-use provide graphical entry points for interacting with AI agents that control browser environments. These interfaces abstract the underlying complexity of the Agent and Browser modules, allowing users to define tasks, select models, and view execution results through interactive web-based dashboards.
The primary purpose of these interfaces is to facilitate rapid testing and demonstration of agent capabilities, such as navigating websites, extracting content, and automating multi-step browser workflows.
Sources: examples/ui/README.md:1-10, README.md:55-65
Architecture and Data Flow
The web interfaces act as a wrapper around the Agent class. They capture user input via form elements and trigger asynchronous browser tasks. The execution flow follows a request-response pattern where the UI component communicates with the backend agent logic.
UI to Agent Communication Flow
The following diagram illustrates how user inputs from the web interface are processed by the underlying agent system.
The flow starts with user-provided parameters (task, model, and credentials) which are passed to the Agent instance. The Controller then maps LLM-generated decisions to specific browser actions.
Sources: examples/ui/gradio_demo.py:53-100, browser_use/controller/service.py:40-150
Gradio Interface
The Gradio-based interface provides a simple, structured layout for task automation. It specifically supports standard OpenAI models and allows for headless or headed browser execution.
Key Components
| Component | Description |
|---|---|
run_browser_task | An asynchronous function that sets the environment API key, initializes the Agent, and returns the execution result. |
create_ui | Defines the layout using gr.Blocks, including input fields for tasks and dropdowns for model selection. |
ActionResult | A dataclass used to structure the output of browser actions, containing status, content, and errors. |
Sources: examples/ui/gradio_demo.py:20-50, examples/ui/gradio_demo.py:82-95
UI Configuration Parameters
The Gradio implementation exposes several parameters to the user to customize the agent's behavior.
| Parameter | Type | Default | Description |
|---|---|---|---|
OpenAI API Key | Textbox (Password) | N/A | Required for authenticating with the chosen LLM provider. |
Task Description | Textbox | N/A | The natural language command for the agent (e.g., "Find flights"). |
Model | Dropdown | gpt-4 | Selection between supported OpenAI models. |
Run Headless | Checkbox | True | Whether to show the browser window during execution. |
Sources: examples/ui/gradio_demo.py:84-93
Streamlit Interface
The Streamlit interface offers an alternative web-based dashboard for running browser automation tasks. While functionally similar to the Gradio demo, it integrates into the Streamlit ecosystem, which is often used for data science and internal tooling.
Execution Model
The Streamlit interface is executed as a module using the command:
python -m streamlit run streamlit_demo.py
It leverages Streamlit's reactive programming model to update the UI as the agent progresses through its task steps.
Sources: examples/ui/README.md:10, README.md:15-20
Interaction with Underlying Systems
The web interfaces rely on the Controller to translate high-level UI requests into browser commands. The Controller manages default actions like search_google, go_to_url, and click_element.
Action Registry Sequence
When a task is submitted via the UI, the agent utilizes a sequence of tool calls managed by the MessageManager.
The MessageManager ensures that the UI context and task history are maintained within token limits during long-running tasks initiated from the web interface.
Sources: browser_use/agent/message_manager/service.py:50-100, browser_use/controller/service.py:350-400
Conclusion
The Gradio and Streamlit interfaces serve as the primary human-in-the-loop components for the browser-use project. By providing standardized input fields and real-time output displays, they lower the barrier for testing complex browser automation scripts. These interfaces connect the high-level task definitions to the Agent logic and Playwright-powered browser context, enabling a seamless transition from natural language prompts to executable browser actions.