Web UI Interface
Relevant source files
The following files were used as context for generating this wiki page:
The Web UI Interface is a comprehensive Gradio-based application designed to provide a user-friendly graphical interface for interacting with AI agents that control web browsers. It serves as the primary entry point for users to configure Large Language Models (LLMs), set browser parameters, and execute automated tasks or deep research iterations. The interface abstracts the underlying complexity of the browser-use library, allowing for real-time monitoring of agent actions through live browser views and recording playbacks.
Sources: webui.py:441-450, README.md:14-16
Architecture and Components
The Web UI is structured as a single-page application with multiple tabs, each dedicated to specific functional areas of the agent's operation. It manages global states for the browser, context, and agent instances to support features like persistent sessions.
Interface Structure
The UI is divided into the following primary tabs:
- Agent Settings: Configures agent types (original or custom), maximum steps, and vision capabilities.
- LLM Configuration: Handles provider selection (OpenAI, Anthropic, Ollama, etc.) and model-specific parameters.
- Browser Settings: Manages browser lifecycle, window dimensions, and recording paths.
- Run Agent: The execution hub where tasks are described and live feedback is displayed.
- Deep Research: A specialized module for iterative, multi-step information gathering and report generation.
- Results & Recordings: Displays final outputs, errors, and historical media.
- Configuration: Allows saving and loading full UI states via
.pklfiles.
Sources: webui.py:539-780, src/utils/default_config_settings.py:10-33
System Orchestration
When running in a containerized environment, the Web UI is part of a larger stack managed by supervisord. This ensures that the Web UI, the browser (via Playwright/Chromium), and a VNC server for remote viewing are all synchronized.
The diagram shows the process hierarchy where supervisord manages the lifecycle of the Web UI and its dependencies.
Sources: supervisord.conf:1-105, docker-compose.yml:14-25
Execution Flow
The Web UI supports two main execution modes: Standard Agent Run and Deep Research.
Standard Agent Execution
The execution flow involves initializing the LLM, setting up the browser context, and streaming the agent's progress back to the Gradio interface.
This sequence illustrates the iterative communication between the UI, the LLM, and the browser automation layer.
Sources: webui.py:126-215, webui.py:341-438
Deep Research Module
The Deep Research module performs multiple iterations of search, extraction, and summarization to generate a markdown report.
- Logic: It uses a
CustomControllerwith a specializedextract_contentaction (utilizing Jina Reader). - Iteration: It generates search queries, executes them using parallel or sequential agents, and records information into a structured history.
- Finalization: A dedicated
generate_final_reportfunction uses the collected history to write a publication-ready markdown document.
Sources: src/utils/deep_research.py:35-130, src/utils/deep_research.py:220-270
Configuration and Parameters
The Web UI manages a wide array of settings, which are summarized in the table below.
| Category | Parameter | Default Value | Description |
|---|---|---|---|
| Agent | agent_type | custom | Choose between org (original) or custom logic. |
max_steps | 100 | Maximum iterations before the agent stops. | |
| LLM | llm_provider | openai | Supported: OpenAI, Anthropic, Google, DeepSeek, etc. |
llm_temperature | 1.0 | Controls randomness in model outputs. | |
| Browser | headless | False | Run without a visible GUI. |
window_w | 1280 | Width of the browser window. | |
keep_browser_open | False | Retain session between different tasks. | |
| Paths | save_recording_path | ./tmp/record_videos | Location for .mp4/.webm recordings. |
Sources: src/utils/default_config_settings.py:10-33, webui.py:539-650, src/utils/utils.py:133-145
Technical Implementation Details
LLM Integration
The get_llm_model function in src/utils/utils.py acts as a factory, instantiating the correct LangChain chat model based on the UI selection. It specifically handles authentication by mapping provider names to environment variables like OPENAI_API_KEY.
Sources: src/utils/utils.py:36-130
Live View Streaming
Live feedback is achieved in run_with_stream by periodically capturing screenshots of the active Playwright page, encoding them as base64 strings, and yielding them as HTML <img> tags to the Gradio browser_view component.
# From webui.py:400-410
encoded_screenshot = await capture_screenshot(_global_browser_context)
if encoded_screenshot is not None:
html_content = f'<img src="data:image/jpeg;base64,{encoded_screenshot}" ...>'
Sources: webui.py:395-415, src/utils/utils.py:175-210
Persistence Logic
Global variables _global_browser and _global_browser_context are used to maintain state. If keep_browser_open is enabled, these objects are not closed at the end of a task execution, allowing the next task to inherit the same browser state (e.g., logged-in sessions).
Sources: webui.py:40-45, webui.py:325-340
Conclusion
The Web UI Interface provides a robust framework for managing AI-driven browser automation. By integrating real-time streaming, extensive LLM support, and specialized research capabilities, it serves as an essential tool for both simple browser tasks and complex information gathering. Its architecture ensures scalability through containerization while maintaining a simplified user experience through the Gradio frontend.