Key Features
Relevant source files
The following files were used as context for generating this wiki page:
The web-ui project is a comprehensive interface and extended toolkit for browser-use, enabling AI agents to interact with web browsers to perform complex tasks. It provides a user-friendly Gradio-based interface, multi-provider LLM support, and specialized modules for deep research and persistent browsing.
The core purpose of the system is to bridge the gap between high-level AI reasoning and low-level browser automation. By integrating various LLM providers (OpenAI, Anthropic, DeepSeek, etc.) and providing both standard and custom agent implementations, the project allows developers and users to automate web-based workflows with granular control over agent behavior, visual context, and session persistence.
Sources: README.md:14-25, webui.py:465-475
1. Multi-Provider LLM Integration
The system provides a unified interface for interacting with various Large Language Model (LLM) providers. This is managed through a central utility that handles API keys, base URLs, and model-specific configurations.
Supported Providers and Models
The get_llm_model function in src/utils/utils.py serves as the factory for creating LLM instances compatible with LangChain.
| Provider | Default Model | Custom Logic |
|---|---|---|
| OpenAI | gpt-4o | Supports custom endpoints via OPENAI_ENDPOINT. |
| Anthropic | claude-3-5-sonnet-20240620 | standard LangChain implementation. |
| DeepSeek | deepseek-chat | Includes special handling for deepseek-reasoner (R1) models. |
gemini-2.0-flash-exp | Uses ChatGoogleGenerativeAI. | |
| Ollama | qwen2.5:7b | Supports local execution with custom context length (num_ctx). |
| Azure OpenAI | gpt-4o | Requires api_version and azure_endpoint. |
Sources: src/utils/utils.py:27-133, src/utils/utils.py:157-167
DeepSeek R1 Integration
A specialized feature of this project is the enhanced support for DeepSeek-R1, which allows the agent to engage in "deep thinking." This is implemented through custom classes that extract and log reasoning_content from the model's response.
2. Deep Research System
The Deep Research feature implements a multi-step iterative process where an AI agent plans, executes multiple search queries, records information, and generates a final comprehensive report.
Deep Research Workflow
The system uses a loop to refine its search until a maximum number of iterations is reached or the goal is satisfied.
Sources: src/utils/deep_research.py:32-152
Key Components
- Search System Prompt: Defines the "Deep Researcher" persona and constraints (e.g., maximum search queries per iteration).
- Record System Prompt: Instructs the LLM to summarize findings while minimizing information loss and avoiding redundancy.
- Report Generation: Uses a specialized "Writer" prompt to structure the final output with citations and professional formatting.
Sources: src/utils/deep_research.py:75-150, src/utils/deep_research.py:196-248
3. Custom Agent Architecture
While the project supports the original browser-use agent, it introduces a CustomAgent that provides enhanced state tracking, visual context processing, and more descriptive logging.
Enhanced State Tracking
The CustomAgent utilizes a CustomAgentStepInfo dataclass to maintain context across steps:
- Memory: Cumulative important contents recorded during operations.
- Task Progress: A summary of completed items.
- Future Plans: Remaining steps needed to fulfill the instruction.
Sources: src/agent/custom_views.py:11-19, src/agent/custom_agent.py:186-212
Response Format Logic
The custom agent enforces a strict JSON response format from the LLM to ensure structured reasoning and action sequencing.
| Field | Description |
|---|---|
prev_action_evaluation | Analysis of whether the previous action succeeded based on page elements. |
important_contents | Data closely related to the user instruction found on the current page. |
thought | Reflection on completed and upcoming operations. |
action | A sequence of actions (e.g., click_element, input_text). |
Sources: src/agent/custom_prompts.py:18-35, src/agent/custom_agent.py:119-138
4. Browser Session Management
The system offers flexible browser management, including Dockerized environments and persistent sessions.
Persistence Modes
- Standard Mode: Browser opens and closes with each task to ensure a clean state and lower resource usage.
- Persistent Mode: Controlled via
CHROME_PERSISTENT_SESSION=true. The browser stays open between tasks, maintaining history, cookies, and login states. - Own Browser: Users can point the agent to their local Chrome installation using
CHROME_PATHandCHROME_USER_DATA.
Sources: README.md:130-145, webui.py:176-185, docker-compose.yml:25-27
Recording and Traces
The UI provides real-time "Live Browser View" by capturing screenshots from the Playwright context and encoding them as base64 for the Gradio HTML component. Additionally, it supports full video recording and Playwright trace generation.
Sources: webui.py:417-458, src/utils/utils.py:228-268
5. Deployment and Containerization
The project is designed to run in a headless Docker environment, utilizing a virtual display (Xvfb) and providing remote access via VNC.
Configuration Parameters
| Parameter | Env Variable | Default | Description |
|---|---|---|---|
| Resolution | RESOLUTION | 1920x1080x24 | Screen dimensions for the virtual display. |
| VNC Password | VNC_PASSWORD | vncpassword | Security for the noVNC interface. |
| Chrome Debug | CHROME_DEBUGGING_PORT | 9222 | Port for remote debugging. |
Sources: docker-compose.yml:30-40, src/utils/default_config_settings.py:10-30
Conclusion
The web-ui project provides a robust framework for browser-based AI agents. By combining iterative research capabilities, multi-LLM support (notably DeepSeek-R1), and sophisticated session persistence, it offers a complete environment for both automated web research and interactive browser assistance. Its containerized architecture ensures that these features can be deployed consistently across different platforms.
Sources: README.md:14-25, webui.py:650-660