Testing Framework
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The testing framework for browser-use is designed to ensure the reliability and accuracy of AI-driven browser automation. It leverages the pytest framework alongside pytest-asyncio to handle the asynchronous nature of Playwright-based interactions. The framework spans across multiple layers of the application, including core agent functionality, browser context management, DOM extraction logic, and message management.
The scope of the framework includes unit tests for individual components, integration tests for end-to-end agent tasks, and specialized dataset evaluations. It provides a structured way to validate that the AI agent can correctly perceive web states, execute planned actions (such as clicking, typing, or scrolling), and manage complex message histories within token limits.
Core Architecture and Test Categories
The testing infrastructure is organized into distinct categories to target specific modules of the system.
Core Functionality Testing
This layer focuses on the end-to-end capabilities of the Agent. It validates that the agent can successfully translate natural language tasks into specific browser actions. Key actions tested include searching Google, navigating to URLs, handling tab switching, and executing scrolling operations.
The diagram shows the flow from a pytest execution through the Agent to the final validation of action history.
Sources: tests/test_core_functionality.py:34-150
Browser Context and State Testing
Tests within tests/test_context.py verify the low-level browser management logic. This includes the conversion of XPaths to CSS selectors, URL allowlist enforcement, and state capture (screenshots and scroll info).
| Feature | Description | Tested Component |
|---|---|---|
| URL Filtering | Validates that navigation is restricted to allowed_domains. | _is_url_allowed |
| Selector Conversion | Ensures XPaths with indices (e.g., [2]) map to correct CSS (e.g., :nth-of-type(2)). | _convert_simple_xpath_to_css_selector |
| State Reset | Verifies that context resets close all tabs and clear cached state. | reset_context |
| Visual Capture | Ensures screenshots are correctly encoded as Base64 strings. | take_screenshot |
Sources: tests/test_context.py:11-200
Specialized Testing Modules
Message Management and Token Handling
The MessageManager is critical for maintaining the conversation history within the LLM's token constraints. Tests in this module simulate long interaction sequences to ensure that the history is pruned correctly without losing essential system or task information.
This sequence illustrates how the MessageManager tracks token usage and prunes history during interaction steps.
Sources: browser_use/agent/message_manager/tests.py:100-200
Dataset and Benchmark Testing
The framework includes integration with external datasets like Mind2Web to evaluate agent performance against real-world web navigation tasks. These tests load processed JSON task descriptions and validate the agent's ability to complete them in a headless environment.
Mind2Web Test Requirements:
- Max Steps: 50
- Test Subset Size: 10
- Data Integrity: Checks for
website,confirmed_task, andaction_reprsfields.
Sources: tests/test_mind2web.py:22-105
DOM Extraction and Interactive Element Logic
Tests in browser_use/dom/tests/extraction_test.py evaluate the efficiency of the DOM tree building process. They measure the impact of viewport_expansion on element count and token consumption.
- Expansionpx (0 to 1000): Only includes elements within or near the viewport.
- All elements (-1): Captures the entire page's interactive elements.
- Focus Element: Tests the system's ability to isolate a specific element for the LLM.
Sources: browser_use/dom/tests/extraction_test.py:25-60
Configuration and Fixtures
The framework utilizes standard pytest fixtures to manage lifecycle and resource cleanup.
@pytest.fixture(scope='function')
async def browser():
browser_instance = Browser(config=BrowserConfig(headless=True))
yield browser_instance
await browser_instance.close()
@pytest.fixture
async def context(browser):
async with await browser.new_context() as context:
yield context
Sources: tests/test_core_functionality.py:18-35
Conclusion
The browser-use testing framework provides a multi-layered approach to validating AI browser automation. By combining low-level unit tests for DOM and context management with high-level integration tests for Agent tasks and dataset benchmarks, the framework ensures that the system remains stable as LLM capabilities and web structures evolve. Specific focus on token management and interactive element extraction ensures the agent remains cost-effective and accurate during long-running tasks.