Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• tests/test_browser_use.py
• tests/test_llm_api.py
• src/utils/utils.py
• src/agent/custom_agent.py
• webui.py
• src/utils/deep_research.py

The Testing Strategy for the web-ui project focuses on validating the integration between Large Language Models (LLMs), automated browser controllers, and custom agent logic. The strategy ensures that various LLM providers function correctly, browser interactions are executed as intended, and complex multi-step tasks like deep research are handled reliably across different environments.

Testing is primarily conducted through specialized test scripts that simulate agent tasks, verify API connectivity, and exercise the browser-use framework. These tests cover individual components like LLM wrappers and high-level workflows involving autonomous navigation and information extraction.

Sources: tests/test_llm_api.py, tests/test_browser_use.py

LLM Integration Testing

The project employs a robust testing suite to verify the compatibility and performance of multiple LLM providers. This ensures that the unified interface for model invocation works consistently regardless of the underlying backend.

Provider Validation

The test_llm_api.py module defines a structured approach to testing different providers including OpenAI, Google Gemini, Azure OpenAI, DeepSeek, and Ollama. Tests focus on:

• Text Generation: Verifying basic chat and reasoning capabilities.
• Vision Capabilities: Testing the model's ability to describe images via base64 encoding.
• Reasoning Content: Specifically validating models like deepseek-reasoner or deepseek-r1 which provide separate reasoning paths.

Sources: tests/test_llm_api.py:37-124, src/utils/utils.py:22-127

LLM Test Configuration Structure

The testing framework uses a LLMConfig dataclass to standardize the parameters passed to the LLM factory during test execution.

Parameter	Type	Description
provider	string	The LLM service provider (e.g., 'openai', 'ollama').
model_name	string	The specific model identifier.
temperature	float	Controls randomness (default 0.8).
base_url	string	Custom API endpoint for the provider.
api_key	string	Authentication key for the provider.

Sources: tests/test_llm_api.py:16-22

LLM Request Flow

The following diagram illustrates the flow from a test case through the utility layer to the LLM provider.

The diagram shows how the get_llm_model utility acts as a central factory for routing test requests to the appropriate model class based on the provider.
Sources: src/utils/utils.py:22-127, tests/test_llm_api.py:46-77

Browser Agent Testing

Testing the browser agent involves verifying that the AI can successfully perceive web page states and execute actions via the Playwright-based controller.

Functional Testing Scenarios

Tests in test_browser_use.py simulate real-world user requests to validate the agent's decision-making and execution loop:

1. Navigation and Extraction: Navigating to a URL (e.g., Google), performing a search, and extracting specific results.
2. State Persistence: Testing the ability to use "own browser" (persistent sessions) vs. ephemeral contexts.
3. Parallel Execution: Validating the system's ability to handle multiple agents running concurrent tasks using asyncio.gather.

Sources: tests/test_browser_use.py:64-75, tests/test_browser_use.py:157-175, tests/test_browser_use.py:257-268

Agent Execution Sequence

The testing of a browser agent follows a specific sequence of initialization and execution steps.

This sequence represents the standard loop exercised during functional tests to ensure the agent correctly processes visual and structural page information.
Sources: tests/test_browser_use.py:64-100, src/agent/custom_agent.py:333-365

Advanced Feature Testing

Deep Research Validation

The Deep Research module is tested for its recursive search and reporting capabilities. This involves testing:

• Query Generation: The ability of the agent to generate multiple relevant search queries based on a plan.
• Recursive Processing: Iterative searching where subsequent queries depend on previous findings.
• Markdown Reporting: Validating that the final output is a properly formatted report with citations.

Sources: src/utils/deep_research.py:34-150, webui.py:456-476

Custom Agent Logic

The CustomAgent class, which extends the base Agent, is tested for specific enhancements:

• Deep Thinking Integration: Handling reasoning content for models like DeepSeek-R1.
• Memory Management: Correctly updating the CustomAgentStepInfo including task progress and future plans.
• GIF Creation: Verifying the post-process step that generates a visual history of the task.

Sources: src/agent/custom_agent.py:173-196, src/agent/custom_agent.py:440-549

Testing Infrastructure and Tools

The project utilizes several utilities and configurations to facilitate testing across different environments.

Tool/Component	Purpose	Reference File
Playwright	Core engine for browser automation and screenshot capture.	webui.py, tests/test_browser_use.py
Gradio	Used to provide an interactive interface for manual "Smoke Testing".	webui.py
dotenv	Manages environment variables like API keys for test model access.	tests/test_llm_api.py
AgentHistoryList	Captures errors, thoughts, and actions for assertion in tests.	tests/test_browser_use.py

Environment Configuration for Testing

Tests require specific environment variables to be set, often managed via a .env file. These include CHROME_PATH for persistent browser tests and various provider-specific API keys.

# Example of environment variable retrieval in tests
chrome_path = os.getenv("CHROME_PATH", None)
api_key = os.getenv("OPENAI_API_KEY", "")

Sources: tests/test_browser_use.py:49-54, tests/test_llm_api.py:37-44

Summary

The Testing Strategy effectively combines unit-level validation of LLM APIs with complex integration tests of autonomous browser agents. By leveraging both automated scripts and interactive UI elements, the project ensures that the orchestration of LLMs, custom prompts, and browser controllers remains stable across diverse task descriptions and model providers. This multi-layered approach is critical for maintaining reliability in non-deterministic AI-driven workflows.

Sources: tests/test_browser_use.py, webui.py, src/agent/custom_agent.py