Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• CLAUDE.md
• CONTRIBUTING.md
• pyproject.toml
• tests/backend/test_backend.py
• tests/skills/test_skill_manager.py
• tests/skills/test_skill_tool.py
• tests/autocompletion/test_ui_chat_autocompletion.py

The QQcode testing strategy is built around a comprehensive suite of automated tests that ensure the reliability of the CLI coding assistant's core logic, backend integrations, and user interface components. The project emphasizes strong static typing and modern Python 3.12+ practices to maintain high code quality and prevent regressions during rapid development.

The testing infrastructure leverages pytest as the primary test runner, supplemented by specialized plugins for asynchronous operations, UI snapshots, and parallel execution. This approach allows developers to validate everything from low-level YAML parsing to complex LLM streaming interactions and TUI (Textual User Interface) behavior.

Sources: CLAUDE.md:20-30, CONTRIBUTING.md:58-62, pyproject.toml:79-90

Testing Infrastructure

The project uses uv to manage the development environment and execute tests. The testing suite is configured to run in parallel by default to optimize performance.

Test Runner and Configuration

The project is configured with pytest and several key plugins:

• pytest-asyncio: For testing asynchronous functions and coroutines.
• pytest-xdist: Enables parallel test execution (running with -n auto by default).
• pytest-timeout: Prevents tests from hanging by enforcing a 10-second timeout per test.
• pytest-textual-snapshot: Used for validating the visual state of the Textual-based TUI.

Sources: pyproject.toml:80-90, pyproject.toml:142-145, CLAUDE.md:20-25

Execution Commands

Common commands used for testing the repository:

Command	Description
uv run pytest	Runs all tests in parallel
uv run pytest -v	Runs tests with verbose output
uv run pytest <path>	Runs a specific test file
uv run ruff check .	Executes linting checks
uv run pyright	Executes static type checking

Sources: CLAUDE.md:20-43, CONTRIBUTING.md:76-85

Backend and LLM Testing

The backend testing strategy focuses on validating the interaction between the QQcode agent and various LLM providers (Mistral, Fireworks, etc.). To ensure reliability without relying on live API calls, the project uses respx to mock HTTP responses.

Mocking Strategy

The system uses real LLM provider API responses as a baseline for test data. These responses are simplified and stored in data modules to ensure tests remain accurate to real-world scenarios.

The diagram shows how respx intercepts backend HTTP requests to provide static, pre-defined LLM response data for validation.

Sources: tests/backend/test_backend.py:1-12, tests/backend/test_backend.py:61-75

Tested Backend Behaviors

• Synchronous Completion: Validating message content, finish reasons, and token usage from standard API calls.
• Streaming Completion: Ensuring LLMChunk objects correctly capture partial content and tool calls during a stream.
• Error Handling: Mocking 500 (Internal Server Error) and 429 (Rate Limit) responses to verify BackendError is raised correctly.
• User-Agent Validation: Verifying that the correct user-agent header is sent with both standard and streaming requests.

Sources: tests/backend/test_backend.py:77-105, tests/backend/test_backend.py:151-180, tests/backend/test_backend.py:230-245

Skill System Testing

The Skill system is tested through two primary layers: management logic and tool execution.

Skill Management

test_skill_manager.py validates the lifecycle of "Skills"—custom workflows defined in Markdown.

• Frontmatter Parsing: Ensures YAML metadata (name, description) is correctly extracted from SKILL.md files.
• Discovery: Validates that the SkillManager correctly identifies valid skill directories within the project structure while skipping invalid ones.
• Prompt Generation: Tests the generation of XML-based prompt sections that inform the LLM about available skills.

Sources: tests/skills/test_skill_manager.py:35-50, tests/skills/test_skill_manager.py:100-115, tests/skills/test_skill_manager.py:205-220

Skill Tool Execution

test_skill_tool.py focuses on the functional aspect of skills when called as tools by the agent.

• Content Loading: Verifies that invoking a skill tool returns the correct Markdown content and file path.
• Error States: Ensures ToolError is raised when non-existent skills are requested or when the manager is uninitialized.
• UI Display: Tests how skill tool calls and results are formatted for the user interface.

Sources: tests/skills/test_skill_tool.py:75-85, tests/skills/test_skill_tool.py:90-100, tests/skills/test_skill_tool.py:135-150

User Interface Testing

UI testing focuses on the Textual-based TUI, specifically the interactive chat and autocompletion features.

Autocompletion and Navigation

Tests use a pilot object to simulate user keystrokes and verify the behavior of the CompletionPopup.

This sequence illustrates the interaction flow during path autocompletion testing.

Sources: tests/autocompletion/test_ui_chat_autocompletion.py:30-45, tests/autocompletion/test_ui_chat_autocompletion.py:155-170

Functional UI Coverage

• Slash Commands: Verifies that typing / triggers the command completion popup and that selecting an item populates the input.
• Path Completion: Validates recursive file searching and fuzzy matching for the @ symbol.
• History Navigation: Ensures that navigating through command history (Up/Down arrows) does not inadvertently trigger the autocompletion popup.

Sources: tests/autocompletion/test_ui_chat_autocompletion.py:115-130, tests/autocompletion/test_ui_chat_autocompletion.py:220-235, tests/autocompletion/test_ui_chat_autocompletion.py:260-275

Code Quality and Linting

The testing strategy is supported by strict linting and type checking to ensure code maintainability.

• Ruff: Used for both linting and formatting. It enforces a line length of 88 characters and checks for a variety of Python best practices (e.g., modern type hints, unused imports).
• Pyright: Used for static type checking, ensuring compliance with the project's modern Python 3.12+ typing requirements (e.g., using | for unions and built-in generics like list).
• Pre-commit Hooks: Automates these checks (Ruff, Pyright, Typos, YAML/TOML validation) to prevent low-quality code from being committed.

Sources: CLAUDE.md:32-43, CONTRIBUTING.md:87-105, pyproject.toml:105-135

Summary

The QQcode testing strategy combines unit tests for core logic, mocked integration tests for LLM backends, and interactive UI tests for the TUI components. By utilizing pytest with parallel execution and enforced timeouts, the project maintains a fast and reliable feedback loop. The inclusion of strict linting and static analysis further guarantees that the codebase adheres to modern Python standards and remains robust against regressions.