Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• TESTING.md
• run_all_tests.sh
• test_full_pipeline.py
• test_basic.py
• test_feeds.py
• pyproject.toml
• monitor/tests/test_article_parser.py
• WARP.md

The testing strategy for the Technical Blog Monitor is comprehensive, employing a multi-layered approach that includes unit tests, integration tests, static analysis, and type checking. This ensures code quality, reliability, and maintainability. The entire test suite is designed to be run easily via a single script, and it leverages tools like pytest, ruff, and mypy to automate quality assurance.

A key aspect of the strategy is the ability to test core application logic without heavy dependencies or external services. This is achieved through the use of dummy clients and in-memory databases, allowing for fast, isolated, and reliable testing of individual components and the basic application architecture.

Sources: TESTING.md, run_all_tests.sh, test_basic.py:5-12, WARP.md:38-48

Test Suite Execution

The project provides a convenient shell script to execute the complete test suite, which covers all aspects of quality assurance from linting to full pipeline integration tests.

Sources: TESTING.md:22-31, run_all_tests.sh

Master Test Script

The run_all_tests.sh script is the primary entry point for running all checks. It sequentially executes unit tests, multiple integration tests, code linting, and type checking. The script is designed to exit with a non-zero status code if any of the test suites fail.

Sources: run_all_tests.sh:4-49

The following diagram illustrates the execution flow of the run_all_tests.sh script.

Manual Execution

Developers can also run specific test suites or individual test files manually using uv run.

Command	Description
uv run pytest	Runs the complete unit test suite.
uv run pytest monitor/tests/test_article_parser.py	Runs a specific unit test file.
uv run python test_basic.py	Runs the basic integration tests.
uv run python test_feeds.py	Runs the feed parsing integration tests.
uv run python test_full_pipeline.py	Runs the end-to-end pipeline test.
uv run ruff check .	Runs the code linter.
uv run mypy monitor/	Runs the static type checker.

Sources: TESTING.md:33-55, run_all_tests.sh:30-41

Test Categories

The testing strategy is divided into several categories, each targeting a different aspect of the application.

Unit Tests

Unit tests are managed by pytest and are located in the monitor/tests/ directory. They focus on testing individual modules and functions in isolation. The configuration in pyproject.toml sets the test path and default options, including coverage reporting with a failure threshold of 80%.

Sources: pyproject.toml:76-78, TESTING.md:33-45, WARP.md:121-122

Key pytest configuration from pyproject.toml:

[tool.pytest.ini_options]
testpaths = ["monitor/tests"]
asyncio_mode = "auto"
addopts = "-q --cov=monitor --cov-report=term-missing --cov-fail-under=80"

Sources: pyproject.toml:76-78

An example unit test is monitor/tests/test_article_parser.py, which verifies the extract_article_content function by mocking dependencies like httpx and the cache client to ensure the parser logic works as expected.

Sources: monitor/tests/test_article_parser.py

Integration Tests

Integration tests are provided as standalone Python scripts in the root directory. They test the collaboration between different components of the system.

Sources: TESTING.md:47-55

Script	Purpose	Dependencies Mocked
test_basic.py	Tests core functionality: content extraction, embedding, vector DB operations, and caching.	Uses a dummy embedding client and an in-memory vector database. Does not require Playwright.
test_feeds.py	Tests feed parsing for specific blogs (e.g., Uber, Meta AI).	Uses a dummy browser pool. Focuses on get_feed_processor and process_feed_posts.
test_full_pipeline.py	Tests the complete end-to-end application lifecycle for all configured feeds.	Uses the full app_lifecycle context, including real browser rendering and component initialization.

Sources: TESTING.md:80-101, test_basic.py:5-12, test_feeds.py:5-13, test_full_pipeline.py:5-8

test_basic.py Workflow

This script validates the fundamental building blocks of the application without external services.

The diagram below shows the sequence of operations in test_basic.py.

test_full_pipeline.py Workflow

This script provides an end-to-end test of the entire monitoring process, from loading settings to processing each configured feed.

The sequence diagram below shows how test_full_pipeline.py orchestrates the application.

Static Analysis and Quality

The project enforces code quality through static analysis tools, which are integrated into the main test script.

• Linting: ruff is used for static analysis and linting to enforce style consistency and catch potential bugs.
• Type Checking: mypy is used for static type checking to ensure type safety.
• Security Scanning: bandit is used to scan for common security vulnerabilities in the codebase.

Sources: run_all_tests.sh:38-41, WARP.md:44-48, pyproject.toml:72-74, 80-83

Testing without External Services

A core part of the testing strategy is the ability to run tests without dependencies on external services like OpenAI, Qdrant, or live web pages. This is achieved by using dummy and in-memory implementations of the core service clients.

• Dummy Embeddings: EMBEDDING__TEXT_MODEL_TYPE=custom in the configuration will load a DummyEmbeddingClient that generates placeholder vectors.
• In-Memory Vector DB: The InMemoryVectorDBClient provides a transient, in-memory store that mimics the behavior of a real vector database like Qdrant.
• Filesystem Cache: The CACHE__ENABLED=true setting with a local path enables caching without needing a Redis server.

Sources: TESTING.md:112-118, test_basic.py:108-115, 148-155, DEMO.md:40-44

The following diagram shows how test-specific implementations are used instead of production clients.

This approach allows test_basic.py to run quickly and reliably in any environment, including CI/CD pipelines, by testing the internal logic without network latency or external service failures.

Sources: test_basic.py:5-12