Running Locally

This document provides a comprehensive guide to setting up, configuring, and running the Technical Blog Monitor on a local development machine. The application is a Python daemon designed for high-performance monitoring of technical blogs, featuring an asynchronous architecture and pluggable components for extensibility.

Running locally is essential for development, debugging, and testing individual components of the processing pipeline. The system can be executed in a one-shot mode for quick checks or as a continuous daemon to mimic production behavior. For more information on the testing strategy, see the Testing Guide.

1. Prerequisites and Installation

Before running the application, ensure your local environment is set up with the necessary dependencies. The project uses uv for dependency management and Playwright for headless browser rendering.

Sources: README.md:29-39, WARP.md:10-17, pyproject.toml:6-7

1.1. System Requirements

• Python >= 3.12
• uv package manager

Sources: pyproject.toml:6-7, README.md:32-33

1.2. Installation Steps

The following diagram illustrates the one-time setup process.

1. Clone the repository:

   git clone https://github.com/NickJLange/technical-blog-monitor.git
   cd technical-blog-monitor
   

   Sources: README.md:30-31

2. Install dependencies: Use uv to sync the virtual environment with the dependencies specified in pyproject.toml.

   # Install uv (if not already installed)
   curl -LsSf https://astral.sh/uv/install.sh | sh
   
   # Install dependencies
   uv sync
   

   Sources: README.md:32-36

3. Install Playwright browsers: This one-time command downloads the browser binaries (e.g., Chromium) required for headless rendering.

   uv run playwright install
   

   Sources: README.md:38-39, WARP.md:13-14

4. (Optional) Setup pre-commit hooks: For development, install the dev dependencies and pre-commit hooks to ensure code quality.

   uv sync --group dev
   uv run pre-commit install
   

   Sources: README.md:106-108, WARP.md:16-17

2. Configuration

The application is configured via environment variables, which are loaded from a .env file by default. The configuration is managed by Pydantic settings, providing type validation and clear structure.

Sources: monitor/config.py, README.md:42-50

2.1. Environment File

Create a .env file by copying the example file. This is the primary way to configure the monitor for local runs.

cp .env.example .env

Sources: README.md:45-46

All settings are defined in monitor/config.py. Nested configuration keys use a double underscore (__) as a delimiter. For example, BROWSER__HEADLESS maps to the headless attribute within the BrowserConfig object.

Sources: README.md:92-93, monitor/config.py:270-275

2.2. Essential Configuration

The following table lists the key environment variables to set in your .env file.

Sources: WARP.md:120-137, monitor/config.py

For local testing without external services, you can use dummy clients:

• EMBEDDING__TEXT_MODEL_TYPE=custom
• VECTOR_DB__DB_TYPE=qdrant with VECTOR_DB__CONNECTION_STRING="memory://"

Sources: DEMO.md:50-53, test_basic.py:112-117, test_basic.py:146-152

2.3. Configuration Structure

The main Settings class in monitor/config.py aggregates all configuration components.

This diagram shows the nested structure of the configuration settings.

3. Execution

The application can be run from the command line using the monitor script entry point defined in pyproject.toml. The main entry point is the main() function in monitor/main.py.

Sources: monitor/main.py:539-577, pyproject.toml:27

3.1. Command-Line Arguments

The main function parses several command-line arguments to control execution behavior.

Sources: monitor/main.py:511-537

3.2. Execution Modes

One-Shot Run

This mode is useful for debugging and testing. It processes all enabled feeds once and then exits. The --log-level DEBUG flag provides verbose output.

uv run monitor --once --log-level DEBUG

Sources: README.md:53-55, monitor/main.py:500-509

To process only a single, specific feed:

uv run monitor --feed "Google Cloud Blog" --once

Sources: WARP.md:23-24, monitor/main.py:564-569

Daemon Mode

This is the default mode. The application runs continuously as a daemon, polling feeds at their configured intervals. It can be stopped with SIGINT (Ctrl+C) or SIGTERM.

uv run monitor

Sources: README.md:58-59, monitor/main.py:481-498

3.3. Application Lifecycle

The app_lifecycle context manager in monitor/main.py orchestrates the setup and teardown of all application components.

The following flowchart shows the application startup and shutdown sequence.

The sequence diagram below details the process for a one-shot run.

4. Running the Web Dashboard

The project includes a FastAPI-based web dashboard to view monitoring statistics and recently processed posts.

Sources: monitor/web/app.py, monitor/dashboard.py

To run the dashboard locally:

python monitor/dashboard.py

Sources: monitor/dashboard.py:17-41

Alternatively, use the provided shell script:

./run_dashboard.sh

Sources: run_dashboard.sh

The dashboard will be available at http://localhost:8080. It provides a simple UI and several API endpoints for retrieving data.

Sources: monitor/web/app.py:60-176, monitor/web/templates/index.html:56-59

5. Local Testing

The repository includes a suite of tests that can be run locally to verify functionality. For a complete guide, refer to TESTING.md.

5.1. Running All Checks

A convenience script runs all tests, linting, and type-checking.

./run_all_tests.sh

Sources: run_all_tests.sh

5.2. Running Specific Test Scripts

Individual test scripts can be executed to validate specific parts of the system without external dependencies. These scripts use in-memory or dummy implementations of external services.

Sources: TESTING.md:65-85, test_basic.py, test_feeds.py

For example, running test_basic.py executes a series of functions to test the main components in isolation.

Conclusion

Running the Technical Blog Monitor locally is a straightforward process involving dependency installation, environment configuration, and command-line execution. The system's modularity and use of dummy clients for testing allow developers to work on individual components efficiently. The available run modes (--once vs. daemon) and the web dashboard provide flexible options for debugging, manual testing, and observing the monitor's activity in real-time.