CLI Reference

Relevant source files

The following files were used as context for generating this wiki page:

• abx_dl/cli.py
• abx_dl/__main__.py
• README.md
• pyproject.toml
• skills/abx-dl/SKILL.md
• tests/test_cli.py

The abx-dl Command Line Interface (CLI) is the primary entry point for the ArchiveBox downloader tool. It is built using the rich-click library to provide a polished, interactive terminal experience with live progress updates, structured tables, and colored output. The CLI serves as a simplified wrapper around the ArchiveBox plugin ecosystem, allowing users to download and extract content from URLs using various tools like wget, yt-dlp, and headless Chrome.

The CLI architecture employs a "default command" pattern where bare URLs provided as arguments are automatically routed to the download logic. It manages complex multi-phase execution, including dependency installation, environment setup, and snapshot extraction, while maintaining a live UI driven by an internal EventBus.

Sources: abx_dl/cli.py:1-20, README.md:1-20, pyproject.toml:130-131

Command Structure and Routing

The CLI is organized into a main group with several subcommands. A specialized DefaultGroup class handles command resolution, allowing the tool to intelligently guess when a user wants to start a download without requiring the explicit dl subcommand.

Command Resolution Logic

The DefaultGroup.resolve_command method checks if the first argument looks like a URL (containing ://). If it does, and it doesn't match an existing subcommand name, it automatically prepends the dl command to the argument list.

The diagram shows the logic used by DefaultGroup to route bare URLs to the download engine.
Sources: abx_dl/cli.py:101-137, tests/test_cli.py:461-471

Primary Commands

Command	Description	Example
dl	The default command. Downloads a URL using enabled plugins.	abx-dl dl 'https://example.com'
plugins	Lists available plugins, their status, and metadata.	abx-dl plugins --install
install	Shortcut to install dependencies for specific plugins.	abx-dl install wget chrome
config	Manages persistent and session-level configuration.	abx-dl config --set TIMEOUT=120

Sources: abx_dl/cli.py:656-915, README.md:135-145

Download Execution (dl)

The dl command manages the full lifecycle of a URL archive attempt. It initializes an EventBus, sets up a LiveBusUI for terminal feedback, and executes the orchestrator's download function.

Execution Phases

The download process follows a strict sequence of events captured by the CLI:

1. Install Phase: Resolves BinaryRequestEvent to ensure external tools (e.g., chromium) are available.
2. CrawlSetup Phase: Configures shared state or daemons (e.g., launching Chrome).
3. Snapshot Phase: Executes the actual extraction tools against the target URL.
4. Cleanup Phase: Tears down daemons and finalizes output files.

The sequence diagram illustrates how the CLI interacts with the EventBus and UI during a download.
Sources: abx_dl/cli.py:688-782, README.md:52-65, skills/abx-dl/SKILL.md:46-55

Configuration Options

The dl command supports several flags to tune behavior:

• --plugins: Comma-separated list (e.g., wget,screenshot).
• --output: Sets the destination directory (defaults to CWD).
• --timeout: Per-hook execution limit.
• --max-urls / --max-size: Limits for crawl sessions.
• --no-install: Prevents lazy-loading of missing binaries.

Sources: abx_dl/cli.py:695-711, tests/test_cli.py:382-390

Interactive UI and Progress Monitoring

The LiveBusUI class provides a real-time view of the download process using rich.live. It consumes events from the EventBus to update a multi-section display.

Live UI Components

• Active Hook Table: Displays currently running processes, their phase (Install, Setup, Snapshot), elapsed time, and live output.
• Progress Bar: Shows overall completion status based on the total number of expected hooks.
• Streamed Results: As hooks complete, their ArchiveResult metadata (status, size, output path) is printed to the terminal above the live area.

Table Rendering Logic

The CLI uses sophisticated logic to format the "Output" column in tables, including:

• _compact_output: Collapses whitespace and truncates long strings.
• _render_output_size_cell: Applies color styles (green, yellow, red, blink) based on file size thresholds.
• _render_hook_name_cell: Color-codes hook names (e.g., cyan for event names, magenta for order).

Size Threshold	Style
< 2MB	SIZE_GREEN_STYLE
< 50MB	SIZE_YELLOW_STYLE
< 100MB	SIZE_ORANGE_STYLE
> 1GB	SIZE_FLASHING_STYLE (Blink)

Sources: abx_dl/cli.py:72-88, abx_dl/cli.py:349-436, tests/test_cli.py:108-140

Configuration Management (config)

The config command allows users to inspect and modify the environment-based configuration system. It handles three layers of config:

1. Global Defaults: Hardcoded system defaults.
2. User Config: Stored in ~/.config/abx/config.env.
3. Derived Cache: Runtime-derived values (like binary paths) stored in derived.env.

The CLI provides --get for specific keys and --set for persistent updates. It resolves aliases automatically using plugin schemas (e.g., setting USE_WGET=false maps to WGET_ENABLED=false).

Sources: abx_dl/cli.py:894-935, README.md:85-115

Plugin Management (plugins / install)

The plugins command serves both as an information tool and a dependency manager. It lists all discovered plugins from the abx-plugins package and checks for the existence of required binaries.

Dependency Installation

When install or plugins --install is called:

1. It triggers a BinaryRequestEvent for every required binary.
2. Provider plugins (like brew, apt, npm, or pip) intercept these events.
3. Successful installations emit a BinaryEvent containing the absolute path and version.
4. The CLI caches these in derived.env to skip subsequent checks for 24 hours.

Sources: abx_dl/cli.py:795-850, README.md:147-170, TODOs.md:12-25

The abx-dl CLI provides a robust interface for the ArchiveBox ecosystem, balancing automated "one-shot" usage with granular control over plugins, dependencies, and configuration. Its event-driven UI ensures transparency during complex, multi-tool archival operations.