Event Lifecycle

Relevant source files

The following files were used as context for generating this wiki page:

• abx_dl/events.py
• abx_dl/cli.py
• abx_dl/orchestrator.py
• README.md
• TODOs.md
• server/server.py

The Event Lifecycle in abx-dl is a structured, asynchronous sequence of phases managed by the abxbus event bus. It governs how the system resolves dependencies, initializes crawl environments, extracts content via snapshot hooks, and cleans up background processes. This lifecycle ensures that plugins can run concurrently as daemons while maintaining a strict execution order for core stages.

Sources: abx_dl/events.py:1-24, README.md:129-140

Execution Phases Overview

The lifecycle is divided into two primary root phases: the Install Phase and the Crawl Phase. Each phase is triggered by a root event and decomposes into specific child events and hook executions.

Phase Order Diagram

The following flowchart illustrates the high-level transitions between phases:

---

1. Install Phase

The Install Phase is the orchestrator-only preflight stage responsible for resolving binary dependencies required by plugins.

• InstallEvent: Emitted by the orchestrator before any crawling begins. It triggers the BinaryService to read config.json > required_binaries for all enabled plugins.
• BinaryRequestEvent: For every required binary, a request is emitted. This event is handled by provider plugins (e.g., apt, pip, npm, brew) via on_BinaryRequest__* hooks.
• BinaryEvent: Once a provider successfully resolves or installs a binary, it emits a BinaryEvent containing the absolute path (abspath), version, and other metadata.

Sources: abx_dl/events.py:72-83, abx_dl/events.py:276-320, TODOs.md:16-18

---

2. Crawl & Snapshot Lifecycle

The Crawl Phase represents the core runtime of the application, managing the transition from environment setup to actual content extraction.

Crawl Lifecycle Events

Event	Purpose	Source
CrawlEvent	The root lifecycle event that kicks off setup, start, and cleanup.	abx_dl/events.py:86-94
CrawlSetupEvent	Runs on_CrawlSetup__* hooks to launch daemons and shared runtime state.	abx_dl/events.py:97-106
CrawlStartEvent	Marks the end of setup and triggers the SnapshotEvent.	abx_dl/events.py:109-118
CrawlCleanupEvent	Sends SIGTERM to all background crawl daemons.	abx_dl/events.py:121-128
CrawlCompletedEvent	Final informational event indicating the lifecycle has ended.	abx_dl/events.py:131-138

Snapshot Sequence

Within the CrawlStartEvent, the system initiates the Snapshot Lifecycle. This is where the actual extraction of URLs occurs.

---

3. Process Execution & Feedback

Subprocess management is handled via ProcessEvent. This system decouples the execution of external tools from the main event bus logic.

• ProcessEvent: Handled by ProcessService. It uses event_concurrency='parallel' to allow background hooks (daemons) to run concurrently with the main serial handler chain.
• ProcessStdoutEvent: Emitted for every line of stdout produced by a hook. The SnapshotService or BinaryService parses these lines to translate JSONL records into typed events like ArchiveResultEvent or BinaryEvent.
• ProcessCompletedEvent: Signals that a hook has exited. It carries the exit code, stdout/stderr, and a list of output_files produced.

Sources: abx_dl/events.py:210-235, abx_dl/events.py:261-273, abx_dl/cli.py:596-650

---

4. Control & Interactive Lifecycle

The lifecycle supports interactive interruptions such as pausing or aborting, typically triggered by user signals (SIGINT).

Control Event	Behavior
CrawlPauseEvent	Requests interruption of the current foreground hook and pauses the bus.
CrawlAbortEvent	Aborts the entire crawl after the current hook finishes.
CrawlResumeAndRetryEvent	Resumes by retrying the interrupted foreground hook.
CrawlResumeAndSkipEvent	Resumes but skips the interrupted hook.

Sources: abx_dl/events.py:141-160, abx_dl/cli.py:821-837

Summary

The abx-dl event lifecycle provides a robust framework for multi-phase execution. By utilizing specialized events for installation, setup, and snapshotting, the system ensures that dependencies are satisfied before execution, daemons are correctly managed as background processes, and results are captured through a standardized stdout-to-event routing mechanism.

Sources: abx_dl/events.py:1-45, README.md:129-140