CLI Reference: Find & Grab
Relevant source files
The following files were used as context for generating this wiki page:
ProxyBroker2 provides two primary command-line interfaces for discovering proxies: find and grab. While both commands utilize the project's asynchronous engine to extract proxy data from over 50 sources, they serve distinct operational needs based on whether validation is required.
The find command identifies, checks, and filters proxies to ensure they are active and meet specific criteria before presentation. In contrast, the grab command prioritizes speed by scraping and returning proxies immediately without performing health checks or anonymity validation. Both commands support various output formats, including standard text and JSON.
Sources: README.md:20-22, CLAUDE.md:148-150, tests/test_cli.py:53-73
The find Command
The find command executes a full pipeline: discovery, validation, and filtering. It uses a Checker to verify proxy status and detect anonymity levels against judge servers.
Find Workflow
The following diagram illustrates the data flow when a user executes the find command.
The find command coordinates between the Broker and the Checker to ensure only functional proxies reach the user.
Sources: CLAUDE.md:76-80, tests/test_integration.py:27-46
Configuration Options for Find
The find command requires the --types argument and supports various filters.
| Parameter | Type | Description |
|---|---|---|
--types | List | Required. Proxy protocols: HTTP, HTTPS, SOCKS4, SOCKS5, CONNECT:80, CONNECT:25. |
--limit | Integer | Maximum number of proxies to find. Default is 100. |
--countries | List | Filter proxies by ISO country codes (e.g., US, GB). |
--lvl | String | Anonymity level: Transparent, Anonymous, High. |
--strict | Flag | Forces strict matching of protocols and anonymity levels. |
--format | String | Output format: default (text) or json. |
Sources: README.md:105-110, tests/test_cli.py:56-61, tests/test_integration.py:86-105
The grab Command
The grab command provides a "raw" discovery mode. It skips the Checker component entirely, resulting in significantly faster execution at the expense of proxy reliability.
Grab Sequence
This sequence diagram shows the simplified interaction during a grab operation.
In this mode, the Broker initiates discovery but bypasses the Checker initialization.
Sources: tests/test_integration.py:126-140, CLAUDE.md:148-150
Configuration Options for Grab
| Parameter | Type | Description |
|---|---|---|
--outfile | Path | File path to save the found proxies. |
--countries | List | Filters discovery to specific countries. |
--limit | Integer | Number of raw proxies to extract. |
--format | String | Defines presentation: default or json. |
Sources: README.md:115-117, tests/test_cli.py:65-73
Global CLI Configuration
Both commands share global parameters defined at the top level of the proxybroker entry point. These parameters control the underlying Broker behavior and resource utilization.
# Example of global parameter propagation in Broker initialization
broker = Broker(
timeout=args.timeout,
max_conn=args.max_conn,
max_tries=args.max_tries,
verify_ssl=args.verify_ssl
)
Sources: tests/test_integration.py:165-175, proxybroker/cli.py
Shared Execution Parameters
--timeout: Sets the connection timeout in seconds for finding and checking proxies.--max-conn: Limits the maximum number of concurrent connections.--max-tries: Defines the number of attempts to check a proxy before discarding it.--provider-dir: Specifies custom directories containing YAML/JSON provider configurations.--log: Sets the logging level (DEBUG, INFO, WARNING, ERROR).
Sources: tests/test_cli.py:165-188, CLAUDE.md:175-182
Custom Provider Integration
Users can extend the discovery capabilities of both find and grab by providing custom source files. The CLI resolves these directories using a specific priority logic.
The _resolve_provider_dirs function ensures that CLI flags take the highest priority, followed by environment variables, and finally the Docker-standard /configs directory.
Sources: tests/test_cli.py:192-233, CLAUDE.md:175-180
Summary
The find and grab commands represent the two primary modes of operation for ProxyBroker2's CLI. find serves users requiring validated, high-quality proxies for immediate use, while grab serves those collecting large volumes of raw proxy data for external processing. Both commands benefit from modern async implementation, allowing for high concurrency and deterministic protocol prioritization.