The Broker: Core Orchestrator

Relevant source files

The following files were used as context for generating this wiki page:

• proxybroker/api.py
• proxybroker/cli.py
• README.md
• CLAUDE.md
• tests/test_api.py
• proxybroker/server.py
• proxybroker/init.py

The Broker class manages the high-level coordination of the ProxyBroker2 pipeline. It discovery, validates, and serves public proxies by orchestrating specialized components such as Providers, Checkers, and the ProxyPool. The Broker provides the primary developer interface for three main operations: finding validated proxies, grabbing raw proxy data, and running a rotating proxy server.

Sources: proxybroker/api.py, CLAUDE.md:46-49

Core Architecture and Workflows

The Broker acts as the central hub for async operations. It initializes an asynchronous environment to handle concurrent tasks, including network requests to proxy providers and validation checks against judge servers.

The diagram shows how the Broker directs user requests to specific internal components based on the selected method.
Sources: proxybroker/api.py, CLAUDE.md:46-72, README.md:214-230

Initialization and Configuration

The Broker constructor accepts several parameters to tune the discovery and validation process. If a user does not provide an explicit asyncio.Queue, the Broker creates one internally to facilitate communication between providers and consumers.

Parameter	Type	Default	Description
queue	asyncio.Queue	None	Queue for discovered/validated proxies.
timeout	int	8	Request timeout in seconds.
max_conn	int	200	Max concurrent proxy checks.
max_tries	int	3	Max attempts to check a single proxy.
judges	list	None	List of judge server URLs.
providers	list	None	List of proxy provider URLs/instances.
verify_ssl	bool	False	Whether to verify SSL certificates.

Sources: proxybroker/api.py, proxybroker/cli.py:108-142, tests/test_api.py:31-59

Primary Methods

find()

The find() method performs an end-to-end discovery and validation process. It fetches proxies from providers and immediately passes them to the Checker for validation against protocols like HTTP, HTTPS, SOCKS4, and SOCKS5. Validated proxies that meet criteria (anonymity level, country, etc.) are placed into the result queue.

Sources: proxybroker/api.py, README.md:214-230, tests/test_api.py:75-87

grab()

The grab() method facilitates fast discovery by bypassing the validation stage. It extracts proxy metadata from providers and places them directly into the queue. This method is used when raw speed is prioritized over proxy reliability or anonymity verification.

Sources: proxybroker/api.py, proxybroker/cli.py:84-95, tests/test_api.py:60-74

serve()

The serve() method launches a local proxy server. It utilizes a ProxyPool to manage a heap-based priority queue of proxies, distributing incoming client requests to external proxies based on performance metrics like average response time.

Sources: proxybroker/api.py, proxybroker/server.py, tests/test_api.py:102-125, proxybroker/cli.py:97-124

Execution Flow

The following sequence diagram illustrates the lifecycle of a find() operation coordinated by the Broker.

Resource Management and Cleanup

The Broker includes mechanisms to prevent memory leaks and ensure clean shutdowns. The stop() method removes signal handlers and cancels pending tasks. In CLI mode, the Broker handles KeyboardInterrupt to trigger a graceful exit, ensuring that network connections and background tasks are terminated properly.

# Signal Handler Cleanup from Broker.stop() logic
def stop(self):
    """Properly removes signal handlers to prevent memory leaks."""
    if self._loop and self._stop_broker_on_sigint:
        try:
            self._loop.remove_signal_handler(signal.SIGINT)
            self._loop.remove_signal_handler(signal.SIGTERM)
        except (NotImplementedError, RuntimeError):
            pass
    # ... task cancellation logic ...

Sources: proxybroker/api.py, CLAUDE.md:144-145, tests/test_api.py:155-165

Conclusion

The Broker serves as the high-level interface of ProxyBroker2. It abstracts the complexity of concurrent provider fetching and proxy validation into a simple set of asynchronous methods. By managing internal components like the Checker and ProxyPool, it ensures that users receive reliable, filtered proxy data for both scripting and server-based applications.