Flow Control & Routing Blocks

Relevant source files

The following files were used as context for generating this wiki page:

• include/gnuradio/cuda/throttle.h
• include/gnuradio/cuda/copy.h
• include/gnuradio/cuda/nop.h
• include/gnuradio/cuda/tee.h
• include/gnuradio/cuda/cuda_block_helper.h
• README.md
• python/cuda/qa_nop.py
• python/cuda/qa_tee.py

Flow Control and Routing blocks within the gr-cuda ecosystem are specialized primitives designed to manage the movement, rate, and distribution of data streams across the GPU-accelerated pipeline. These blocks facilitate essential operations such as rate limiting, data duplication, and simple passthrough/copying while maintaining the efficiency of the cuda_buffer zero-copy architecture.

These blocks act as the "glue" in a complex signal processing chain, ensuring that data is correctly synchronized between host (CPU) and device (GPU) memory without manual cudaMemcpy calls. By utilizing standard GNU Radio scheduling alongside CUDA-specific event synchronization, these blocks allow for overlapping data transfers and kernel executions.
Sources: README.md:3-8, include/gnuradio/cuda/cuda_block_helper.h:18-35

Core Flow Control Architecture

Flow control blocks in gr-cuda differ from standard GNU Radio blocks by incorporating GPU-side event synchronization. This prevents data hazards such as reading stale data or overwriting buffers currently in use by the GPU.

Synchronization Mechanism

Every flow control block typically implements a two-stage synchronization pattern using helper functions:

1. Wait for Work: Before any data is processed, the block calls wait_for_work(). This ensures input buffers are ready on the device and output buffers are safe to write by checking device-ready and read-done events.
2. Mark Work Done: After the processing (or routing) is complete, the block calls mark_work_done(). This records a DEV_READY event for downstream consumers and a READ_DONE event for upstream producers.

The diagram shows the standard execution flow for a CUDA-enabled routing block, ensuring host-device synchronization without blocking the CPU thread.
Sources: include/gnuradio/cuda/cuda_block_helper.h:37-65, include/gnuradio/cuda/cuda_block_helper.h:87-110

Specific Routing Blocks

Tee Block

The tee block is the primary routing primitive for duplicating a single input stream into multiple output streams. It supports both CPU-based and GPU-based duplication. When gpu=true, the block operates on CUDA device pointers, allowing for efficient data branching within the GPU pipeline.

• Class: gr::cuda::tee
• Functionality: Replicates n items from one input to M outputs.
• Parameters:
  • itemsize: Size of a single stream item in bytes.
  • gpu: Boolean flag to determine if the block should use CUDA-managed buffers.
    Sources: include/gnuradio/cuda/tee.h:35-50, python/cuda/qa_tee.py:22-35

Throttle Block

The throttle block controls the flow of samples to match a specific average rate (samples per second). This is critical in simulations where the processing speed of the GPU might otherwise exceed the intended sample rate of the system.

• Class: gr::cuda::throttle
• Key Methods:
  • set_sample_rate(double rate): Updates the target throughput.
  • sample_rate(): Retrieves the current rate.
    Sources: include/gnuradio/cuda/throttle.h:35-52

Copy and Nop Blocks

The copy and nop (No-Operation) blocks are used for testing, benchmarking, and explicit data movement.

• Copy: Moves data from input to output. In a CUDA context, if the input is host-side and output is device-side, it triggers a Host-to-Device (H2D) transfer.
• Nop: Inherits from sync_block and effectively does nothing in its work function, serving as a baseline for measuring the overhead of the cuda_buffer management system and H2D/D2H transfers.

This diagram illustrates how a simple nop or copy block exercises the underlying cuda_buffer transfer logic between Host and Device.
Sources: include/gnuradio/cuda/copy.h:34-47, include/gnuradio/cuda/nop.h:34-47, python/cuda/qa_nop.py:46-55

Block Configuration Summary

The following table summarizes the routing and flow control blocks available in the gr-cuda module:

Block Name	Primary Purpose	Key Parameter	Data Domain
copy	Data movement	itemsize	CPU/GPU
nop	Benchmarking overhead	itemsize	CPU/GPU
tee	Stream duplication	gpu (boolean)	CPU or GPU
throttle	Execution rate limiting	samples_per_sec	CPU/GPU
null_source	Generates empty stream	itemsize	GPU
null_sink	Consumes and discards	itemsize	GPU

Sources: include/gnuradio/cuda/copy.h:47, include/gnuradio/cuda/nop.h:47, include/gnuradio/cuda/tee.h:50, include/gnuradio/cuda/throttle.h:52

Implementation Details

Routing blocks leverage the std::dynamic_pointer_cast<gr::cuda_buffer> to check if the buffers connected to their ports are CUDA-managed. If they are, they utilize the CUDA stream associated with the block to perform asynchronous operations.

In general_work() implementations for routing blocks, mark_work_done() must be called before the GNU Radio produce() and consume() methods. This is because produce() immediately advances the write pointer, and for Device-to-Device (D2D) buffers, failing to mark completion beforehand would lead downstream consumers to see stale synchronization events.
Sources: include/gnuradio/cuda/cuda_block_helper.h:75-85, include/gnuradio/cuda/cuda_block_helper.h:92-98

Flow control and routing blocks provide the necessary infrastructure to build complex, multi-stage GPU processing pipelines. By abstracting the complexities of CUDA event synchronization into simple wait and mark calls, gr-cuda allows these blocks to maintain high-performance data movement without introducing race conditions or manual memory management overhead.