Block Synchronization Mechanisms

Introduction

Block synchronization in the gr-cuda project is designed to facilitate a high-performance, zero-copy GPU pipeline. The primary goal is to manage data movement and execution ordering between the CPU and GPU, as well as between successive GPU blocks, without requiring manual cudaMemcpy calls or blocking the CPU thread unnecessarily.

The system relies on CUDA streams and event-based synchronization. By utilizing custom buffers (cuda_buffer), the architecture ensures that GPU kernels run back-to-back while overlapping Host-to-Device (H2D) and Device-to-Host (D2H) transfers. This is achieved through a specific protocol of waiting for input readiness and signaling output completion.

Sources: README.md:10-15, include/gnuradio/cuda/cuda_block_helper.h:20-35

Core Architecture and Components

The synchronization framework is built upon three main pillars: the cuda_block base class, the cuda_buffer implementation, and the synchronization helper functions.

The cuda_block Base Class

Every GPU-enabled block inherits from cuda_block. This class manages the lifecycle of a non-blocking CUDA stream (d_stream), which is the primary execution context for all GPU kernels associated with that block.

This diagram shows the inheritance structure where a user-defined block combines GNU Radio's standard block logic with the cuda_block synchronization context.
Sources: include/gnuradio/cuda/cuda_block.h:35-55, include/gnuradio/cuda/cuda_block.h:112-132

Synchronization Helpers

Two critical inline functions in cuda_block_helper.h manage the GPU event state machine:

1. wait_for_work: Called before kernel launch. It ensures the GPU stream waits for input data readiness and confirms that output buffers are safe to overwrite (downstream consumers have finished reading).
2. mark_work_done: Called after kernel launch. It records events signaling that outputs are ready for downstream blocks and that inputs have been fully consumed.

Sources: include/gnuradio/cuda/cuda_block_helper.h:41-55, include/gnuradio/cuda/cuda_block_helper.h:80-95

Execution Flow and Event Logic

The synchronization logic prevents data hazards (Read-After-Write and Write-After-Read) using two primary event types: DEV_READY and READ_DONE.

Block Execution Cycle

The following flowchart illustrates the logical steps taken during a single work() or general_work() call to ensure proper synchronization.

The flow ensures that the kernel execution on the GPU stream is gated by the status of upstream and downstream dependencies.
Sources: include/gnuradio/cuda/cuda_block_helper.h:46-75, include/gnuradio/cuda/cuda_block.h:74-90

Event Mechanism Summary

Sources: include/gnuradio/cuda/cuda_block_helper.h:55-70, include/gnuradio/cuda/cuda_block_helper.h:95-110

Python and CuPy Synchronization

For Python-based blocks, synchronization is automated via a mixin and decorators found in python/cuda/cuda_block.py. This allows developers to use CuPy without manually managing CUDA events.

The @_wrap_work Decorator

The system wraps the Python work and general_work methods to inject synchronization calls. A key feature is the interception of the produce() call:

• If a block calls self.produce() explicitly, mark_work_done is triggered before the write pointer advances. This prevents downstream blocks from seeing a stale device-ready event.
• If the block returns a sample count (standard sync_block behavior), the wrapper calls mark_work_done automatically after the method returns.

This diagram illustrates how the Python wrapper manages the timing of synchronization relative to GNU Radio's buffer management.
Sources: python/cuda/cuda_block.py:40-85, python/cuda/cuda_block.py:92-105

Critical Requirements for general_work()

In blocks where produce() and consume() are called manually (typically basic_block or general_work), the timing of mark_work_done is critical.

• C++: gr::cuda::mark_work_done(detail(), d_stream) must be called before consume_each() and produce().
• Python: The cuda_block mixin handles this automatically by overriding the produce method of the block instance during the execution of work.

Sources: include/gnuradio/cuda/cuda_block_helper.h:97-105, python/cuda/cuda_block.py:53-65

Conclusion

The synchronization mechanisms in gr-cuda abstract the complexities of CUDA event management into a standardized "wait-then-signal" pattern. By integrating this logic into the cuda_block base class and the Python cuda_block mixin, the project enables high-throughput GPU processing with minimal boilerplate while maintaining the strict data integrity requirements of the GNU Radio scheduler.