CUDA Buffer Management

Relevant source files

The following files were used as context for generating this wiki page:

• include/gnuradio/cuda/cuda_buffer.h
• include/gnuradio/cuda/cuda_block.h
• include/gnuradio/cuda/cuda_block_helper.h
• python/cuda/cuda_block.py
• README.md
• python/cuda/qa_cuda_buffer_compaction.py

CUDA Buffer Management in gr-cuda provides a high-performance, zero-copy pipeline for GPU-accelerated signal processing within the GNU Radio framework. It abstracts complex memory management and synchronization tasks, allowing data to move automatically between the host (CPU) and device (GPU) without requiring manual cudaMemcpy calls within processing blocks.

The system leverages the CUDA Virtual Memory Management (VMM) API to implement hardware-accelerated circular buffers on the GPU, mirroring the double-mapping technique GNU Radio uses for CPU buffers. This ensures that GPU kernels can always access a contiguous span of data, even when the data wraps around the physical memory boundary, eliminating the need for buffer compaction.

Sources: README.md:9-13, include/gnuradio/cuda/cuda_buffer.h:32-47

Architecture and Memory Model

The cuda_buffer class inherits from gr::buffer_single_mapped but replaces standard linear allocations with double-mapped virtual address ranges for both the host and the device.

Double-Mapping Mechanism

• Device Buffer: Uses the CUDA VMM API (cuMemAddressReserve, cuMemCreate, cuMemMap). A virtual address space of size $2N$ is reserved, where $N$ is the physical buffer size. The physical memory is mapped twice into this range, making any contiguous span up to $N$ items valid regardless of the starting index.
• Host Buffer: Implemented using POSIX mmap with memfd_create for double-mapping on the CPU side. These pages are pinned via cudaHostRegister to enable asynchronous DMA transfers.

Sources: include/gnuradio/cuda/cuda_buffer.h:35-43, include/gnuradio/cuda/cuda_buffer.h:122-127

Buffer Relationships

The following diagram illustrates the relationship between the cuda_buffer and the underlying ring management classes.

This diagram shows how cuda_buffer extends the standard GNU Radio buffer to include GPU-specific ring buffers and synchronization events.
Sources: include/gnuradio/cuda/cuda_buffer.h:80-170

Synchronization Model

The management system uses three primary CUDA events to coordinate overlap between producers and consumers. This model prevents data hazards (Read-After-Write and Write-After-Read) without blocking the CPU thread unnecessarily.

Event	Purpose	Triggered By
Device-Ready	Signals that the producer has finished writing data to the GPU memory.	mark_device_ready()
Host-Ready	Signals that an asynchronous Device-to-Host (D2H) copy has completed.	mark_host_ready()
Read-Done	Tracks when all consumers have finished reading so the producer can overwrite.	mark_read_done()

Sources: include/gnuradio/cuda/cuda_buffer.h:54-62, include/gnuradio/cuda/cuda_block_helper.h:74-84

Block Synchronization Flow

The synchronization logic is typically handled by wait_for_work() and mark_work_done() helpers.

This flow ensures that kernels do not read stale data and do not overwrite data currently in use by downstream blocks.
Sources: include/gnuradio/cuda/cuda_block_helper.h:32-105

Integration with GNU Radio Blocks

gr-cuda provides base classes for both C++ and Python to simplify buffer management.

C++ Integration

C++ blocks should inherit from gr::cuda::cuda_block. This class provides a managed d_stream (a non-blocking CUDA stream) and members for kernel launch configurations (d_block_size, d_min_grid_size).

int my_block_impl::work(int noutput_items, ...) {
    gr::cuda::wait_for_work(detail(), d_stream);
    // ... kernel launch ...
    gr::cuda::mark_work_done(detail(), d_stream);
    return noutput_items;
}

Sources: include/gnuradio/cuda/cuda_block.h:35-85

Python (CuPy) Integration

Python blocks inherit from cuda.sync_block, cuda.decim_block, cuda.interp_block, or cuda.basic_block. The Python wrapper automatically:

1. Allocates CUDA buffers via the block's IO signature.
2. Performs GPU-side waits before calling work().
3. Converts raw pointers to CuPy arrays for zero-copy access.
4. Records completion events after work() returns.

Sources: python/cuda/cuda_block.py:46-95, README.md:52-65

Configuration and Tuning

CPU Synchronization Mode

The system supports two modes for CPU-side event synchronization (e.g., when the host needs to wait for the GPU to finish a transfer).

Option	Parameter	Behavior	Benefit
Busy Wait	gpu_busy_wait = true	CPU spin-polls the CUDA event.	Lowest latency.
Blocking Sync	gpu_busy_wait = false	CPU thread sleeps until event.	Lower CPU usage; frees cores.

Configuration is set in the GNU Radio user preferences (e.g., ~/.config/gnuradio/config.conf) under the [cuda_buffer] section.
Sources: include/gnuradio/cuda/cuda_buffer.h:64-77

Conclusion

CUDA Buffer Management in gr-cuda eliminates the typical overhead of GPU data handling in GNU Radio. By using VMM-based circular buffers, it removes the need for data compaction and manual memory management. The event-based synchronization model allows for significant overlap between computation and data transfer, maximizing GPU throughput while maintaining thread safety across the flowgraph.

Sources: README.md:14-18, include/gnuradio/cuda/cuda_buffer.h:45-47