Image & Video Media Support

Relevant source files

The following files were used as context for generating this wiki page:

• kaleidux-daemon/src/video.rs
• kaleidux-daemon/src/main.rs
• kaleidux-daemon/src/queue.rs
• kaleidux-daemon/src/cache.rs
• kaleidux-daemon/src/metrics.rs
• config.example.toml

Kaleidux provides a high-performance media handling system designed to render both static images and looping video content as desktop wallpapers. The system leverages hardware acceleration through WGPU for rendering and GStreamer for video decoding, ensuring low CPU overhead and smooth visual transitions. Media support is divided into distinct pipelines for image decoding and video playback, coordinated by an orchestration layer that manages monitor-specific queues.

Sources: README.md:20-25, kaleidux-daemon/src/main.rs:188-195

Media Discovery and Identification

Kaleidux uses a parallelized discovery system to identify compatible media files within configured directories. Files are validated by checking their magic bytes rather than just extensions, ensuring robustness against mislabeled files.

Content Type Detection

The system classifies media into two primary categories: Image and Video.

Content Type	Magic Bytes / Headers Identified
Image	JPEG (FF D8 FF), PNG (89 50 4E 47), GIF (GIF8), WebP (RIFF...WEBP)
Video	EBML/MKV/WebM (1A 45 DF A3), MP4 (ftyp header)

Sources: kaleidux-daemon/src/queue.rs:114-142

Discovery Pipeline

The discovery process is offloaded to a background thread pool using jwalk for parallel directory traversal. To optimize subsequent startups, file metadata (mtime, size, and type) is stored in a redb database.

Image Processing Architecture

Static images are handled via a specialized decoding pipeline that prevents memory exhaustion during high-resolution asset loading.

Concurrent Decode Limiting

Because 4K RGBA images can occupy 35-40MB of RAM each, the system utilizes a global semaphore (IMAGE_DECODE_SEMAPHORE) to limit concurrent decodes to a maximum of two tasks. This prevents memory spikes when multiple monitors update simultaneously.

Image Pipeline Flow

1. Request: The main loop triggers an image load for a specific output.
2. Semaphore: The task waits for a permit from the semaphore.
3. Blocking Decode: The image crate opens the file and converts it to raw RGBA8 format within a tokio::task::spawn_blocking block.
4. Channel Transmission: The LoadedImage struct is sent through a bounded channel (limit 6) to the renderer.
5. GPU Upload: The renderer calls upload_image_data to transfer the buffer to a GPU texture.

Sources: kaleidux-daemon/src/main.rs:21-25, kaleidux-daemon/src/main.rs:88-154

Video Playback System

Video support is implemented through a GStreamer-based integration that provides seamless looping and volume control.

Video Player Lifecycle

The VideoPlayer manages a GStreamer pipeline. When a video is selected, a new session ID is generated to prevent stale frames from previous videos from being rendered during transitions.

Performance and Memory Management

• Frame Throttling: The main loop processes only the latest frame per source to prevent frame accumulation and memory leaks.
• CPU-RGBA Roundtrip: Currently, video frames are decoded on the CPU and uploaded as RGBA buffers.
• Aggressive Cleanup: A texture pool cleanup runs every 3 seconds to reclaim GPU memory from inactive video sessions.

Sources: kaleidux-daemon/src/main.rs:650-680, kaleidux-daemon/src/metrics.rs:320-335

Media Configuration

Users can control the balance and behavior of media types via the config.toml file.

Parameter	Type	Default	Description
video-ratio	Integer	50	Probability (0-100) of picking a video over an image.
volume	Float	1.0	Audio volume for video wallpapers (0.0 to 10.0).
duration	String	"5m"	How long a media item stays active before cycling.
path	Path	N/A	Directory containing the image/video assets.

Sources: config.example.toml:22-43, kaleidux-daemon/src/queue.rs:56-65

Metrics and Monitoring

The PerformanceMetrics system tracks the efficiency of media operations to identify bottlenecks or leaks.

• Video First Frame Time: Tracks the latency between requesting a video and the first frame appearing on screen.
• Component CPU Tracking: Separates CPU time spent in video_cpu_time (decoding/processing) versus renderer_cpu_time (GPU interaction).
• Channel Tracking: Monitors frame_channel_size_samples and image_channel_size_samples to detect backpressure in the media pipeline.

Sources: kaleidux-daemon/src/metrics.rs:18-50, kaleidux-daemon/src/metrics.rs:160-185

Conclusion

The Image & Video Media Support in Kaleidux is a dual-track system that prioritizes stability and performance. By utilizing concurrent decode limits for images and a dedicated GStreamer backend for videos, the project achieves high-fidelity wallpaper rendering without compromising system resources. The integration of caching and metrics ensures that the media pipeline remains efficient even with large asset libraries or multi-monitor configurations.