Network Streaming (MJPEG, UDP)

Relevant source files

The following files were used as context for generating this wiki page:

• examples/mjpeg_server.py
• picamera2/encoders/mjpeg_encoder.py
• examples/mjpeg_server_2.py
• examples/mjpeg_server_with_rotation.py
• picamera2/encoders/libav_mjpeg_encoder.py
• picamera2/picamera2.py
• CHANGELOG.md
• picamera2/encoders/encoder.py

The picamera2 library provides robust capabilities for streaming video over a network. The primary methods supported and demonstrated through examples are Motion-JPEG (MJPEG) streaming over HTTP and, to a lesser extent, video streaming over UDP. MJPEG is particularly useful for creating simple web-based video feeds that are viewable in any modern browser without special plugins.

This document outlines the architecture and components involved in setting up network streams. It covers the different MJPEG encoders available, the server implementation for HTTP streaming, and the available information on UDP streaming.

MJPEG Streaming over HTTP

MJPEG streaming is achieved by setting up a simple Python HTTP server that continuously sends JPEG frames to the client. The picamera2 examples provide a clear and reusable architecture for this purpose.

Architecture

The MJPEG streaming server consists of several key components working together. The Picamera2 object captures frames, an Encoder compresses them into JPEGs, a custom StreamingOutput object buffers the latest frame, and a StreamingHandler serves these frames to web clients.

This diagram illustrates the high-level architecture of the MJPEG server.

Core Components

StreamingOutput

This class acts as a bridge between the camera's encoder and the HTTP server handler. It is an io.BufferedIOBase subclass that holds the most recent frame in memory. A threading.Condition is used to signal the StreamingHandler whenever a new frame is written by the encoder.

class StreamingOutput(io.BufferedIOBase):
    def __init__(self):
        self.frame = None
        self.condition = Condition()

    def write(self, buf):
        with self.condition:
            self.frame = buf
            self.condition.notify_all()

Sources: examples/mjpeg_server.py:27-36

StreamingHandler

This server.BaseHTTPRequestHandler subclass manages client connections.

• For the root path (/), it redirects to /index.html.
• For /index.html, it serves a basic HTML page with an <img> tag pointing to the stream.
• For /stream.mjpg, it sends a multipart/x-mixed-replace response. It then enters a loop, waiting on the StreamingOutput.condition. When a new frame is available, it writes the frame as a part of the multipart response.

This sequence diagram shows the interaction between a client, the StreamingHandler, and the StreamingOutput.

StreamingServer

A socketserver.ThreadingMixIn is used with server.HTTPServer to create a multi-threaded server capable of handling multiple streaming clients simultaneously.

Sources: examples/mjpeg_server.py:68-70

MJPEG Encoders

picamera2 offers several encoders that can produce MJPEG streams. The choice of encoder can depend on performance requirements and platform capabilities.

Encoder Class	Type	Basis	Key Characteristics	Relevant Files
MJPEGEncoder	Hardware	V4L2	Utilizes the hardware MJPEG encoder if available on the platform. Generally offers the best performance with low CPU usage.	picamera2/encoders/mjpeg_encoder.py, examples/mjpeg_server_2.py
JpegEncoder	Software	simplejpeg	A software-based encoder. It's used in the main mjpeg_server.py example.	examples/mjpeg_server.py, tests/quality_check.py
LibavMjpegEncoder	Software	libav	An alternative software encoder based on the av (PyAV) library, which wraps FFmpeg/libav. It expects yuv420p input format.	picamera2/encoders/libav_mjpeg_encoder.py

The following class diagram shows the inheritance structure of the available encoders.

Sources: picamera2/encoders/encoder.py, picamera2/encoders/mjpeg_encoder.py, picamera2/encoders/libav_mjpeg_encoder.py

Example Implementations

The repository provides several examples for MJPEG streaming:

• mjpeg_server.py: The primary example, using the software JpegEncoder.
• mjpeg_server_2.py: A variation that uses the hardware MJPEGEncoder for better performance.
• mjpeg_server_with_rotation.py: Demonstrates how to handle 90 or 270-degree rotation. It achieves this by injecting an EXIF orientation tag into the JPEG header of each frame. This allows browsers to correctly orient the image.

Here is the snippet that injects the EXIF header for rotation:

# examples/mjpeg_server_with_rotation.py:12-28
ROTATION = 270  # Use 0, 90 or 270
WIDTH = 640
HEIGHT = 480

rotation_header = bytes()
if ROTATION:
    WIDTH, HEIGHT = HEIGHT, WIDTH
    code = 6 if ROTATION == 90 else 8
    exif_bytes = piexif.dump({'0th': {piexif.ImageIFD.Orientation: code}})
    exif_len = len(exif_bytes) + 2
    rotation_header = bytes.fromhex('ffe1') + exif_len.to_bytes(2, 'big') + exif_bytes

# ...

class StreamingOutput(io.BufferedIOBase):
    def write(self, buf):
        with self.condition:
            self.frame = buf[:2] + rotation_header + buf[2:]
            self.condition.notify_all()

Sources: examples/mjpeg_server_with_rotation.py:12-28

UDP Streaming

The picamera2 library also supports streaming over UDP, although it is less prominently featured in the examples compared to MJPEG/HTTP.

The primary evidence for this feature is in the project's changelog, which states:

Large encoded video frames are now split automatically for streaming over UDP.

Sources: CHANGELOG.md:275-276

This functionality is likely handled by the FfmpegOutput class, which can be configured to stream to a UDP endpoint. The start_and_record_video method in the main Picamera2 class uses FfmpegOutput for container formats like .mp4 and .ts, which are suitable for streaming. A user could presumably pass a UDP URL (e.g., udp://<ip_address>:<port>) to FfmpegOutput to initiate a UDP stream.

The automatic splitting of large frames is a crucial feature for UDP, as UDP packets have a size limit, and single video frames (especially keyframes) can often exceed this limit.

Sources: picamera2/picamera2.py:2020-2027

Summary

picamera2 provides flexible options for network video streaming. For simple, universally compatible web-based viewing, the MJPEG-over-HTTP server examples offer a complete, high-performance solution using either hardware or software encoders. For other use cases, such as custom client/server applications, the library appears to support UDP streaming, likely via FfmpegOutput, with built-in handling for large frame fragmentation.