Home
Relevant source files
The following files were used as context for generating this wiki page:
Picamera2 is a Python library for interfacing with cameras on the Raspberry Pi, specifically those supported by the libcamera framework. It serves as a replacement for the original Picamera library, which was designed for the legacy camera stack. Picamera2 provides a user-friendly, high-level API that simplifies camera configuration, previewing, and capturing images and videos. It is designed to be versatile, supporting simple command-line operations as well as integration into complex GUI applications and machine learning pipelines.
The library is pre-installed on Raspberry Pi OS images (Bullseye or later) but not on Lite images. It is compatible with all Raspberry Pi boards, though performance may vary on less powerful devices. The core of the library is the Picamera2 class, which manages camera initialization, configuration, and the capture lifecycle. It handles interactions with libcamera asynchronously, using an event loop to process completed capture requests.
Sources: README.md:8-10, README.md:20-22, picamera2/picamera2.py:110-112
Installation and Dependencies
The recommended method for installing Picamera2 is through the apt package manager, as this ensures that the library and its underlying libcamera dependencies are compatible.
- Full Installation (with GUI components):
sudo apt install python3-picamera2 - Reduced Installation (for Lite OS):
sudo apt install python3-picamera2 --no-install-recommends
The library relies on several Python packages, which are defined in the setup.py file.
| Dependency | Description |
|---|---|
numpy | Core library for numerical and array operations. |
Pillow | Python Imaging Library fork for image manipulation. |
simplejpeg | Fast JPEG encoding. |
piexif | For reading and writing EXIF data in JPEGs. |
v4l2-python3 | Python bindings for the Video4Linux2 API. |
python-prctl | A Python wrapper for the prctl system call. |
av | Python bindings for FFmpeg libraries. |
pyopengl, PyQt5 | Optional dependencies for GUI previews (gui extra). |
Sources: README.md:29-39, setup.py:29-32
Core Architecture
The Picamera2 architecture is centered around the main Picamera2 class, which orchestrates camera operations. It uses a CameraManager to handle libcamera events and an event loop, typically driven by a Preview object, to process capture requests.
Sources: picamera2/picamera2.py:110-112
The Picamera2 Class
This is the main class and the primary entry point for all camera operations. Its responsibilities include:
- Initializing and closing the camera.
- Creating and applying camera configurations.
- Starting and stopping the camera stream and preview.
- Dispatching capture requests and managing encoders.
A typical lifecycle involves creating an instance, configuring it, starting it, performing captures, and then stopping and closing it.
This diagram shows the main Picamera2 class and its relationship with key components like CameraManager, CompletedRequest, Encoder, and Preview.
Sources: picamera2/picamera2.py
Camera Manager and Event Loop
Picamera2 uses a CameraManager singleton to manage all camera instances. The manager runs a background thread that listens for events from libcamera's event file descriptor. When a request is completed, the CameraManager adds it to a queue for the corresponding Picamera2 instance and notifies the main event loop.
The event loop is typically driven by a Preview object (even a NullPreview). It waits for notifications and then calls picam2.process_requests(), which handles completed frames, executes pending jobs (like captures), and passes frames to encoders.
The following diagram illustrates the asynchronous event handling flow.
This sequence shows how a frame completion event from libcamera is handled by the CameraManager and processed by the Picamera2 instance within the event loop driven by the Preview object.
Sources: picamera2/picamera2.py:127-183, picamera2/previews/null_preview.py:16-30, picamera2/previews/q_gl_picamera2.py:155-158
Camera Configuration
Before starting the camera, it must be configured. A configuration is a Python dictionary that defines stream properties, camera controls, and other parameters. Picamera2 provides helper methods to create standard configurations.
Configuration Helpers
create_preview_configuration(): For live preview, typically with aXBGR8888format.create_still_configuration(): For high-resolution still captures, often using the full sensor resolution andBGR888format.create_video_configuration(): For video recording, commonly at 720p or 1080p.
These helpers return a dictionary that can be modified before being passed to picam2.configure().
Sources: picamera2/picamera2.py:821-974
Configuration Structure
A configuration dictionary is structured with keys for different streams and general settings.
| Key | Type | Description |
|---|---|---|
main | dict | Configuration for the main, high-resolution stream. |
lores | dict or None | Optional low-resolution YUV stream, useful for ML or secondary outputs. |
raw | dict or None | Optional raw sensor data stream. |
transform | libcamera.Transform | Image orientation (e.g., horizontal/vertical flip). |
colour_space | libcamera.ColorSpace | The color space for the output images (e.g., Sycc, Rec709). |
buffer_count | int | Number of frame buffers to allocate. Higher counts help avoid frame drops. |
controls | dict | A dictionary of control values to apply when the camera starts. |
display | str | The name of the stream to show in the preview window (e.g., "main", "lores"). |
encode | str | The name of the stream to send to the video encoder. |
queue | bool | If True, picamera2 keeps the last frame available for quick captures. |
Sources: picamera2/picamera2.py:821-829, CHANGELOG.md:144-145
Applying a Configuration
The configure() method validates the configuration, translates it into a libcamera configuration, and applies it to the camera.
This flowchart shows the process of creating, optionally modifying, and applying a camera configuration.
Sources: picamera2/picamera2.py:1082-1188
Capturing Data
Once the camera is started, you can capture frames, which are delivered as CompletedRequest objects.
The CompletedRequest Object
This object is a wrapper around a completed libcamera request. It is the primary way to access image data and metadata. It is reference-counted and must be release()d when no longer needed to return its buffers to the camera system.
Key methods include:
make_array(name): Returns a NumPy array of the specified stream (e.g., "main", "lores").make_image(name): Returns a PillowImageobject.get_metadata(): Returns a dictionary of the frame's metadata (e.g.,ExposureTime,AnalogueGain).save(name, file_output): Saves the specified stream to a file, typically as a JPEG.save_dng(file_output): Saves the raw stream as a DNG file.
Sources: picamera2/request.py:44-245
Capture Methods
Picamera2 provides several methods to capture data, supporting both synchronous (blocking) and asynchronous (non-blocking) operations.
| Method | Description | Return (Sync) | Return (Async) |
|---|---|---|---|
capture_file(file, ...) | Captures a frame and saves it to a file. | dict (metadata) | Job |
capture_array(name, ...) | Captures a frame and returns it as a NumPy array. | np.ndarray | Job |
capture_image(name, ...) | Captures a frame and returns it as a Pillow Image. | Image.Image | Job |
capture_request(...) | Captures a raw CompletedRequest object for low-level access. Must be released. | CompletedRequest | Job |
switch_mode_and_capture_file(...) | Switches to a temporary capture mode, captures a file, and switches back. | dict (metadata) | Job |
For asynchronous calls (wait=False), the method returns a Job object. The result can be retrieved later by calling picam2.wait(job).
Sources: picamera2/picamera2.py:1413-1786
Asynchronous Capture Flow
This diagram illustrates how an asynchronous capture request is handled, returning a Job object immediately and delivering the result later.
Sources: picamera2/picamera2py:1366-1390
Video Encoding
Picamera2 supports video recording through a flexible encoder system. You can use built-in encoders or create your own.
- Encoders:
H264EncoderandMJPEGEncoderare provided. On Pi 5,Libavencoders are used. - Outputs:
FileOutputwrites to a file, whileFfmpegOutputcan be used to create standard containers like.mp4and include audio. - Starting/Stopping:
start_encoder(encoder, output): Attaches and starts an encoder.stop_encoder(): Stops and detaches the encoder.start_recording(...): A high-level convenience function that configures, starts the camera, and starts the encoder.stop_recording(): Stops the camera and the encoder.
The full application in apps/app_full.py demonstrates starting and stopping an H264Encoder to record an .mp4 file using FfmpegOutput.
Sources: picamera2/picamera2.py:1788-1875, apps/app_full.py:168-192
Device-Specific Features
The library can be extended with helper classes for specific camera modules or devices, such as the IMX500 AI sensor. The IMX500 class in picamera2/devices/imx500/imx500.py demonstrates this by:
- Loading a neural network model (
.rpkfile) onto the device. - Configuring the inference ROI (Region of Interest).
- Parsing output tensors from the camera's metadata.
This modular approach allows for specialized hardware features to be exposed through the Python API without cluttering the core Picamera2 class. The imx500_segmentation_demo.py example shows how to use this helper to run a segmentation model and overlay the results on the preview.
Sources: picamera2/devices/imx500/imx500.py, examples/imx500/imx500_segmentation_demo.py
Conclusion
Picamera2 provides a powerful and modern Python interface for libcamera on the Raspberry Pi. Its architecture, based on an asynchronous event loop, allows for efficient, non-blocking camera operations suitable for a wide range of applications, from simple still captures to real-time video processing and machine learning. The clear separation of configuration, capture, encoding, and previewing makes the library both flexible and easy to use. The extensive set of examples provides practical guidance for leveraging its features.
Sources: README.md:8-10, tests/test_list.txt## FULL FILE CONTEXT SUMMARY
Query: You are an expert technical writer and software architect.
Your task is to generate a comprehensive and accurate technical wiki page in Markdown format about a specific feature, system, or module within a given software project.
You will be given:
- The "[WIKI_PAGE_TOPIC]" for the page you need to create.
- A list of "[RELEVANT_SOURCE_FILES]" from the project that you MUST use as the sole basis for the content. You MUST use AT LEAST 5 relevant source files for comprehensive coverage - if fewer are provided, search for additional related files in the codebase.
CRITICAL STARTING INSTRUCTION:
The very first thing on the page MUST be a
Format it exactly like this:
Relevant source files
Remember, do not provide any acknowledgements, disclaimers, apologies, or any other preface before the <details> block. JUST START with the <details> block.
The following files were used as context for generating this wiki page:
Immediately after the
Based ONLY on the content of the [RELEVANT_SOURCE_FILES]:
-
Introduction: Start with a concise introduction (1-2 paragraphs) explaining the purpose, scope, and high-level overview of "Home" within the context of the overall project. If relevant, and if information is available, link to other potential wiki pages using the format Link Text.
-
Detailed Sections: Break down "Home" into logical sections using H2 (##) and H3 (###) Markdown headings. For each section:
- Explain the architecture, components, data flow, or logic relevant to the section's focus, as evidenced in the source files.
- Identify key functions, classes, data structures, API endpoints, or configuration elements pertinent to that section.
-
Mermaid Diagrams:
- EXTENSIVELY use Mermaid diagrams (e.g., flowchart TD, sequenceDiagram, classDiagram, erDiagram, graph TD) to visually represent architectures, flows, relationships, and schemas found in the source files.
- Ensure diagrams are accurate and directly derived information from the [RELEVANT_SOURCE_FILES].
- Provide a brief explanation before or after each diagram to give context.
- CRITICAL diagram requirements for optimal rendering:
- Flow diagrams: Use "graph TD" (top-down) or "flowchart TD" directives for better vertical layout
- Node labels: Keep node labels concise but descriptive (under 25 characters for readability)
- Node IDs: Use alphanumeric characters and underscores only (avoid spaces, special characters)
- Connections: Use standard arrow syntax (-->, ---, -.->) for reliable rendering
- Sequence diagrams:
- Start with "sequenceDiagram" directive on its own line
- Use participant names without spaces (use underscores if needed)
- Define participants explicitly when needed:
participant User as "User Interface" - Use correct arrow types:
- ->> for synchronous calls/requests
- -->> for responses
- -x for failed/error messages
- --x for failed responses
- Add activation boxes sparingly with +/- notation
- Use "Note over" or "Note right of" for clarifications
- Class diagrams: Use standard UML notation with proper relationship syntax
- ER diagrams: Use entity relationship syntax with proper cardinality notations
- Quality guidelines:
- Test diagram syntax mentally before including
- If a diagram becomes too complex, split into multiple focused diagrams
- Ensure proper closing of all brackets and quotes
- Use consistent naming conventions within each diagram
- Add line breaks for readability in complex diagrams
- Example patterns (GOOD vs BAD):
GOOD Flow Diagram:
BAD Flow Diagram:
GOOD Sequence Diagram:
```
4. Tables:
* Use Markdown tables to summarise information such as:
* Key features or components and their descriptions.
* API endpoint parameters, types, and descriptions.
* Configuration options, their types, and default values.
* Data model fields, types, constraints, and descriptions.
-
Code Snippets (ENTIRELY OPTIONAL):
- Include short, relevant code snippets (e.g., Python, Java, JavaScript, SQL, JSON, YAML) directly from the [RELEVANT_SOURCE_FILES] to illustrate key implementation details, data structures, or configurations.
- Ensure snippets are well-formatted within Markdown code blocks with appropriate language identifiers.
-
Source Citations (EXTREMELY IMPORTANT):
- For EVERY piece of significant information, explanation, diagram, table entry, or code snippet, you MUST cite the specific source file(s) and relevant line numbers from which the information was derived.
- Place citations at the end of the paragraph, under the diagram/table, or after the code snippet.
- Use the exact format:
Sources: [filename.ext:start_line-end_line](https://github.com/przkos/picamera2/blob/HEAD/filename.ext%3Astart_line-end_line)for a range, orSources: [filename.ext:line_number](https://github.com/przkos/picamera2/blob/HEAD/filename.ext%3Aline_number)for a single line. Multiple files can be cited:Sources: [file1.ext:1-10](https://github.com/przkos/picamera2/blob/HEAD/file1.ext#L1-L10), [file2.ext:5](https://github.com/przkos/picamera2/blob/HEAD/file2.ext#L5), [dir/file3.ext](https://github.com/przkos/picamera2/blob/HEAD/dir/file3.ext)(if the whole file is relevant and line numbers are not applicable or too broad). - If an entire section is overwhelmingly based on one or two files, you can cite them under the section heading in addition to more specific citations within the section.
- You MUST cite AT LEAST 5 different source files throughout the wiki page to ensure comprehensive coverage.
-
Technical Accuracy: All information must be derived SOLELY from the [RELEVANT_SOURCE_FILES]. Do not infer, invent, or use external knowledge about similar systems or common practices unless it's directly supported by the provided code. If information is not present in the provided files, do not include it or explicitly state its absence if crucial to the topic.
-
Clarity and Conciseness: Use clear, professional, and concise technical language suitable for other developers working on or learning about the project. Avoid unnecessary jargon, but use correct technical terms where appropriate.
-
Conclusion/Summary: End with a brief summary paragraph if appropriate for "Home", reiterating the key aspects covered and their significance within the project.
IMPORTANT: Generate the content in English language.
Remember:
- Ground every claim in the provided source files.
- Prioritize accuracy and direct representation of the code's functionality and structure.
- Structure the document logically for easy understanding by other developers.
Total Files Included: 18
Total Files Skipped: 0
Total Context Tokens: 79,056 / 800,000 (9.9%)
Remaining Token Budget: 720,944
Included Files:
- README.md (1,115 tokens)
- tools/checkstyle.py (4,996 tokens)
- CHANGELOG.md (3,283 tokens)
- setup.py (415 tokens)
- tests/test_list.txt (656 tokens)
- tests/colour_spaces.py (635 tokens)
- tests/app_full.py (10,793 tokens)
- apps/app_full.py (10,793 tokens)
- picamera2/picamera2.py (23,637 tokens)
- picamera2/request.py (5,667 tokens)
- pyrightconfig.json (184 tokens)
- examples/imx500/imx500_segmentation_demo.py (805 tokens)
- picamera2/devices/imx500/imx500.py (7,017 tokens)
- examples/tensorflow/yolo_v5_real_time_with_labels.py (1,450 tokens)
- picamera2/previews/q_gl_picamera2.py (4,066 tokens)
- examples/tensorflow/real_time_with_labels.py (1,218 tokens)
- examples/tensorflow/compositing.py (1,676 tokens)
- picamera2/previews/null_preview.py (650 tokens)
Picamera2 is currently available here as a beta release. This means there may still be some issues and bugs which we shall work on fixing, and where users identify particularly useful features we may still consider implementing them. Mostly we shall be working on bugs, stability, support, examples and documentation, as well as keeping up with ongoing libcamera development. There will also be quite a strong presumption against making signficant code changes unless it seems absolutely necessary, especially any that break existing behaviour or APIs.
Picamera2 is the libcamera-based replacement for Picamera which was a Python interface to the Raspberry Pi's legacy camera stack. Picamera2 also presents an easy to use Python API.
You can find documentation here which should help you to get started.
There are also many examples in the examples folder of this repository, and some further Qt application examples in the apps folder.
Installation
Picamera2 is only supported on Raspberry Pi OS Bullseye (or later) images, both 32 and 64-bit. As of September 2022, Picamera2 is pre-installed on Raspberry Pi OS images, but not on Raspberry Pi OS Lite images. It works on all Raspberry Pi boards right down to the Pi Zero, although performance in some areas may be worse on less powerful devices.
Picamera2 is not supported on:
- Images based on Buster or earlier releases.
- Raspberry Pi OS Legacy images.
- Bullseye (or later) images where the legacy camera stack has been re-enabled.
On systems where Picamera2 is supported but not pre-installed, you can install it with
sudo apt install python3-picamera2 --no-install-recommends
to get a slightly reduced installation with fewer of the window system related elements (this would be suitable for installing on a Raspberry Pi OS Lite system), or
sudo apt install python3-picamera2
for a full installation.
Picamera2 can be installed using pip, however, we recommend installing through apt as this guarantees you will get versions of Picamera2 and the underlying libcamera libraries that have been confirmed as working together.
Contributing
Please note that the "main" branch of this repository corresponds to the currently released version of Picamera2 so that the examples there can be referred to by users. Development for forthcoming releases happens on the "next" branch.
We are happy to receive pull requests (normally for the "next" branch) that will fix bugs, add features and generally improve the code. Pull requests should be:
- Restricted to one change or feature each. Please try to avoid "drive-by fixes" especially in a larger set of changes, as it can make them harder to review.
- The commit history should consist of a number of commits that are as easy to review as possible. In particular this means:
- Where one commit is fixing errors in an earlier commit in the set, please simply merge them.
- Where a commit is reverting a commit from earlier in the set, please remove the commit entirely.
- Please avoid adding merge commits or any other unnecessary commits.
- The commit message should have a short single line at the top which is nonetheless as descriptive as possible. After that we encourage more lines explaining in a little more detail exactly what has been done.
- In general, we don't need to see all the trials, errors and bug-fixes that went into this change, we only want to understand how it works now!
- Try to ensure that the automated tests are working after all the commits in the set. This avoids other developers going back to an arbitrary earlier commit and finding that things don't work. There can be occasions when other problems cause test failures beyond our control, so we'll just have to remain alert to these and work around them as best we can.
- Where changes are likely to be more involved, or may change public APIs, authors should start a discussion with us first so that we can agree a good way forward.
- Before submitting a pull request, please ensure that all the automated tests are passing. They can be run using the
tools/run_testsscript. Please usetools/run_tests --helpfor more information. - Any documentation should be updated accordingly. New examples and tests should be included wherever possible. Also consider making an entry in the change log.
- The author of the pull request needs to agree that they are donating the work to this project and to Raspberry Pi Ltd., so that we can continue to distribute it as open source to all our users. To indicate your agreement to this, we would ask that you finish commit messages with a blank line followed by
Signed-off-by: Your Name <your.email@your.domain>. - We'd like to conform to the common Python PEP 8 coding style wherever possible. To facilitate this we would recommend using our pre-commit hook, which can be installed by doing the following in the root of the repository:
pip3 install pre-commit
pre-commit install
This will install flake8 in a venv and will perform linting of changes prior to committing.
Thank you!
================================================================================
#!/usr/bin/python3
import argparse
import difflib
import fnmatch
import os.path
import re
import shutil
import subprocess
import sys
dependencies = {
'clang-format': True,
'git': True,
}
class Colours:
Default = 0
Black = 0
Red = 31
Green = 32
Yellow = 33
Blue = 34
Magenta = 35
Cyan = 36
LightGrey = 37
DarkGrey = 90
LightRed = 91
LightGreen = 92
Lightyellow = 93
LightBlue = 94
LightMagenta = 95
LightCyan = 96
White = 97
@staticmethod
def fg(colour):
if sys.stdout.isatty():
return '\033[%um' % colour
else:
return ''
@staticmethod
def bg(colour):
if sys.stdout.isatty():
return '\033[%um' % (colour + 10)
else:
return ''
@staticmethod
def reset():
if sys.stdout.isatty():
return '\033[0m'
else:
return ''
class DiffHunkSide(object):
"""A side of a diff hunk, recording line numbers"""
def __init__(self, start):
self.start = start
self.touched = []
self.untouched = []
def __len__(self):
return len(self.touched) + len(self.untouched)
class DiffHunk(object):
diff_header_regex = re.compile(
r'@@ -([0-9]+),?([0-9]+)? +([0-9]+),?([0-9]+)? @@')
def __init__(self, line):
match = DiffHunk.diff_header_regex.match(line)
if not match:
raise RuntimeError("Malformed diff hunk header '%s'" % line)
self.__from_line = int(match.group(1))
self.__to_line = int(match.group(3))
self.__from = DiffHunkSide(self.__from_line)
self.__to = DiffHunkSide(self.__to_line)
self.lines = []
def __repr__(self):
s = '%s@@ -%u,%u +%u,%u @@\n' % \
(Colours.fg(Colours.Cyan),
self.__from.start, len(self.__from),
self.__to.start, len(self.__to))
for line in self.lines:
if line[0] == '-':
s += Colours.fg(Colours.Red)
elif line[0] == '+':
s += Colours.fg(Colours.Green)
if line[0] == '-':
spaces = 0
for i in range(len(line)):
if line[-i - 1].isspace():
spaces += 1
else:
break
spaces = len(line) - spaces
line = line[0:spaces] + Colours.bg(Colours.Red) + line[spaces:]
s += line
s += Colours.reset()
s += '\n'
return s[:-1]
def append(self, line):
if line[0] == ' ':
self.__from.untouched.append(self.__from_line)
self.__from_line += 1
self.__to.untouched.append(self.__to_line)
self.__to_line += 1
elif line[0] == '-':
self.__from.touched.append(self.__from_line)
self.__from_line += 1
elif line[0] == '+':
self.__to.touched.append(self.__to_line)
self.__to_line += 1
self.lines.append(line.rstrip('\n'))
def intersects(self, lines):
for line in lines:
if line in self.__from.touched:
return True
return False
def side(self, side):
if side == 'from':
return self.__from
else:
return self.__to
def parse_diff(diff):
hunks = []
hunk = None
for line in diff:
if line.startswith('@@'):
if hunk:
hunks.append(hunk)
hunk = DiffHunk(line)
elif hunk is not None:
hunk.append(line)
if hunk:
hunks.append(hunk)
return hunks
class CommitFile:
def init(self, name):
info = name.split()
self.__status = info[0][0]
# For renamed files, store the new name
if self.__status == 'R':
self.__filename = info[2]
else:
self.__filename = info[1]
@property
def filename(self):
return self.__filename
@property
def status(self):
return self.__status
class Commit:
def init(self, commit):
self.commit = commit
self._parse()
def _parse(self):
# Get the commit title and list of files.
ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-status',
self.commit],
stdout=subprocess.PIPE).stdout.decode('utf-8')
files = ret.splitlines()
if files[1]:
self._files = [CommitFile(f) for f in files[1:]]
else:
self._files = []
self._title = files[0]
def files(self, filter='AMR'):
return [f.filename for f in self._files if f.status in filter]
@property
def title(self):
return self._title
def get_diff(self, top_level, filename):
diff = subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit),
'--', '%s/%s' % (top_level, filename)],
stdout=subprocess.PIPE).stdout.decode('utf-8')
return parse_diff(diff.splitlines(True))
def get_file(self, filename):
return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)],
stdout=subprocess.PIPE).stdout.decode('utf-8')
class StagedChanges(Commit):
def init(self):
Commit.init(self, '')
def _parse(self):
ret = subprocess.run(['git', 'diff', '--staged', '--name-status'],
stdout=subprocess.PIPE).stdout.decode('utf-8')
self._title = "Staged changes"
self._files = [CommitFile(f) for f in ret.splitlines()]
def get_diff(self, top_level, filename):
diff = subprocess.run(['git', 'diff', '--staged', '--',
'%s/%s' % (top_level, filename)],
stdout=subprocess.PIPE).stdout.decode('utf-8')
return parse_diff(diff.splitlines(True))
class Amendment(StagedChanges):
def init(self):
StagedChanges.init(self)
def _parse(self):
# Create a title using HEAD commit
ret = subprocess.run(['git', 'show', '--pretty=oneline', '--no-patch'],
stdout=subprocess.PIPE).stdout.decode('utf-8')
self._title = 'Amendment of ' + ret.strip()
# Extract the list of modified files
ret = subprocess.run(['git', 'diff', '--staged', '--name-status', 'HEAD~'],
stdout=subprocess.PIPE).stdout.decode('utf-8')
self._files = [CommitFile(f) for f in ret.splitlines()]
def get_diff(self, top_level, filename):
diff = subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--',
'%s/%s' % (top_level, filename)],
stdout=subprocess.PIPE).stdout.decode('utf-8')
return parse_diff(diff.splitlines(True))
class ClassRegistry(type):
def new(cls, clsname, bases, attrs):
newclass = super().new(cls, clsname, bases, attrs)
if bases:
bases[0].subclasses.append(newclass)
return newclass
class CommitChecker(metaclass=ClassRegistry):
subclasses = []
def __init__(self):
pass
#
# Class methods
#
@classmethod
def checkers(cls):
for checker in cls.subclasses:
yield checker
class CommitIssue(object):
def init(self, msg):
self.msg = msg
class StyleChecker(metaclass=ClassRegistry):
subclasses = []
def __init__(self):
pass
#
# Class methods
#
@classmethod
def checkers(cls, filename):
for checker in cls.subclasses:
if checker.supports(filename):
yield checker
@classmethod
def supports(cls, filename):
for pattern in cls.patterns:
if fnmatch.fnmatch(os.path.basename(filename), pattern):
return True
return False
@classmethod
def all_patterns(cls):
patterns = set()
for checker in cls.subclasses:
patterns.update(checker.patterns)
return patterns
class StyleIssue(object):
def init(self, line_number, line, msg, offset=None):
self.line_number = line_number
self.line = line
self.msg = msg
self.offset = offset
class IncludeChecker(StyleChecker):
patterns = ('.cpp', '.h', '*.hpp')
headers = ('assert', 'ctype', 'errno', 'fenv', 'float', 'inttypes',
'limits', 'locale', 'setjmp', 'signal', 'stdarg', 'stddef',
'stdint', 'stdio', 'stdlib', 'string', 'time', 'uchar', 'wchar',
'wctype')
include_regex = re.compile('^#include <c([a-z]*)>')
def __init__(self, content):
super().__init__()
self.__content = content
def check(self, line_numbers):
issues = []
for line_number in line_numbers:
line = self.__content[line_number - 1]
match = IncludeChecker.include_regex.match(line)
if not match:
continue
header = match.group(1)
if header not in IncludeChecker.headers:
continue
issues.append(StyleIssue(line_number, line,
'C compatibility header <%s.h> is preferred' % header))
return issues
class Pep8Checker(StyleChecker):
patterns = ('.py',)
results_regex = re.compile('stdin:([0-9]+):([0-9]+)(.)')
def __init__(self, content):
super().__init__()
self.__content = content
def check(self, line_numbers):
issues = []
data = ''.join(self.__content).encode('utf-8')
try:
ret = subprocess.run(['pycodestyle', '--ignore=E501', '-'],
input=data, stdout=subprocess.PIPE)
except FileNotFoundError:
issues.append(StyleIssue(
0, None, "Please install pycodestyle to validate python additions"))
return issues
results = ret.stdout.decode('utf-8').splitlines()
for item in results:
search = re.search(Pep8Checker.results_regex, item)
line_number = int(search.group(1))
position = int(search.group(2))
msg = search.group(3)
if line_number in line_numbers:
line = self.__content[line_number - 1]
issues.append(StyleIssue(line_number, line, msg, position))
return issues
class ShellChecker(StyleChecker):
patterns = ('*.sh',)
results_line_regex = re.compile('In - line ([0-9]+):')
def __init__(self, content):
super().__init__()
self.__content = content
def check(self, line_numbers):
issues = []
data = ''.join(self.__content).encode('utf-8')
try:
ret = subprocess.run(['shellcheck', '-Cnever', '-'],
input=data, stdout=subprocess.PIPE)
except FileNotFoundError:
issues.append(StyleIssue(
0, None, "Please install shellcheck to validate shell script additions"))
return issues
results = ret.stdout.decode('utf-8').splitlines()
for nr, item in enumerate(results):
search = re.search(ShellChecker.results_line_regex, item)
if search is None:
continue
line_number = int(search.group(1))
line = results[nr + 1]
msg = results[nr + 2]
# Position - Determined, but not yet used
_ = msg.find('^') + 1
if line_number in line_numbers:
issues.append(StyleIssue(line_number, line, msg))
return issues
class Formatter(metaclass=ClassRegistry):
subclasses = []
def __init__(self):
pass
#
# Class methods
#
@classmethod
def formatters(cls, filename):
for formatter in cls.subclasses:
if formatter.supports(filename):
yield formatter
@classmethod
def supports(cls, filename):
for pattern in cls.patterns:
if fnmatch.fnmatch(os.path.basename(filename), pattern):
return True
return False
@classmethod
def all_patterns(cls):
patterns = set()
for formatter in cls.subclasses:
patterns.update(formatter.patterns)
return patterns
class CLangFormatter(Formatter):
patterns = ('.c', '.cpp', '.h', '.hpp')
@classmethod
def format(cls, filename, data):
ret = subprocess.run(['clang-format', '-style=file',
'-assume-filename=' + filename],
input=data.encode('utf-8'), stdout=subprocess.PIPE)
return ret.stdout.decode('utf-8')
class IncludeOrderFormatter(Formatter):
patterns = ('.cpp', '.h', '*.hpp')
include_regex = re.compile('^#include ["<](https://github.com/przkos/picamera2/blob/HEAD/%5B%5E%22%3E%5D*)[">]')
@classmethod
def format(cls, filename, data):
lines = []
includes = []
# Parse blocks of #include statements, and output them as a sorted list
# when we reach a non #include statement.
for line in data.split('\n'):
match = IncludeOrderFormatter.include_regex.match(line)
if match:
# If the current line is an #include statement, add it to the
# includes group and continue to the next line.
includes.append((line, match.group(1)))
continue
# The current line is not an #include statement, output the sorted
# stashed includes first, and then the current line.
if len(includes):
includes.sort(key=lambda i: i[1])
for include in includes:
lines.append(include[0])
includes = []
lines.append(line)
# In the unlikely case the file ends with an #include statement, make
# sure we output the stashed includes.
if len(includes):
includes.sort(key=lambda i: i[1])
for include in includes:
lines.append(include[0])
includes = []
return '\n'.join(lines)
class StripTrailingSpaceFormatter(Formatter):
patterns = ('.c', '.cpp', '.h', '.hpp', '*.py', 'CMakelists.txt')
@classmethod
def format(cls, filename, data):
lines = data.split('\n')
for i in range(len(lines)):
lines[i] = lines[i].rstrip() + '\n'
return ''.join(lines)
def check_file(top_level, commit, filename):
# Extract the line numbers touched by the commit.
commit_diff = commit.get_diff(top_level, filename)
lines = []
for hunk in commit_diff:
lines.extend(hunk.side('to').touched)
# Skip commits that don't add any line.
if len(lines) == 0:
return 0
# Format the file after the commit with all formatters and compute the diff
# between the unformatted and formatted contents.
after = commit.get_file(filename)
formatted = after
for formatter in Formatter.formatters(filename):
formatted = formatter.format(filename, formatted)
after = after.splitlines(True)
formatted = formatted.splitlines(True)
diff = difflib.unified_diff(after, formatted)
# Split the diff in hunks, recording line number ranges for each hunk, and
# filter out hunks that are not touched by the commit.
formatted_diff = parse_diff(diff)
formatted_diff = [
hunk for hunk in formatted_diff if hunk.intersects(lines)]
# Check for code issues not related to formatting.
issues = []
for checker in StyleChecker.checkers(filename):
checker = checker(after)
for hunk in commit_diff:
issues += checker.check(hunk.side('to').touched)
# Print the detected issues.
if len(issues) == 0 and len(formatted_diff) == 0:
return 0
print('%s---' % Colours.fg(Colours.Red), filename)
print('%s+++' % Colours.fg(Colours.Green), filename)
if len(formatted_diff):
for hunk in formatted_diff:
print(hunk)
if len(issues):
issues = sorted(issues, key=lambda i: i.line_number)
for issue in issues:
print('%s#%u: %s' %
(Colours.fg(Colours.Yellow), issue.line_number, issue.msg))
if issue.line is not None:
print('+%s%s' % (issue.line.rstrip(), Colours.reset()))
if issue.offset is not None:
print(" " * (issue.offset) + "^")
return len(formatted_diff) + len(issues)
def check_style(top_level, commit):
separator = '-' * len(commit.title)
print(separator)
print(commit.title)
print(separator)
issues = 0
# Apply the commit checkers first.
for checker in CommitChecker.checkers():
for issue in checker.check(commit, top_level):
print('%s%s%s' %
(Colours.fg(Colours.Yellow), issue.msg, Colours.reset()))
issues += 1
# Filter out files we have no checker for.
patterns = set()
patterns.update(StyleChecker.all_patterns())
patterns.update(Formatter.all_patterns())
files = [f for f in commit.files() if len(
[p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])]
for f in files:
issues += check_file(top_level, commit, f)
if issues == 0:
print("No issue detected")
else:
print('---')
print("%u potential %s detected, please review" %
(issues, 'issue' if issues == 1 else 'issues'))
return issues
def extract_commits(revs):
"""Extract a list of commits on which to operate from a revision or revision range."""
ret = subprocess.run(['git', 'rev-parse', revs], stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if ret.returncode != 0:
print(ret.stderr.decode('utf-8').splitlines()[0])
return []
revlist = ret.stdout.decode('utf-8').splitlines()
# If the revlist contains more than one item, pass it to git rev-list to list
# each commit individually.
if len(revlist) > 1:
ret = subprocess.run(['git', 'rev-list', *revlist],
stdout=subprocess.PIPE)
revlist = ret.stdout.decode('utf-8').splitlines()
revlist.reverse()
return [Commit(x) for x in revlist]
def git_top_level():
"""Get the absolute path of the git top-level directory."""
ret = subprocess.run(['git', 'rev-parse', '--show-toplevel'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if ret.returncode != 0:
print(ret.stderr.decode('utf-8').splitlines()[0])
return None
return ret.stdout.decode('utf-8').strip()
def main(argv):
# Parse command line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--staged', '-s', action='store_true',
help='Include the changes in the index. Defaults to False')
parser.add_argument('--amend', '-a', action='store_true',
help='Include changes in the index and the previous patch combined. Defaults to False')
parser.add_argument('revision_range', type=str, default=None, nargs='?',
help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.')
args = parser.parse_args(argv[1:])
# Check for required dependencies.
for command, mandatory in dependencies.items():
found = shutil.which(command)
if mandatory and not found:
print("Executable %s not found" % command)
return 1
dependencies[command] = found
# Get the top level directory to pass absolute file names to git diff
# commands, in order to support execution from subdirectories of the git
# tree.
top_level = git_top_level()
if top_level is None:
return 1
commits = []
if args.staged:
commits.append(StagedChanges())
if args.amend:
commits.append(Amendment())
# If none of --staged or --amend was passed
if len(commits) == 0:
# And no revisions were passed, then default to HEAD
if not args.revision_range:
args.revision_range = 'HEAD'
if args.revision_range:
commits += extract_commits(args.revision_range)
issues = 0
for commit in commits:
if not commit._files:
print(f"{'*' * 80}\n{commit.commit}\n{'*' * 80}")
msg = "No files found - is this a merge commit?"
print('%s%s%s' % (Colours.fg(Colours.Yellow), msg, Colours.reset()))
issues += 1
else:
issues += check_style(top_level, commit)
print('')
if issues:
return 1
else:
return 0
if name == 'main':
sys.exit(main(sys.argv))
================================================================================
Unreleased (on "next" branch)
Added
Changed
0.3.27 Beta Release 26
Added
Changed
- Fixed a couple of bugs in 0.3.26, including failing to set exposure/gain when
starting the camera.
0.3.26 Beta Release 25
Added
- New SplittableOutput, allows switching between other outputs seamlessly at I frames.
- Support for PySide2, PySide6 and PyQt6 widgets.
Changed
- YUV420 format can be encoded directly to JPEG now, which uses less memory and is faster.
- Support DRM displays when XB24 is not available.
- Fix V4L2 encoder lock-up when a frame produces no output.
- Ongoing improvements to type hints.
0.3.25 Beta Release 24
Added
- Support for software camera sync.
Changed
0.3.24 Beta Release 23
Added
- FfmpegOutput support custom audio filter
Changed
- Updated for newer version of PyAV which we can use to encode
more efficiently.
0.3.23 Beta Release 22
Added
- rpi::ScalerCrops control support
- Hailo multi-model support
- Stereo preview example script
- Add Pyav