Silence Detection
Relevant source files
The following files were used as context for generating this wiki page:
Silence detection is a core pillar of the Crisp engine, responsible for identifying regions of low audio energy in video recordings to facilitate automatic jump-cuts. By analyzing real audio energy rather than relying solely on speech-to-text timestamps, Crisp ensures high accuracy in finding pauses even when speech timing is ambiguous or missing.
The system operates as the first stage of the analysis pipeline, feeding detected silence intervals into the editing module to determine which segments of the video should be kept or removed. This process is entirely local, leveraging ffmpeg's silencedetect filter to process extracted 16kHz mono audio.
Sources: README.md:46-51, CLAUDE.md:129-131
Architectural Overview
The silence detection logic is orchestrated through a multi-step pipeline that transitions from raw video to a list of timestamped silence intervals.
The diagram shows the sequential flow from the source video file to the final filtered silence intervals used for cutting.
Sources: apps/desktop/Resources/engine/crisp/pipeline.py:463-485, apps/desktop/Resources/engine/crisp/detect.py:84-118
Key Components
| Component | Responsibility | Relevant File |
|---|---|---|
extract_audio | Converts source video to 16kHz mono PCM S16LE WAV for consistent analysis. | detect.py:73-82 |
detect_silences | Executes FFmpeg and parses silence_start and silence_end from stderr. | detect.py:85-118 |
filter_silences | Filters raw silence detections based on user-defined minimum duration. | detect.py:121-130 |
analyze | Orchestrates extraction and detection for live UI previews. | pipeline.py:440-459 |
Audio Extraction for Analysis
Before detection occurs, the engine extracts a standardized audio stream. The engine uses ffmpeg to create a temporary mono WAV file at a sample rate of 16,000 Hz. This normalization ensures that detection parameters (noise thresholds and durations) behave consistently regardless of the source video's original audio format or bitrate.
Sources: apps/desktop/Resources/engine/crisp/detect.py:73-82, apps/desktop/Resources/engine/crisp/pipeline.py:476-477
Detection Logic and Parsing
The core detection utilizes the silencedetect audio filter. This filter analyzes audio energy and logs metadata to the stderr stream.
Detection Parameters
The detection process relies on two primary tunables provided by the user or system presets:
- Noise Threshold (
noise_db): The decibel level below which audio is considered "silence." - Minimum Duration (
min_pause): The minimum length of time the audio must remain below the noise threshold to be flagged.
Parsing Mechanism
The detect_silences function monitors the subprocess stderr for specific patterns:
silence_start: Captures the timestamp when audio energy drops below the threshold.silence_end: Captures the timestamp when energy rises above the threshold, concluding the interval.
Sources: apps/desktop/Resources/engine/crisp/detect.py:85-118
The sequence diagram illustrates the interaction between the engine pipeline, the detection utility, and the FFmpeg subprocess during the parsing of silence intervals.
Sources: apps/desktop/Resources/engine/crisp/detect.py:91-118
Multi-Purpose Filtering
To optimize performance, Crisp often performs a single high-sensitivity silence scan and then filters the results. This is particularly relevant when both Silence Detection and Retake Detection are enabled.
- Retake Anchoring: Requires shorter pauses (typically 0.3s) to identify where a speaker might have restarted a sentence.
- Jump-Cuts: Use a longer user-defined pause threshold (e.g., 0.5s to 2.0s).
The filter_silences function allows the engine to derive both sets from a single FFmpeg pass by extracting all pauses longer than the shortest required threshold and then sub-filtering for the longer cut intervals.
Sources: apps/desktop/Resources/engine/crisp/detect.py:121-130, apps/desktop/Resources/engine/crisp/pipeline.py:486-498
Zero-Crossing Snapping
A technical refinement in the silence detection system is "Zero-Crossing Snapping." To prevent audible clicks and pops at the jump-cut boundaries, the engine nudges the detected silence start/end points to the nearest point where the audio waveform crosses the zero-amplitude line.
This process uses a 16-bit mono analysis WAV and a default window of 12ms (--snap-ms) to ensure that the splice occurs at a silent point in the waveform, reducing the reliance on audio fades.
Sources: apps/desktop/Resources/engine/NOTES.md:23-28, apps/desktop/Resources/engine/crisp/pipeline.py:560-562
Summary
Silence detection in Crisp is a robust, energy-based analysis system that operates independently of speech recognition. By leveraging FFmpeg's low-level filters and a standardized 16kHz extraction process, the engine provides accurate timestamps for pauses. These intervals are then refined through zero-crossing snapping to produce high-quality, professional jump-cuts without audio artifacts.