Feature: Duplicate Detection

Relevant source files

The following files were used as context for generating this wiki page:

• src/pages/Duplicates.tsx
• src/lib/perceptualHash.ts
• src/lib/db.ts
• electron/main.ts
• electron/preload.ts
• README.md

The Duplicate Detection feature in PIXONA is designed to help users identify and manage both exact and visually similar (near-duplicate) photos within their library. Its primary goal is to free up storage space by providing a safe and intuitive way to review and delete redundant images. The system leverages perceptual hashing to compare images based on their content rather than just file metadata.

This feature is a key part of the application's smart organization capabilities, as outlined in the development roadmap. It integrates with the core photo database and the underlying Electron file system for a seamless user experience.

Sources: src/pages/Duplicates.tsx:202-206, README.md:120-124

Architecture Overview

The duplicate detection system is composed of several key components that work together, spanning the renderer process (React UI), the perceptual hashing library, the IndexedDB database, and the Electron main process for file operations.

The following diagram illustrates the high-level architecture and flow of the duplicate detection process.

Sources: src/pages/Duplicates.tsx, src/lib/perceptualHash.ts, electron/main.ts

Detailed Logic

1. User Interface and State Management

The user interface for duplicate detection is managed by the Duplicates.tsx React component. It handles the presentation of the scan button, progress indicators, and the final grouped results.

Sources: src/pages/Duplicates.tsx

Key State Variables

State Variable	Type	Description
scanning	boolean	Tracks if a scan is currently in progress.
progress	number	Stores the percentage completion of the scan (0-100).
duplicateGroups	DuplicateGroup[]	An array of groups, where each group contains an original and its duplicates.
selectedForDeletion	Set<string>	A set of photo IDs that the user has marked for deletion.

Sources: src/pages/Duplicates.tsx:20-23

Core Functions

• scanForDuplicates(): This is the primary function that orchestrates the detection process. It sets the scanning state, retrieves all photos from the database, invokes the perceptual hashing service, and populates the duplicateGroups state with the results.
• toggleSelection(photoId): Adds or removes a photo's ID from the selectedForDeletion set, allowing users to mark duplicates for removal.
• handleDeleteSelected(): After user confirmation, this function iterates through the selectedForDeletion set. For each photo, it makes an IPC call via window.electron.moveToTrash() to move the file to the system's trash and then calls db.deletePhotos() to remove the corresponding entries from the application's database.

Sources: src/pages/Duplicates.tsx:28-82, src/pages/Duplicates.tsx:109-141

2. Perceptual Hashing

The core of the similarity detection lies in the PerceptualHashService. This service generates a "fingerprint" (hash) for each image that represents its visual content.

Sources: src/lib/perceptualHash.ts

Hashing Process

The generateHash method follows these steps:

1. Load the image from its path into an Image object.
2. Draw the image onto an 8x8 pixel <canvas>.
3. Iterate through the 64 pixels, calculating the average grayscale brightness of the entire canvas.
4. For each pixel, compare its brightness to the average.
5. Build a 64-bit hash string: '1' if the pixel is brighter than or equal to the average, '0' otherwise.

Sources: src/lib/perceptualHash.ts:4-34

Similarity Comparison

The findSimilarImages function is responsible for comparing all photos.

Sources: src/lib/perceptualHash.ts:49-80

Two hashes are considered similar if their Hamming distance (the number of differing bits) is below a certain threshold. The default threshold is 5.

Sources: src/lib/perceptualHash.ts:44-47

3. Data Storage and Models

The feature relies on the IndexedDB database, managed by db.ts, to store and retrieve photo metadata.

PhotoMetadata Schema

The PhotoMetadata interface includes a hash field specifically for duplicate detection. This field stores the perceptual hash generated for the image.

export interface PhotoMetadata {
  id: string;
  path: string;
  // ... other fields
  hash?: string; // for duplicate detection
}

Sources: src/lib/db.ts:10-37

The database schema includes an index on the hash field to allow for efficient lookups, although the current implementation in findSimilarImages fetches all photos and compares them in memory.

Sources: src/lib/db.ts:71, src/lib/perceptualHash.ts:50-51

4. Deletion Process

Deleting duplicates is a two-step process to ensure data integrity: moving the physical file to trash and then removing its record from the database.

The sequence diagram below details the interaction when a user deletes selected photos.

Sources: src/pages/Duplicates.tsx:109-141, electron/main.ts:241-247, electron/preload.ts:25-26, src/lib/db.ts:222-234

The moveToTrash functionality is exposed securely from the Electron main process to the renderer process via the preload script.

• electron/main.ts: Defines the ipcMain.handle('move-to-trash', ...) listener that uses Electron's shell.trashItem() API.
• electron/preload.ts: Exposes the moveToTrash function on the window.electron object.

Sources: electron/main.ts:241-247, electron/preload.ts:25-26

Summary

The Duplicate Detection feature is a powerful tool within PIXONA for library maintenance. It combines a sophisticated perceptual hashing algorithm to find visually similar images with a safe, user-controlled deletion workflow. The architecture effectively separates concerns between the UI, data processing, and low-level file system operations, leveraging Electron's IPC mechanism for secure access to native capabilities like moving files to the trash.