Feature: Face Recognition (People View)

Relevant source files

The following files were used as context for generating this wiki page:

• src/pages/People.tsx
• src/lib/faceRecognition.ts
• src/components/FaceScanDialog.tsx
• src/lib/db.ts
• src/pages/SettingsPage.tsx
• package.json
• README.md

The Face Recognition feature provides users with the ability to automatically scan their photo library, detect faces, and group photos by the people in them. This system is powered by the @vladmandic/face-api library and integrates with the application's IndexedDB database to persist the results. The main user interface for this feature is the "People" page, which displays the discovered individuals and allows users to initiate a face scan.

The core functionality involves loading pre-trained machine learning models, detecting face descriptors in each image, clustering similar faces together, and storing these groupings as "Person" records in the database. The process is initiated by the user through a dedicated dialog and is designed to run as a background task within the renderer process.

Sources: src/pages/People.tsx, src/lib/faceRecognition.ts, README.md

Architecture and Core Components

The feature is composed of several key components that work together to provide the end-to-end face recognition experience. The primary interaction is between the UI components, the face recognition service, and the database.

The following diagram illustrates the high-level interaction between these components.

Component Roles

Component	File	Role
People Page	src/pages/People.tsx	Main view for the feature. Displays grouped people and provides the entry point to start a scan.
FaceScanDialog	src/components/FaceScanDialog.tsx	A modal dialog that confirms and initiates the face scanning process.
FaceRecognitionService	src/lib/faceRecognition.ts	A singleton service that encapsulates all face detection logic, including model loading, face detection, and clustering.
PhotoDatabase	src/lib/db.ts	Manages the IndexedDB instance, including storing, retrieving, and updating Person data.

Sources: src/pages/People.tsx, src/components/FaceScanDialog.tsx, src/lib/faceRecognition.ts, src/lib/db.ts

Face Scanning and Clustering Process

The face scanning process is initiated by the user and follows a multi-step pipeline to analyze photos and group faces.

High-Level Flow

The following sequence diagram details the entire process from user interaction to data persistence.

Core Logic Details

1. Model Loading

The FaceRecognitionService first ensures that the required face-api.js models are loaded. It uses Promise.all to load three models concurrently from the /public/models directory. A modelsLoaded flag prevents redundant loading.

• ssdMobilenetv1: For face detection.
• faceLandmark68Net: For detecting facial landmarks.
• faceRecognitionNet: For computing the face descriptor from a detected face.

Sources: src/lib/faceRecognition.ts:7-22

2. Face Detection

For each image, the detectFaces method is called. It fetches the image and uses faceapi to detect all faces, their landmarks, and their descriptors. The descriptor is a 128-element Float32Array that represents the features of a face.

// src/lib/faceRecognition.ts:28-34
const detections = await faceapi
  .detectAllFaces(img)
  .withFaceLandmarks()
  .withFaceDescriptors();

return detections.map(d => d.descriptor);

Sources: src/lib/faceRecognition.ts:24-38

3. Face Clustering

The clusterFaces method implements a simple clustering algorithm.

• It iterates through every face descriptor found across all photos.
• For each descriptor, it attempts to find a matching existing cluster by comparing it to the first descriptor in that cluster.
• The comparison is done using faceapi.euclideanDistance. If the distance is less than a hardcoded threshold of 0.6, the photo is added to that cluster.
• If no matching cluster is found, a new cluster is created with a unique ID (e.g., person_0, person_1).

This approach is a basic form of agglomerative clustering, where each new face is either merged into an existing group or forms a new one.

Sources: src/lib/faceRecognition.ts:40-80

Data Models and Storage

All face recognition data is stored in the browser's IndexedDB, managed by src/lib/db.ts.

Person Data Model

The Person interface defines the structure for storing a group of photos belonging to one individual.

Field	Type	Description
id	string	Unique identifier for the person/cluster (e.g., person_0). This is the primary key.
name	string	The name assigned to the person. Defaults to Person X.
photoIds	string[]	An array of photo IDs that belong to this person.
avatarPhotoId	string	The ID of the photo to be used as the avatar. Defaults to the first photo in the cluster.

Sources: src/lib/db.ts:43-48

Database Schema

The people object store is created in IndexedDB to hold Person objects. The id field serves as the keyPath.

User Interface

People Page (People.tsx)

This is the main view for the feature.

• Initial State: If no people are found in the database, it displays a prompt encouraging the user to start a scan.
• Data Display: Once people are detected, it renders a grid of circular avatars. Each avatar represents a person and is sourced from their avatarPhotoId. Below the avatar, the person's name and the total count of their photos are displayed.
• Scan Initiation: A "Scan for Faces" button is always present, which opens the FaceScanDialog.

Sources: src/pages/People.tsx:20-80

FaceScanDialog Component (FaceScanDialog.tsx)

This component orchestrates the scanning process from the user's perspective.

• It provides a brief explanation of what the scan will do.
• The handleScan function is triggered when the user clicks "Start Scan". This function contains the top-level logic for fetching photos, calling the FaceRecognitionService, and saving the results to the database.
• It displays a progress indicator while scanning is in progress, although the progress value is not dynamically updated during the clustering loop in the current implementation.

Sources: src/components/FaceScanDialog.tsx:16-55

Configuration

The application includes UI settings related to face recognition, though their direct effect on the scanning logic is not implemented in the provided files.

Setting	UI Control	Description	File & Line
Enable face detection	Switch	"Detect and group faces in photos"	src/pages/SettingsPage.tsx:191-195
Auto-name faces	Switch	"Suggest names based on metadata"	src/pages/SettingsPage.tsx:197-201

These settings are saved to the database via db.setSetting but are not currently checked within the FaceScanDialog or FaceRecognitionService before initiating a scan.

Sources: src/pages/SettingsPage.tsx:187-203

Summary

The Face Recognition feature is a significant module in Pixona, designed to enhance photo organization. It leverages the powerful face-api.js library to perform client-side face detection and clustering. The architecture is well-defined, separating the UI (People.tsx, FaceScanDialog.tsx), core logic (faceRecognition.ts), and data persistence (db.ts). While the current clustering algorithm is basic and configuration options are not fully wired, the foundation is robust and provides a complete, user-initiated workflow for identifying and grouping people within the user's photo library.

Sources: README.md, src/lib/faceRecognition.ts, src/pages/People.tsx