Face Detection and Embedding Generation

Relevant source files

The following files were used as context for generating this wiki page:

• face_recognition_insightface.py
• app.py
• templates/capture_wajah.html
• static/js/capture-simple.js
• templates/add_employee_form.html
• templates/admin_employees.html

The Face Detection and Embedding Generation system is a core component of the Smart Absen project, responsible for processing images to identify faces and convert them into numerical representations (embeddings) for high-accuracy recognition. This system leverages the state-of-the-art InsightFace/ArcFace library to achieve over 99% accuracy, a significant improvement over traditional KNN methods. The process involves initializing a deep learning model, detecting faces in an image, extracting a 512-dimensional embedding vector for each face, and managing a database of these embeddings for employee identification.

This module is central to two primary workflows: registering new employees by capturing their face data and identifying employees during the attendance process. The system is designed to be robust, with fallbacks to lighter models if the primary model fails to initialize, and it persists the generated embeddings in a serialized file (face_embeddings.pkl) for efficient loading.

Sources: face_recognition_insightface.py:1-10, app.py:31-39

Architecture Overview

The system's architecture is split between the frontend (for data capture) and the backend (for processing and storage). The backend logic is primarily encapsulated within face_recognition_insightface.py, which is called by the main Flask application in app.py.

The following diagram illustrates the high-level data flow for both training a new face and identifying an existing one.

This diagram shows the two main paths: training (top path) where images are saved and embeddings are generated, and identification (bottom path) where a live image is compared against the stored embeddings.

Sources: app.py, face_recognition_insightface.py

Core Recognition Module: face_recognition_insightface.py

This module contains all the logic for face analysis, embedding extraction, and identification using the InsightFace library. It operates as a singleton to avoid re-initializing the deep learning model on every request.

Sources: face_recognition_insightface.py

Model Initialization

The get_insightface_app() function initializes and returns a FaceAnalysis object. It follows a singleton pattern, ensuring the model is loaded into memory only once.

• Primary Model: It first attempts to load the buffalo_l model, which offers the best accuracy, using the CUDA provider for GPU acceleration if available.
• Fallback Model: If the primary model fails (e.g., due to memory constraints or missing CUDA), it falls back to a lighter buffalo_sc model that runs on the CPU.

This lazy-loading and fallback mechanism enhances the system's robustness.

Sources: face_recognition_insightface.py:23-50

Key Functions

The module exposes several key functions for interacting with the InsightFace model.

Function	Description	Return Type	Source File
get_insightface_app()	Initializes and returns the singleton FaceAnalysis instance.	FaceAnalysis or None	face_recognition_insightface.py:23-50
extract_face_embedding(image)	Detects the largest face in an image and returns its normalized 512-dimensional embedding.	np.ndarray or None	face_recognition_insightface.py:53-78
detect_faces_insightface(image)	Detects all faces in an image and returns a list of dictionaries containing bounding box, embedding, and landmarks.	List[Dict]	face_recognition_insightface.py:81-100
train_insightface_model()	Scans the static/faces/ directory, extracts embeddings for all images, and saves them to the database file.	bool	face_recognition_insightface.py:136-177
identify_face_insightface(image, threshold)	Identifies a face by comparing its embedding to the stored database using cosine similarity.	Tuple[str, float]	face_recognition_insightface.py:180-222

New Employee Training Workflow

Adding a new employee's face to the recognition system is a multi-step process involving data entry, image capture, and backend processing.

Step 1: Employee Data Entry

The process begins in the admin panel where a user adds a new employee.

1. The admin navigates to the "Kelola Karyawan" page (/admin/employees).
2. Clicking "Tambah Karyawan Baru" leads to the form at /admin/add_employee_form.
3. The admin fills out the employee's details (name, NIK, department, etc.).
4. Upon submission, the form data is sent to the /admin/add_employee_complete endpoint in app.py.
5. The backend validates the data and saves the new employee to the employees table in the database.
6. If successful, the backend returns a JSON response containing the new employee_id and redirects the admin to the face capture page (/capture_wajah).

Sources: templates/admin_employees.html:300-306, templates/add_employee_form.html, app.py:650-721

Step 2: Face Photo Capture

The face capture UI is handled by capture_wajah.html and its corresponding JavaScript, static/js/capture-simple.js.

1. Camera Initialization: The page requests camera access using navigator.mediaDevices.getUserMedia.
2. Photo Capture: The user can take photos manually or use an auto-capture mode. The capturePhoto() function draws the current video frame onto a canvas and converts it to a base64-encoded JPEG.
3. Data Collection: Each captured photo (as a base64 string) is pushed into the captureState.photos array. The UI provides real-time feedback on the number of photos taken. A minimum of 5 photos is required.

Sources: templates/capture_wajah.html, static/js/capture-simple.js:78-111

Step 3: Saving Training Data and Model Training

Once enough photos are collected, the admin clicks "Selesai & Simpan".

The following sequence diagram shows the process of saving the training photos and retraining the model.

This diagram illustrates how the frontend sends captured images to the backend, which then saves them and triggers the train_model function to update the face embedding database.

API Endpoint: /api/save_face_training (deprecated in favor of /api/save_training_photos)
The endpoint /api/save_training_photos in app.py handles this logic. It receives the employee ID and an array of base64-encoded photos. It retrieves the employee's NIK, saves the decoded images to static/faces/{nik}/, and finally calls train_model() to update the embeddings database.

Sources: app.py:761-855, face_recognition_insightface.py:136-177, static/js/capture-simple.js:141-171

Face Embedding Database

The system stores face embeddings in a Python dictionary which is then serialized to a file using pickle. This avoids re-calculating embeddings for registered employees on every app start.

• File Path: static/face_embeddings.pkl
• Data Structure: A dictionary where keys are employee names (or NIKs) and values are lists of their corresponding 512-d numpy.ndarray embeddings.

This diagram shows that the training function populates the in-memory dictionary, which is then saved to face_embeddings.pkl. For identification, the file is loaded back into memory.

Sources: face_recognition_insightface.py:19, 112-133

Face Identification Process

When a user performs an attendance check, the system captures an image and attempts to identify the person. This process is handled by the identify_face_insightface function.

The flow is as follows:

1. An image is passed to the function.
2. The face database is loaded from the .pkl file if it's not already in memory.
3. The embedding for the face in the input image is extracted (query_embedding).
4. The function iterates through each known identity in the database.
5. For each identity, it calculates the cosine similarity between the query_embedding and every stored embedding for that person.
6. A final confidence score is calculated as a weighted average: 0.7 * max_similarity + 0.3 * avg_similarity.
7. The identity with the highest score is chosen as the best match.
8. If the final confidence score (converted to a percentage) is below a threshold (defaulting to 45%), the identity is rejected and returned as "Unknown".

This process is visualized in the flowchart below.

Conclusion

The face detection and embedding generation module is a sophisticated and critical part of the Smart Absen system. By using InsightFace, it provides high-accuracy recognition that is robust and efficient. The workflow for training new faces is well-defined, flowing from a user-friendly admin interface to backend processing that updates a persistent embedding database. The identification logic, based on cosine similarity and a weighted confidence score, ensures reliable matching during attendance, forming the foundation of the project's "smart" capabilities.

Sources: face_recognition_insightface.py, README.md:10