InsightFace/ArcFace Integration

Relevant source files

The following files were used as context for generating this wiki page:

• face_recognition_insightface.py
• README.md
• INSTALLATION.md
• templates/capture_wajah.html
• static/js/capture-simple.js
• templates/add_employee_form.html
• CLEANUP_REPORT.md

The InsightFace/ArcFace integration is the core machine learning component of the Smart Absen system, providing state-of-the-art face recognition with over 99% accuracy. This module replaces older, less accurate methods and is responsible for face detection, feature extraction (embedding), and identification. It is designed to be robust, requiring only 5-10 photos per person for effective training, making the employee onboarding process highly efficient.

The entire logic is encapsulated within the face_recognition_insightface.py module, which leverages the insightface Python library to perform complex facial analysis. This system is a key part of the project's technology stack, ensuring reliable and secure attendance marking.

Sources: README.md, face_recognition_insightface.py:1-9, templates/capture_wajah.html:11, 41-47

Architecture

The InsightFace integration is built around a central Python module, face_recognition_insightface.py, which manages the ML model, the face database, and all related operations. It operates as a singleton to ensure the ML model is loaded into memory only once, optimizing performance.

The main components are:

1. InsightFace App (FaceAnalysis): The core model instance for detection and embedding extraction.
2. Face Database (face_embeddings.pkl): A serialized Python dictionary storing the 512-dimensional face embeddings for each registered employee.
3. Wrapper Functions: A set of Python functions that provide a simple API for training, identification, and database management.

The following diagram illustrates the high-level architecture of the module.

Initialization

The InsightFace model is initialized using a singleton pattern to prevent reloading the large model on every request. The get_insightface_app() function handles this process.

• It first attempts to load the high-accuracy buffalo_l model, preferably using a GPU (CUDAExecutionProvider).
• If that fails (e.g., no GPU or CUDA), it falls back to a lighter buffalo_sc model running on the CPU.
• The prepared model instance is stored in a global variable _insightface_app for reuse.

Sources: face_recognition_insightface.py:23-48

Face Database

The system stores face embeddings in a Python pickle file located at static/face_embeddings.pkl. This file contains a dictionary where keys are employee names (strings) and values are lists of their corresponding 512-dimensional face embeddings (NumPy arrays).

The structure is as follows:
{ "employee_name": [embedding_1, embedding_2, ...], ... }

This diagram shows the logical structure of the data stored in face_embeddings.pkl.

Key functions for database management:

• load_face_database(): Loads the .pkl file into the global _face_database dictionary.
• save_face_database(): Saves the current state of the _face_database dictionary back to the .pkl file.
• add_face_to_database(): Adds a new embedding for a specific person.
• remove_face_from_database(): Deletes all data for a specific person.

Sources: face_recognition_insightface.py:16-17, 114-135, 223-252

Training Workflow

The process of adding a new employee's face to the system involves both frontend and backend components. It is designed to be user-friendly, starting from the admin panel.

1. Add Employee: An admin fills out the new employee's details in the form at /admin/add_employee_form.
2. Redirect to Capture: Upon successful submission, the backend saves the employee data and redirects the admin to the /capture_wajah page, passing the new employee_id, name, and dept as URL parameters.
3. Capture Photos: The capture_wajah.html page, powered by capture-simple.js, initializes the webcam. The user can then capture 5-10 photos of the employee.
4. Submit for Training: Once enough photos are collected, the "Save" button is clicked. The frontend sends the employee ID and an array of base64-encoded images to the /api/save_face_training endpoint.
5. Backend Processing: The backend receives the data. For each image, it uses extract_face_embedding() to get the 512-d vector and then saves it to the database using add_face_to_database().

The following sequence diagram illustrates this workflow.

Identification Workflow

When a user attempts to check in via face recognition, the system follows a precise workflow to identify them.

1. Extract Embedding: The system captures an image from the webcam and passes it to extract_face_embedding() to get the query embedding.
2. Load Database: The identify_face_insightface() function ensures the face database is loaded from the pickle file if it's not already in memory.
3. Compare Embeddings: It iterates through every registered person in the database. For each person, it calculates the cosine similarity between the query embedding and all of that person's stored embeddings.
4. Calculate Confidence Score: To improve robustness, a final score is calculated as a weighted average: 0.7 * max_similarity + 0.3 * avg_similarity.
5. Apply Threshold: This final score (as a percentage) is compared against a confidence threshold (default is 45%). If the score is above the threshold, the person is identified. Otherwise, the result is "Unknown".

This flow is visualized in the diagram below.

Key Functions

The face_recognition_insightface.py module exposes several key functions for interacting with the face recognition system.

Function	Description	Source
get_insightface_app()	Initializes and returns the singleton FaceAnalysis model instance.	face_recognition_insightface.py:23-48
extract_face_embedding(image)	Takes a BGR image and returns a 512-d normalized embedding vector.	face_recognition_insightface.py:51-78
detect_faces_insightface(image)	Detects all faces in an image and returns their bounding boxes, embeddings, and landmarks.	face_recognition_insightface.py:81-100
cosine_similarity(emb1, emb2)	Calculates the cosine similarity between two embedding vectors.	face_recognition_insightface.py:103-108
train_insightface_model()	Batch-trains the model by processing all images in the static/faces/ directory.	face_recognition_insightface.py:138-180
identify_face_insightface(image, threshold)	Identifies a face in an image by comparing it against the database.	face_recognition_insightface.py:183-220
add_face_to_database(name, image)	Adds a single new face embedding for a given person to the database.	face_recognition_insightface.py:223-241
remove_face_from_database(name)	Removes all data associated with a person from the database.	face_recognition_insightface.py:244-257

Configuration

The module contains some hardcoded configuration values and also relies on environment variables.

Parameter	Type	Default Value	Description	Source
_database_path	string	static/face_embeddings.pkl	The file path for storing the serialized face embeddings database.	face_recognition_insightface.py:17
Identification Threshold	float	0.45	The minimum cosine similarity score required to consider a face identified.	face_recognition_insightface.py:183
FACE_CONFIDENCE	float	0.6	Environment variable to configure the confidence threshold. Note: This seems to be a general app config, while the module itself uses a hardcoded default of 0.45.	README.md

Dependencies

To function correctly, this module requires several key Python packages to be installed.

Package	Version	Purpose
insightface	≥0.7.0	Provides the core ArcFace models and analysis tools.
onnxruntime	≥1.10.0	The runtime engine for executing the ONNX-format ML models used by InsightFace.
opencv-python	≥4.5.0	Used for image reading, processing, and manipulation (e.g., converting color spaces).
numpy	≥1.20.0	Used for all numerical operations, especially vector math on embeddings.

Sources: INSTALLATION.md

Conclusion

The InsightFace/ArcFace integration is a powerful and central feature of the Smart Absen project. By leveraging a state-of-the-art model, it provides a highly accurate (99%+) and robust foundation for the face recognition attendance system. The module is well-encapsulated, offering simple functions for the main application to perform training and identification, while abstracting away the complexities of the underlying machine learning model. Its efficiency, requiring only a small number of training images, significantly improves the usability and scalability of the system.

Sources: README.md, face_recognition_insightface.py