AI Plate Verification
Relevant source files
The following files were used as context for generating this wiki page:
AI Plate Verification is a security and operational feature within the Outmanage system designed to automate the validation of vehicle license plates during the shipment completion phase. The system utilizes Google's Gemini AI to extract license plate numbers from uploaded photos and compare them against registered vehicle data, ensuring that the correct fleet vehicle is handling the designated shipment.
This feature serves as a prerequisite for marking shipments as SELESAI (Completed). It bridges physical logistics with digital verification, reducing manual data entry errors and providing an audit trail through stored plate photos and AI detection results.
System Architecture and Logic
The verification process follows a multi-step workflow involving file management, AI processing, and shipment state transitions.
Verification Workflow
The process begins once all items in a shipment have reached the COMPLETED or CHOSEN status. Users upload a physical photo of the vehicle's plate, which is stored locally on the server. The AI service then processes this image to perform Optical Character Recognition (OCR) tailored for Indonesian license plate patterns.
The diagram above illustrates the logical flow from the initial request to the final shipment status update. Sources: src/controllers/shipmentController.ts:1330-1435, src/services/shipmentService.ts:1410-1510
AI Integration Component
The geminiAiService acts as the interface for Google's Generative AI. It uses the gemini-2.5-flash-preview-05-20 model to process image inputs. The service includes safety settings to block harmful content and uses a specific prompt to guide the AI in identifying Indonesian plate formats (e.g., "B 1234 ABC").
Sources: src/services/geminiAiService.ts:10-80, README.md:160-175
Component Breakdown
API Endpoints
The following endpoints facilitate the AI verification cycle:
| Method | Endpoint | Description |
|---|---|---|
PATCH | /shipment/:id/upload-plate-photo | Uploads a multipart/form-data image to the server. |
PATCH | /shipment/:id/verify-plate | Triggers the AI extraction and matching logic. |
PATCH | /shipment/:id/verify-plate-manual | Allows manual bypass if AI extraction fails or is unavailable. |
Sources: src/controllers/shipmentController.ts:1330, src/controllers/shipmentController.ts:1400, src/controllers/shipmentController.ts:1480
Database Schema Impacts
Verification results directly modify the Shipment model. The isVerified flag and verifiedAt timestamp are updated upon a successful match.
Sources: prisma/schema.prisma:285-310
AI Extraction Logic
The extractPlateNumberFromImage function handles the core AI interaction. It converts the local file to a base64 generative part and submits it with a specialized prompt.
Prompt Engineering
The system provides explicit instructions to the AI:
- Focus on rectangular mounted plates.
- Return "NO_PLATE_FOUND" if characters are missing or unreadable.
- Indonesian plate patterns: "B 1234 ABC" or "AB 1234 CD".
- Return ONLY raw text without formatting or explanations.
Sources: src/services/geminiAiService.ts:60-90
Matching and Normalization
To account for minor OCR errors or formatting differences (spaces, hyphens), the comparePlateNumbers function normalizes both the extracted and expected strings.
- Normalization: Removes
\s,-,., and/, converting text to uppercase. - Containment Check: Verifies if the expected plate exists within the AI's response.
- Similarity Threshold: If strings don't match exactly, the system calculates a character-by-character match. If at least 80% of characters match in sequence, the verification is considered successful.
Sources: src/services/geminiAiService.ts:145-185
File Management
The fileService handles the storage of plate photos. Images are restricted to 10MB and must be of MIME type image/*.
- Production Storage:
/var/www/sajpoutbound.com/public/plate-photos - Development Storage:
src/public/plate-photos - Naming Convention: Files are renamed using the pattern
plate_photo_{shipmentId}_{timestamp}.{ext}to ensure uniqueness and prevent overwriting.
Sources: src/services/fileService.ts:15-70
Conclusion
AI Plate Verification provides an automated layer of security for the Outmanage shipment workflow. By leveraging Gemini AI for visual data extraction and implementing a flexible matching algorithm with an 80% similarity threshold, the system ensures high reliability even in varied lighting or angled photo conditions. This automation reduces the administrative burden on warehouse personnel while maintaining strict fleet compliance.