Smart AI Categorization
Relevant source files
The following files were used as context for generating this wiki page:
Smart AI Categorization is a core component of the Syllogic platform designed to automate the enrichment and classification of financial transactions. By utilizing a multi-tiered approach—ranging from user-defined overrides and deterministic keyword matching to advanced Large Language Model (LLM) analysis—the system ensures high accuracy in organizing financial data. This module reduces the manual effort required for transaction management and provides the foundation for category-based analytics and spending insights.
Sources: README.md:10-14, backend/app/services/category_matcher.py:1-10
Architecture and Logic Flow
The categorization system operates as a data pipeline within the Python backend, triggered by transaction imports or manual sync requests. It follows a strict hierarchy to determine the most accurate category while minimizing computational costs and API latency.
Multi-Tier Matching Logic
The matching process follows these priority levels:
- User Overrides: Checks if the user has previously manually categorized a similar transaction or provided specific instructions for that merchant/description.
- Deterministic Matching: A fast, rule-based approach using keyword patterns (e.g., "Tesco" -> "Groceries").
- LLM Categorization: An AI-powered fallback (defaulting to GPT-4o-mini) used when previous tiers fail to provide a confident match.
The diagram above illustrates the fallback hierarchy used by the CategoryMatcher to resolve transaction categories.
Sources: backend/app/services/category_matcher.py:65-80, backend/app/services/category_matcher.py:696-740
Component Breakdown
CategoryMatcher Service
The CategoryMatcher class is the primary implementation of the categorization logic. It handles database interaction for fetching user categories and instructions, manages the OpenAI client for AI fallback, and performs text normalization for matching accuracy.
Key functionalities include:
- Text Normalization: Strips common transaction prefixes like "payment to" or "transfer from" and removes special characters to improve matching consistency.
- Instruction Loading: Automatically aggregates categorization instructions from the category level, user transaction overrides, and transaction-level custom notes.
Sources: backend/app/services/category_matcher.py:118-135, backend/app/services/category_matcher.py:382-411
Batch Processing
For performance, especially during CSV imports, the system supports batch categorization. This allows multiple transactions to be sent in a single LLM prompt, significantly reducing costs and processing time compared to individual API calls.
This sequence shows the interaction between background workers and the categorization service during a batch import.
Sources: backend/app/services/category_matcher.py:821-860, README.md:150-160
Data Structures and API
The system uses specific Pydantic schemas to communicate between the frontend and the categorization engine.
Transaction Input/Output
| Field | Type | Description |
|---|---|---|
description | string | Raw transaction description from the bank. |
merchant | string | Identified merchant name. |
amount | Decimal | Transaction value (positive for income, negative for expense). |
method | string | Categorization method: override, deterministic, llm, or none. |
confidence_score | float | Confidence level (0.0 - 100.0). |
Sources: backend/app/schemas.py:126-146
Configuration Parameters
The behavior of the Smart Categorization is controlled via environment variables and class constants.
| Variable | Default | Purpose |
|---|---|---|
CATEGORIZATION_LLM_MODEL | gpt-4o-mini | The OpenAI model used for AI analysis. |
CATEGORIZATION_MIN_CONFIDENCE | 999.0 | Threshold for deterministic matches (effectively disabled by default in some builds). |
CATEGORIZATION_LLM_MAX_RETRIES | 3 | Number of attempts for failed AI calls. |
CATEGORIZATION_LLM_TEMPERATURE | 0.1 | Creativity setting for LLM (low for consistency). |
Sources: backend/app/services/category_matcher.py:84-90, CLAUDE.md:120-130
Implementation Details
Deterministic Keyword Rules
The CategoryMatcher maintains a mapping of keywords to standard categories. This allows for near-instant categorization of common merchants.
# backend/app/services/category_matcher.py:240-270
self._keyword_rules = {
"groceries": ["supermarket", "grocery", "tesco", "sainsbury", "aldi", "lidl"],
"transport": ["uber", "lyft", "taxi", "bus", "train", "tfl", "parking"],
"dining out": ["restaurant", "cafe", "coffee", "starbucks", "deliveroo", "ubereats"],
"subscriptions": ["netflix", "spotify", "disney", "prime video", "apple store"]
}
Sources: backend/app/services/category_matcher.py:240-270
User Overrides and Instructions
The system is designed to learn from user behavior. If a user manually changes a category in the frontend using updateCategory, the backend records this as an override. Subsequent transactions with similar descriptions or merchants will prioritize this user-defined category over system defaults.
Sources: frontend/lib/actions/categories.ts:70-90, backend/app/services/category_matcher.py:155-175
Conclusion
Smart AI Categorization provides a scalable and intelligent way to manage financial transactions. By combining rule-based speed with the reasoning capabilities of LLMs, the system achieves a high degree of automation while remaining customizable through user feedback and explicit instructions. This system is essential for the platform's ability to provide accurate spending analytics and financial health insights.
Sources: README.md:50-60