AI Transaction Enrichment
Relevant source files
The following files were used as context for generating this wiki page:
AI Transaction Enrichment is a core system within the Syllogic platform designed to automate the categorization and identification of financial data. By leveraging OpenAI's Large Language Models (LLMs), the system transforms raw transaction strings from bank exports or CSV imports into structured, meaningful data. This includes mapping unknown CSV columns to internal schemas, identifying merchants, and assigning categories with high confidence levels.
Sources: README.md:12-14, backend/app/services/category_matcher.py:10-18
System Architecture
The enrichment process follows a two-tier approach to balance speed and accuracy. It first attempts deterministic matching using predefined rules and keyword scoring. If confidence thresholds are not met, the system falls back to an AI-powered tier using models like gpt-4o-mini.
Enrichment Data Flow
The following diagram illustrates how raw transaction data is processed through the enrichment pipeline:
The workflow prioritizes local rules to minimize API costs before utilizing cloud-based AI.
Sources: backend/app/services/category_matcher.py:53-75, frontend/lib/actions/csv-import.ts:167-175
CSV Column Mapping Enrichment
During the transaction import phase, the system uses AI to analyze CSV headers and sample data. This enrichment step identifies which columns represent dates, amounts, and merchants, even when column names are ambiguous or in different languages.
AI Mapping Logic
The system sends a subset of CSV data (headers and first 3 sample rows) to an LLM to generate a ColumnMapping object.
| Field | Description | AI Detection Logic |
|---|---|---|
date | Transaction date | Prefers "Completed Date" over "Started Date". |
amount | Transaction value | Detects if amounts are signed (positive for credits). |
merchant | Vendor name | Identifies merchant names separate from descriptions. |
state | Transaction status | Filters for values like "COMPLETED" or "SETTLED". |
dateFormat | Date structure | Infers "DD-MM-YYYY" or "MM-DD-YYYY" from samples. |
Sources: frontend/lib/actions/csv-import.ts:178-220, frontend/app/(dashboard)/transactions/import/mapping/page.tsx:49-61
Automated Transaction Categorization
Categorization is handled by the CategoryMatcher service, which supports single and batch processing. It uses normalized text comparison and user-specific instructions to refine results.
Tier 1: Deterministic Matching
The service maintains a set of keyword rules for categories such as Groceries, Transport, and Utilities. Text is normalized (lowercase, prefix removal, special character stripping) before matching.
Sources: backend/app/services/category_matcher.py:348-375
Tier 2: LLM Categorization
When deterministic matching fails, the system constructs a prompt containing transaction details and available categories. It supports "User Overrides," allowing the AI to learn from previous manual corrections made by the user.
Subscription Enrichment
Beyond simple categorization, the system identifies recurring patterns to suggest subscriptions. This enrichment module calculates confidence scores based on frequency (weekly, monthly, etc.) and transaction amounts.
Verification and Linking
When a subscription is verified by a user, the system enriches the data by:
- Inheriting Categories: Finding the most frequent category assigned to the matched transactions.
- Linking History: Updating historical transactions with the new
recurringTransactionId. - Merchant Normalization: Consolidating various transaction descriptions under a single merchant name.
Sources: frontend/lib/actions/subscription-suggestions.ts:120-160, frontend/lib/actions/subscription-suggestions.ts:245-260
Configuration and Costs
The AI enrichment system is configurable via environment variables, allowing administrators to tune performance and monitor usage.
| Variable | Default Value | Purpose |
|---|---|---|
CATEGORIZATION_LLM_MODEL | gpt-4o-mini | Model used for categorization. |
CATEGORIZATION_MIN_CONFIDENCE | 999.0 | Threshold for deterministic matches. |
OPENAI_API_KEY | None | Required for AI features. |
PRICING (input) | $0.150 / 1M | Estimated cost tracking for gpt-4o-mini. |
Sources: backend/app/services/category_matcher.py:82-95, frontend/lib/actions/settings.ts:14-16
Summary
AI Transaction Enrichment provides an automated bridge between raw banking data and actionable financial insights. By combining deterministic rule engines with LLM-based fallbacks, the system ensures high categorization coverage while providing users with tools to refine the AI's logic through manual overrides and custom instructions.
Sources: README.md:38-45, backend/app/services/category_matcher.py:115-130