Document Classification

Relevant source files

The following files were used as context for generating this wiki page:

• parsing/classifier.py
• parsing/__init__.py
• pipeline.py
• backend.py
• parsing/parser.py
• README.md

Introduction

Document Classification is a critical initial stage in the Rosy tax processing pipeline. Its primary purpose is to identify the type of tax document uploaded by a user—specifically distinguishing between Form W-2 and Form 1099—to ensure that the correct extraction logic and validation rules are applied in subsequent steps. Sources: README.md:10-12, parsing/classifier.py:1-5

By determining the document type and assigning a confidence score, the system can handle multi-document uploads, aggregate relevant financial data, and ultimately populate a draft Form 1040. If a document cannot be confidently identified, it is labeled as "unknown." Sources: pipeline.py:15-30, parsing/classifier.py:33-42

Classification Architecture

The classification system utilizes a heuristic-based approach to analyze the text content of ingested files. It operates within the parsing package and is orchestrated by the global processing pipeline. Sources: parsing/init.py:1-6, pipeline.py:18-24

Logic and Scoring Mechanism

The classifier evaluates text using a scoring system based on "signals" found within the document string. These signals are categorized by strength:

• Strong Signals (+3 points): Explicit mentions of "Form W-2" or "Form 1099".
• Medium Signals (+1 point): Presence of terms like "Employer Identification Number" or "EIN" (W-2) and "Payer/Recipient" pairings (1099).
• Weak Signals (+0.5 to 1 point): Keywords such as "Wages" or "Box 1" (W-2) and "Nonemployee compensation" (1099).

Sources: parsing/classifier.py:11-28

The following flowchart illustrates the classification decision logic:

The confidence score is calculated as a ratio of the winning category's score over the total combined score. Sources: parsing/classifier.py:34-39

Pipeline Integration

Document classification is integrated into the run_pipeline_on_paths workflow. It occurs immediately after the ingestion phase (where PDFs or images are converted to text) and before field extraction. Sources: pipeline.py:46-52

Data Flow for Multi-Document Processing

When multiple files are uploaded, the system performs "per-file" classification. This allows the application to handle a mix of W-2s and 1099s in a single session. Sources: pipeline.py:18-31, backend.py:101-110

Key Components and Functions

Component	Path	Description
detect_document_type	parsing/classifier.py	The core function that implements heuristic scoring to return (doc_type, confidence).
extract_fields	parsing/parser.py	Uses the doc_type identified by the classifier to select appropriate regex patterns.
parse_paths	pipeline.py	Iterates through files and coordinates the call to the classifier for each.
upload endpoint	backend.py	Receives files from the frontend and initiates the classification pipeline.

Sources: parsing/classifier.py:7, parsing/parser.py:32, pipeline.py:15, backend.py:53

Classification Metadata

The classification process produces a specific metadata structure used by the frontend to display results and by the backend for aggregation logic. Sources: pipeline.py:26-30, backend.py:101-115

Field	Type	Description
doc_type	String	'W-2', '1099', or 'unknown'.
confidence	Float	A value between 0.0 and 1.0 representing the relative strength of the classification.
per_file	List[Dict]	An array of objects containing the specific doc_type for every uploaded file.

Sources: parsing/classifier.py:34-42, pipeline.py:26-30

Conclusion

The Document Classification system in Rosy provides a robust, heuristic-based entry point for tax data automation. By accurately distinguishing between document types like W-2 and 1099, the system ensures that downstream components—such as the regex-based parsers and the tax calculation engine—operate on correctly identified data structures, minimizing errors in tax preparation. Sources: README.md:10-15, pipeline.py:46-86