Introduction to Rosy

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• backend.py
• pipeline.py
• forms.py
• taxcalc.py
• parsing/parser.py
• parsing/validator.py
• utils/auto_mapper.py
• security.py

Rosy is an AI-powered tax document processing system designed to automate the preparation of personal federal tax returns. The system allows users to upload various tax documents—such as W-2 and 1099 forms—automatically extracts relevant financial data, computes tax estimates, and generates a completed Form 1040 PDF. It serves as an end-to-end prototype for document classification, data extraction, and automated form filling.

Sources: README.md:1-10, SETUP.md:3-7

The system architecture is modular, separating concerns between document ingestion, heuristic-based parsing, semantic field mapping, and tax logic. It includes a Flask-based web interface for user interaction and a robust back-end pipeline that orchestrates the workflow from file upload to final PDF generation.

Sources: README.md:34-50, backend.py:12-16

Core System Architecture

Rosy's architecture is built around a centralized processing pipeline that coordinates several specialized modules. The system follows a sequential data flow: ingestion, classification, extraction, validation, calculation, and generation.

Processing Pipeline Flow

The following diagram illustrates the high-level workflow of the Rosy system when a user submits documents.

The pipeline orchestrates these steps through the run_pipeline_on_paths function, which handles per-file processing and global aggregation.
Sources: README.md:86-94, pipeline.py:38-51

Key System Components

1. Document Parsing and Classification

The parsing module identifies the type of document (W-2 vs. 1099) using heuristic classification. Once classified, the system uses regex-based extraction to identify specific fields like wages, EINs, and SSNs.

Component	Responsibility	Relevant File
Classifier	Heuristic document type detection	parsing/classifier.py
Parser	Regex-based field extraction	parsing/parser.py
Validator	Numeric sanity and presence checks	parsing/validator.py

Sources: README.md:34-38, parsing/parser.py:28-76, parsing/validator.py:17-50

2. Tax Calculation Engine

The taxcalc.py module performs federal tax computations. It supports progressive tax brackets for different filing statuses (Single, Married) and calculates the final tax due or refund after accounting for standard deductions and withholdings.

def compute_tax_estimate(extracted_fields, filing_status='single', withholding=0.0):
    # Extracts wages/amounts and computes AGI
    # Applies standard deductions (e.g., $13,850 for Single)
    # Computes progressive tax based on loaded brackets
    # Returns AGI, Taxable Income, Gross Tax, and Tax Due

Sources: taxcalc.py:10-15, taxcalc.py:61-103

3. Smart Field Mapping and Form Generation

Rosy utilizes semantic embeddings (via sentence-transformers) to map extracted data fields to the specific AcroForm field names found in the Form 1040 PDF template.

• Auto-Mapper: Uses all-MiniLM-L6-v2 to compute cosine similarity between extracted labels and PDF template names.
• Fallback: If the ML model is unavailable, it uses a normalized string similarity (Levenshtein) fallback.
• Filling: Uses pdfrw to write values directly into PDF Widget annotations.

Sources: utils/auto_mapper.py:27-55, forms.py:116-150

API and Interaction Flow

The system interacts with the user via a Flask web server. The /upload endpoint handles document submission, while the /finalize endpoint triggers the final PDF generation after the user has reviewed and potentially edited the extracted fields.

Security and Data Privacy

Rosy is designed as a local-first system to ensure data privacy. Key security features include:

• PII Masking: SSNs are masked (e.g., XXX-XX-6789) in API responses before being sent to the frontend.
• File Sanitization: Uploaded filenames are cleaned of dangerous characters and path traversal attempts.
• Cleanup: Temporary directories are created for each session and processed files are deleted after ingestion.

Sources: security.py:25-30, security.py:41-65, backend.py:59-63

Conclusion

The Rosy system provides a comprehensive prototype for automating the tax preparation workflow. By combining heuristic parsing with semantic AI mapping and a standard tax calculation engine, it demonstrates how unstructured tax documents can be transformed into structured, fillable PDF forms with minimal manual intervention.

Sources: README.md:120-130