Adding New Tax Forms

Relevant source files

The following files were used as context for generating this wiki page:

• parsing/parser.py
• parsing/validator.py
• forms.py
• utils/auto_mapper.py
• pipeline.py
• tools/generate_sample_1040_template.py

Adding new tax forms to the Rosy Tax Processor involves updating the end-to-end pipeline to support document classification, field extraction, data validation, and PDF form filling. The system is designed with a modular architecture that separates the ingestion of raw text from the logical parsing and mapping of tax data.

Developers must implement logic across three primary layers: the Parsing Layer for identifying and extracting data from OCR/text input, the Validation Layer for ensuring data integrity, and the Form Generation Layer for mapping extracted data to PDF templates.

Document Parsing and Extraction

The first step in adding a new form is defining how the system identifies and extracts relevant fields from raw text. This is handled within the parsing module using heuristic and regex-based strategies.

Implementation Logic

1. Detection: The system must first identify the document type (e.g., W-2, 1099).
2. Extraction: Once identified, specific regex patterns or keyword proximity searches are used to find values like EINs, SSNs, or monetary amounts.

The following flowchart illustrates the data flow within the parser:

The extraction logic relies heavily on helper functions like _find_first_number_near(keyword, text) which searches for numeric values within a specific character window following a keyword.

Helper / Component	Description
_number_re	Regex for matching currency and numeric strings.
_ein_re	Regex for matching Employer Identification Numbers (XX-XXXXXXX).
_ssn_re	Regex for matching Social Security Numbers (XXX-XX-XXXX).
extract_fields	Main entry point for form-specific extraction logic.

Sources: parsing/parser.py:9-15, parsing/parser.py:32-40, pipeline.py:20-25

Data Validation

New forms require specific validation rules to ensure the extracted data is sane before being used in tax calculations. Validation is handled by validate_fields(fields, doc_type).

For a new form, developers should:

• Define mandatory fields (e.g., "missing wages").
• Implement numeric range checks (e.g., preventing negative values or unusually large amounts).
• Validate string formats for identifiers like EINs or SSNs.

Sources: parsing/validator.py:19-35, parsing/validator.py:48-52

PDF Template Integration and Auto-Mapping

The system supports filling AcroForm-based PDF templates. When adding a new form, the extracted data must be mapped to the specific field names defined inside the PDF.

Template Preparation

Developers should place the fillable PDF in the templates/ directory. If the PDF uses standard naming conventions, the auto_mapper.py utility can automatically correlate parsed fields with PDF fields using semantic similarity.

The Auto-Mapping Process

The map_fields function utilizes sentence-transformers (if available) to compute cosine similarity between extracted field keys and PDF field names. If the library is missing, it falls back to a Levenshtein-based string similarity.

Form Filling Functions

Function	File	Description
fill_fillable_1040	forms.py	Uses pdfrw to write values into PDF /Widget annotations.
generate_1040_draft	forms.py	Orchestrates the template search and mapping logic.
create_exact_field_map	forms.py	Provides a hardcoded mapping for known synthetic templates.

Sources: forms.py:118-142, forms.py:164-210, utils/auto_mapper.py:40-65, tools/generate_sample_1040_template.py:18-35

Pipeline Integration

To finalize the addition of a new form, the run_pipeline_on_paths function in pipeline.py must be aware of how to aggregate the new form's data.

# Example of aggregating new form data in the pipeline
for r in per_file:
    f = r.get('fields', {})
    try:
        if 'new_form_income_field' in f:
            total_wages += float(str(f['new_form_income_field']).replace(',', ''))
    except Exception:
        pass

Sources: pipeline.py:49-65, pipeline.py:84-90

Adding a new tax form requires updates to document detection, regex-based extraction, field validation, and the mapping logic that connects extracted data to PDF templates. By following the existing patterns in the parsing and forms modules, developers can extend the system to support a wide variety of IRS documents.