Field Parsing & Extraction
Relevant source files
The following files were used as context for generating this wiki page:
The Field Parsing & Extraction module is the core intelligence component of the Rosy Tax Processor. Its primary purpose is to transform raw text—obtained from OCR or direct PDF extraction—into structured data objects representing tax-related entities such as wages, withholdings, and identification numbers. This module supports specific tax forms including W-2 and 1099, employing a heuristic and regular expression-based approach to identify key financial data points.
Sources: parsing/parser.py:1-5, README.md:10-15
Architectural Overview
The extraction process is integrated into a larger processing pipeline that moves from document ingestion to form generation. The parsing logic is decoupled from the file format, operating on plain text strings. It relies on a classification step to determine which regular expression templates to apply.
The following diagram illustrates the data flow from raw input to structured, validated fields:
The pipeline orchestrates the transition from document-type detection to specific field extraction and subsequent validation.
Sources: pipeline.py:12-32, parsing/parser.py:31-85
Core Extraction Logic
The extraction logic is primarily housed in parsing/parser.py. It utilizes pre-defined regular expressions to identify standard patterns for Employer Identification Numbers (EIN), Social Security Numbers (SSN), and currency values.
Regex Patterns
The system uses the following compiled regular expressions for data identification:
_number_re: Matches currency-formatted strings including commas and decimals._ein_re: Matches theXX-XXXXXXXpattern for Employer Identification Numbers._ssn_re: Matches theXXX-XX-XXXXpattern for Social Security Numbers.
Sources: parsing/parser.py:8-10
Template-Based Extraction
The extract_fields function applies specific logic based on the detected doc_type:
| Document Type | Extracted Fields | Logic Description |
|---|---|---|
| W-2 | wages, ein, employee_ssn, federal_income_tax_withheld | Searches for "Box 1" or "wages" for income. Looks for "Box 2" or "federal income tax withheld" for tax. |
| 1099 | payer_ein, recipient_ssn, amount | Searches for "nonemployee compensation", "Box 1", or "amount". |
| Generic | ein, ssn | Applies global regex searches for EIN and SSN patterns if no specific type is matched. |
Sources: parsing/parser.py:31-85
Validation & Sanity Checks
Once fields are extracted, the parsing/validator.py module performs sanity checks to ensure data integrity before the information is used in tax calculations.
Validation Rules
The validator checks for required presence and numeric ranges:
- W-2 Validation: Ensures
wagesare present, parseable as numbers, non-negative, and do not exceed $10,000,000. - 1099 Validation: Ensures
amountis present and non-negative. - Generic Validation: Issues a warning if no fields were extracted at all.
Sources: parsing/validator.py:18-50
Pipeline Integration & Aggregation
The extraction module is invoked by pipeline.py, which manages the processing of multiple documents. The pipeline aggregates data from various sources to calculate totals.
The sequence shows how the pipeline coordinates different modules to produce a consolidated tax result.
Sources: pipeline.py:46-85
Semantic Auto-Mapping
When the extracted fields need to be mapped to a formal PDF template (such as a 1040 form), the system uses utils/auto_mapper.py. This component handles discrepancies between internal field names (e.g., federal_income_tax_withheld) and PDF AcroForm field names (e.g., FederalIncomeTaxWithheld).
- Primary Strategy: Uses
sentence-transformers(MiniLM-L6-v2) to calculate cosine similarity between field names. - Fallback Strategy: Uses
difflib.SequenceMatcherfor string similarity if ML libraries are unavailable.
Sources: utils/auto_mapper.py:20-64
Security and PII Masking
Because the extraction process handles sensitive information like SSNs, the system includes a masking layer in security.py. Before extraction results are returned via the API in backend.py, the mask_pii_in_result function redacts SSNs, converting them to the format XXX-XX-6789.
Sources: security.py:27-55, backend.py:100-101
Summary
The Field Parsing & Extraction module provides a robust, template-based foundation for tax document automation. By combining regex-based extraction with semantic mapping and strict validation, it ensures that raw document data is reliably converted into high-fidelity inputs for the tax calculation engine and PDF generation modules.