Data Validation

Relevant source files

The following files were used as context for generating this wiki page:

• parsing/validator.py
• pipeline.py
• security.py
• backend.py
• parsing/parser.py
• tests/test_pipeline.py
• tests/test_security.py

Data Validation in the Rosy Tax Processor is a multi-layered system designed to ensure the integrity, security, and accuracy of tax document data as it moves from raw file uploads to final tax calculations. The system encompasses file-level security checks, heuristic field validation based on document types (e.g., W-2, 1099), and PII (Personally Identifiable Information) masking for secure data handling.

The validation process is integrated directly into the processing pipeline, triggering immediately after field extraction to identify missing values or numeric anomalies before data is passed to the tax calculation engine. This ensures that the generated Form 1040 is based on "sane" and verified inputs.
Sources: parsing/validator.py:1-5, pipeline.py:1-6, README.md:Core Capabilities

Validation Architecture and Data Flow

The validation logic is distributed across three primary areas: ingestion security, heuristic field checks, and output sanitization.

Pipeline Integration

Validation occurs as a discrete step in the document processing lifecycle. After text is ingested and fields are extracted, the validate_fields function evaluates the results.

The diagram shows the sequential flow where validation acts as a gatekeeper between extraction and data aggregation.
Sources: pipeline.py:19-35, backend.py:82-105

Ingestion and Security Validation

Before any parsing occurs, the system validates the physical file properties to prevent security vulnerabilities and resource exhaustion.

Security Constraints

The security.py module enforces strict limits on file uploads:

• File Size: A maximum limit of 5 MB per file is enforced to prevent Denial of Service (DoS) attacks.
• File Type: Only specific extensions are permitted: .pdf, .txt, .png, .jpg, and .jpeg.
• Filename Sanitization: Filenames are stripped of path components and dangerous characters, keeping only alphanumerics, dots, dashes, and underscores.

Component	Constraint	Implementation
Max Upload Size	5,242,880 bytes	MAX_UPLOAD_BYTES
Allowed Extensions	{'.pdf', '.txt', '.png', '.jpg', '.jpeg'}	ALLOWED_EXT
Filename Length	Max 200 characters	sanitize_filename

Sources: security.py:8-21, backend.py:89-95

Heuristic Field Validation

The core validation logic resides in parsing/validator.py, which performs type-specific checks on extracted data.

W-2 Validation Rules

For W-2 documents, the system focuses on "Box 1" (wages) and EIN formatting.

• Wages Presence: Checks if the wages field exists.
• Numeric Sanity: Validates that wages are parseable as floats, are non-negative, and do not exceed $10,000,000.
• EIN Check: Verifies that the Employer Identification Number is provided as a string.

1099 Validation Rules

For 1099 documents, the system validates the "Box 1" amount.

• Amount Presence: Ensures the amount field is extracted.
• Numeric Sanity: Checks for parseable floats and non-negative values.
• Payer EIN: Ensures the payer_ein is a valid string.

def validate_fields(fields: Dict[str, Any], doc_type: str) -> List[str]:
    issues: List[str] = []
    if doc_type == 'W-2':
        if 'wages' not in fields:
            issues.append('missing wages (Box 1)')
        else:
            val = _to_float(fields.get('wages'))
            if val is None:
                issues.append('wages not parseable as number')
            elif val < 0:
                issues.append('wages negative')
    # ... additional checks for 1099 and generic docs
    return issues

Sources: parsing/validator.py:18-53, tests/test_pipeline.py:22-29

PII Masking and Data Sanitization

To ensure data privacy in API responses, the system implements a recursive masking utility.

SSN Masking

The system uses regular expressions to detect Social Security Numbers in the format XXX-XX-XXXX. It masks the first five digits, preserving only the last four for identification purposes (e.g., 123-45-6789 becomes XXX-XX-6789).

Recursive Masking

The mask_pii_in_result function traverses the entire result dictionary, including nested dictionaries and lists, to apply masking to any string value containing sensitive patterns.

The diagram illustrates the recursive process of identifying and masking PII within complex data structures.
Sources: security.py:33-58, tests/test_security.py:22-35

Conclusion

Data validation in the Tax-Processor project serves as a critical bridge between unstructured document text and structured financial reporting. By combining rigid file security, heuristic business rules for tax forms, and automated PII masking, the system ensures that the resulting tax estimates are both computationally plausible and securely handled.
Sources: parsing/validator.py, security.py