Semantic Auto-Mapping

Relevant source files

The following files were used as context for generating this wiki page:

• utils/auto_mapper.py
• tests/test_auto_mapper.py
• forms.py
• pipeline.py
• tools/generate_sample_1040_template.py
• README.md

Semantic Auto-Mapping is a core technical feature of the Rosy Tax Processor designed to bridge the gap between extracted tax data and the specific requirements of fillable PDF forms. It uses natural language processing (NLP) to intelligently match parsed fields (like "wages") to corresponding AcroForm field names in a 1040 template (like "FormWages" or "Box1Wages"), even when the names are not identical.

Sources: utils/auto_mapper.py:1-10, README.md:14-25

This system ensures that the automation pipeline remains resilient to changes in PDF templates or variations in document extraction results. By calculating similarity scores between field labels, the system can automatically populate complex tax forms with minimal manual intervention.

Sources: forms.py:145-175, pipeline.py:80-95

Architecture and Logic

The auto-mapping logic is encapsulated within the utils/auto_mapper.py module. It employs a tiered strategy for field matching, prioritizing high-accuracy semantic embeddings while providing a robust string-similarity fallback.

Tiered Matching Strategy

1. Semantic Similarity (Primary): Utilizes the sentence-transformers library and the all-MiniLM-L6-v2 model to generate vector embeddings for both parsed keys and template field names. It then calculates the cosine similarity to find the best match.
2. Normalized String Similarity (Fallback): If the transformer library is unavailable, the system uses difflib.SequenceMatcher combined with a token-based bonus system to identify matches based on common substrings and character patterns.

Sources: utils/auto_mapper.py:13-68

Logic Flow Diagram

The following diagram illustrates how the map_fields function determines the best match for a PDF form field.

Integration with Form Filling

The auto-mapper is integrated directly into the form generation pipeline. When generate_1040_draft is called, it first attempts an "exact mapping" for known standard fields. If the results are insufficient (e.g., fewer than 5 matches), it invokes the semantic auto-mapper to fill the remaining gaps.

Sources: forms.py:165-185, tests/test_auto_mapper.py:5-25

Mapping Components

Component	Description	Source File
map_fields	Public API for matching a dictionary of values to a list of target field names.	utils/auto_mapper.py
_normalize_label	Strips non-alphanumeric characters and lowercases strings for comparison.	utils/auto_mapper.py:13-14
create_exact_field_map	Hardcoded mappings for high-confidence primary fields (Wages, SSN, etc.).	forms.py:28-52
all-MiniLM-L6-v2	The specific NLP model used for generating semantic embeddings.	utils/auto_mapper.py:45

Data Flow in the Pipeline

The following sequence diagram shows how the auto-mapping feature interacts with the broader system during the finalization of a tax return.

Implementation Details

Similarity Thresholding

The system does not blindly accept all matches. In forms.py, a confidence threshold of 0.4 is applied to auto-mapped results to prevent erroneous data entry into the tax forms.

Sources: forms.py:189-192

Fallback Similarity Calculation

The fallback mechanism enhances basic Levenshtein distance by checking if significant tokens (length > 2) from the source key appear within the target template name. Each matching token provides a 0.15 bonus to the base similarity ratio.

Sources: utils/auto_mapper.py:17-31

Field Discovery

To provide the auto-mapper with targets, the system uses a recursive function _collect_fields to traverse the PDF's AcroForm tree, handling both Fields and nested Kids arrays within the PDF structure.

Sources: forms.py:147-167

Conclusion

The Semantic Auto-Mapping system is a critical abstraction layer in the Rosy Tax Processor. By decoupling the extraction logic from the specific layout of the Form 1040 PDF, it allows the system to support various template versions and synthetic forms (like those generated by tools/generate_sample_1040_template.py) without requiring manual code updates for every field name variation.

Sources: README.md:27-35, tools/generate_sample_1040_template.py:11-30