Financial History & Fundamental Normalization

Relevant source files

The following files were used as context for generating this wiki page:

• DO NOT DELETE EDL PIPELINE/build_quarterly_financial_ledger.py
• DO NOT DELETE EDL PIPELINE/src/edl_pipeline/transforms/fundamentals.py
• DO NOT DELETE EDL PIPELINE/src/edl_pipeline/scanner/financials.py
• DO NOT DELETE EDL PIPELINE/README.md
• DO NOT DELETE EDL PIPELINE/src/edl_pipeline/artifacts.py
• DO NOT DELETE EDL PIPELINE/tests/test_scanner_financials.py

Financial History & Fundamental Normalization aggregates, cleanses, and structures corporate financial data for the Nexus Scanner. The system converts raw, pipe-separated fundamental strings from provider responses into a normalized ledger of quarterly and annual statements. It performs complex joins between numeric financial results and regulatory filing timestamps to ensure auditable, point-in-time screening capabilities.

Sources: README.md:162-178, build_quarterly_financial_ledger.py:1-12

Data Architecture and Flow

The financial pipeline operates in distinct phases, moving from raw source snapshots to a standardized, standardized schema used by the frontend and the Python evaluation engine.

The diagram shows the convergence of numeric statement data and filing metadata into the final consolidated artifacts.
Sources: src/edl_pipeline/artifacts.py:68-120, README.md:273-280

Fundamental Normalization

The system normalizes raw data from ScanX, which often arrives as pipe-separated strings representing historical periods (e.g., "202606|202603").

Statement Processing Logic

• Unit Conversion: ScanX provides statement amounts in INR Crore. The pipeline converts these to INR Lakh for internal standardized fields while retaining Crore for specific historical summaries.
• Metric Extraction: The get_value_from_pipe_string function extracts specific periods from these strings, converting them to finite floats or None if data is missing or non-finite.
• Statement Selection: The system prioritizes CONSOLIDATED data. Standalone figures are only used if explicitly requested or if consolidated data is unavailable for a specific symbol. Standalone results never fill a missing consolidated result by default to prevent data inconsistency.

Sources: src/edl_pipeline/transforms/fundamentals.py:108-127, src/edl_pipeline/scanner/financials.py:20-45

Key Metrics and Derivations

Metric	Calculation / Source	Units
P/E Ratio	Stock Price / TTM EPS	Ratio
PEG Ratio	P/E / YoY EPS Growth %	Ratio
Debt to Equity	Total Borrowings / Total Equity (or ScanX Debt2Eq)	Ratio
TTM Revenue	Sum of last 4 consecutive quarters	INR Crore
Interest Coverage	(PBT + Interest) / Interest	Ratio

Sources: src/edl_pipeline/transforms/fundamentals.py:150-185, src/edl_pipeline/scanner/financials.py:85-110

Trailing Twelve Month (TTM) Engine

TTM calculations power the momentum-based fundamental screens. The engine enforces strict continuity rules to ensure data integrity.

• Growth Continuity: Growth calculations (e.g., TTM Revenue Growth) require eight consecutive valid quarters. The engine compares the latest four quarters against the preceding four.
• Gap Handling: If any period in the 8-quarter window is missing or non-finite, the growth metric is returned as null.
• Denominator Safety: If the absolute total of the prior period is zero, the growth percentage is returned as null to avoid division by zero errors.

Sources: tests/test_scanner_financials.py:35-48, src/edl_pipeline/scanner/financials.py:100-115

Quarterly Financial Ledger

The build_quarterly_financial_ledger.py script creates an auditable history by joining numeric results to specific regulatory disclosure timestamps.

Dating Logic

The system does not invent filing dates. It uses regular expressions to scan filing descriptors, captions, and news bodies for phrases like "quarter ended" or "results for".

DATE_PATTERNS = (
    re.compile(r"(?:quarter|quarterly|results?).{0,120}?(?:ended|ending)\s*(?:on\s*)?"
               r"([A-Za-z]+\s+\d{1,2},?\s+\d{4}|\d{1,2}\s+[A-Za-z]+\s+\d{4}|\d{4}-\d{2}-\d{2})", re.I),
)

Sources: build_quarterly_financial_ledger.py:27-32

Joint Verification

A record is only added to the ledger if:

1. A numeric quarter end exists in the fundamental_data.json series.
2. A corresponding filing exists in filing_history.json where the parsed "quarter end" matches the statement period.
3. The filing is identified as a result notice using terms like "Outcome of Board Meeting" or "Financial Results".

Sources: build_quarterly_financial_ledger.py:53-75, build_quarterly_financial_ledger.py:115-135

Historical Query Support

The Python evaluation engine supports complex historical expressions that access the normalized ledger. This allows users to screen for multi-quarter trends.

Supported Syntax

• Financial Value(frequency, metric, offset)
  • Frequency: quarterly or annual.
  • Metric: revenue, sales, net_profit, eps, opm, etc.
  • Offset: 0 (latest), 1 (previous), etc.

Example: Financial Value(quarterly, revenue, 0) > Financial Value(quarterly, revenue, 4) checks if current revenue is higher than the same quarter last year.

Sources: tests/test_scanner_financials.py:59-75, src/edl_pipeline/scanner/financials.py:60-80

Conclusion

Financial History & Fundamental Normalization ensures that the Nexus Scanner operates on high-quality, dated information rather than floating numeric averages. By enforcing consolidated statement priority and strict TTM continuity requirements, the system prevents misleading screens based on incomplete or standalone financial data.