Product Service Layer

Relevant source files

The following files were used as context for generating this wiki page:

• plan.md
• CLAUDE.md
• src/scrapers/base/BaseScraper.ts
• src/scrapers/turkey/MigrosScraper.ts
• src/scrapers/spain/MercadonaScraper.ts

The Product Service Layer is a core component of the "WhereIsLifeCheaper" system, responsible for the lifecycle management of grocery product data. It sits between the modular scraping architecture and the persistent storage layer, facilitating the intake, normalization, and cross-country matching of product information. Its primary objective is to transform raw, heterogeneous data scraped from various international supermarkets into a standardized format that enables accurate cost-of-living comparisons.

This layer handles complex tasks such as extracting quantity information (e.g., "1.5L", "500g"), mapping supermarket-specific items to a master catalog, and maintaining historical price points for trend analysis. It ensures that a product like "Coca-Cola" in Turkey can be programmatically compared to its equivalent in Spain or Montenegro.

Sources: plan.md:7-13, CLAUDE.md:7-12

Core Architecture and Data Flow

The Product Service Layer operates within a multi-tiered architecture. Data enters the system via modular scrapers, passes through the service layer for processing, and is finally committed to a PostgreSQL database.

Data Ingestion Pipeline

The following diagram illustrates the flow from a web request to a stored product record:

The pipeline begins when the Scraper Service orchestrates a run. Scrapers extend BaseScraper to handle browser automation and raw data extraction. Once raw product data is gathered, it is passed to the Product Service.
Sources: plan.md:20-30, CLAUDE.md:46-55

Components of the Product Service

Component	Responsibility	Relevant Files
Product Normalizer	Standardizes names (lowercase, removing special characters) and extracts units.	src/utils/normalizer.ts
Matching Engine	Identifies existing products via barcode or brand/name/unit combinations.	src/services/product.service.ts
Price Manager	Records current price points and links them to historical product_mappings.	src/services/price.service.ts

Sources: plan.md:143-157, CLAUDE.md:33-36

Product Normalization & Extraction

A critical function of the service layer is resolving inconsistencies in how products are listed across different retailers.

Normalization Logic

The system applies a specific normalization strategy to product strings to facilitate matching:

1. Case Normalization: Converting all names to lowercase.
2. Character Stripping: Removing non-alphanumeric characters.
3. Whitespace Normalization: Collapsing multiple spaces into a single space.

// Example Logic from Plan
function normalizeProductName(name: string): string {
  return name
    .toLowerCase()
    .replace(/[^a-z0-9\s]/g, '')
    .replace(/\s+/g, ' ')
    .trim();
}

Sources: plan.md:243-255

Quantity and Unit Extraction

Scrapers use specialized utilities to parse strings like "1.5L PET" or "500g" into distinct unit (e.g., 'l', 'g', 'kg') and unitQuantity fields. This allows the system to calculate "price per unit," which is essential for fair comparison across different packaging sizes.
Sources: src/scrapers/spain/MercadonaScraper.ts:312-340, src/scrapers/turkey/MigrosScraper.ts:219-245

Product Matching Strategy

To enable cross-country comparisons, the service layer implements a prioritized matching algorithm. This prevents the duplication of "Master" products in the catalog when they are found in different supermarkets.

Matching Priority

The ProductService attempts to find a match using the following hierarchy:

1. Exact Barcode Match: The most reliable method, using the universal barcode (if available).
2. Brand + Normalized Name + Unit: Matches based on standardized identifying attributes.
3. Fuzzy Matching: Within a specific category to catch minor naming variations.
4. Manual Mapping: Using a product_mappings or canonical_products table for curated links.

Database Schema Integration

The Product Service Layer interacts with several core tables in the PostgreSQL database to maintain the product catalog.

Table	Purpose in Service Layer
products	The master catalog containing normalized_name, brand, unit, and barcode.
product_mappings	Links a master product to a specific supermarket's external_id and url.
prices	Stores time-series data: price, is_on_sale, and scraped_at timestamp.
canonical_products	Acts as a cross-country identifier for equivalent items (e.g., "Standard Milk").

Sources: plan.md:85-110, CLAUDE.md:33-44

Implementation Example: Scraper to Service Interaction

The concrete scraper implementations (like MigrosScraper or MercadonaScraper) do not save to the database directly. Instead, they format data into a ProductData object and return it to the service layer.

// From MercadonaScraper.ts
const product: ProductData = {
  name: productName,
  price: price,
  currency: 'EUR',
  originalPrice: isOnSale ? previousPrice : undefined,
  isOnSale: !!isOnSale,
  imageUrl: item.thumbnail,
  productUrl: item.share_url,
  brand,
  unit,
  unitQuantity,
  isAvailable: true,
  externalId: item.id,
};

Sources: src/scrapers/spain/MercadonaScraper.ts:269-283

Conclusion

The Product Service Layer is the architectural pivot that enables the transition from raw web-scraped data to actionable economic insights. By implementing rigorous normalization, multi-stage product matching, and a structured data model, it provides the foundation for comparing grocery prices across diverse markets like Turkey, Spain, and Ukraine. This layer ensures data integrity and consistency, which are paramount for a reliable cost-of-living comparison tool.