Content Scripts & Extractors
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Content Scripts and Extractors module serves as the primary data acquisition layer for the MarketSage extension. This system is responsible for identifying supported e-commerce platforms, parsing Document Object Model (DOM) structures, and transforming raw web content into structured product data. By operating directly within the context of the user's browser tabs, these scripts enable real-time analysis of products on Chinese marketplaces like Temu, AliExpress, and DHgate.
The architecture relies on specialized extractor services that utilize platform-specific selectors to isolate critical information such as product IDs, pricing, seller credentials, and descriptive metadata. This extracted data forms the foundation for downstream features including scam detection, price analytics, and AI-powered shopping assistance.
Sources: src/manifest.json:20-28, src/content/services/productExtractor.ts:1-20, src/common/types/index.ts:1-10
Extraction Architecture & Flow
The extraction process is orchestrated by the ProductExtractor, which delegates specialized tasks to the PriceExtractor and PlatformDetector. The workflow begins with a platform support check and culminates in the generation of a unified Product object.
Extraction Sequence
The following diagram illustrates the sequence of events when a product is analyzed on a supported page:
The extraction logic uses a "fallback" strategy, attempting to find data in meta tags before searching URL parameters or specific CSS selectors.
Sources: src/content/services/productExtractor.ts:13-33, tests/unit/services/priceExtractor.test.ts:30-45
Core Extractor Components
ProductExtractor
The ProductExtractor class is the main entry point for content parsing. It aggregates data from various parts of the webpage to build a comprehensive product profile.
- Product ID Extraction: Attempts to retrieve the ID from
<meta property="product:id">, followed by URL parameters (idorproductId), and finally the URL pathname. - Metadata Extraction: Normalizes titles and descriptions using utility functions to ensure consistency across different marketplace formats.
- Image Processing: Scans multiple gallery selectors (e.g.,
[data-role="product-image"],.product-gallery img) and filters out placeholder images or empty sources.
Sources: src/content/services/productExtractor.ts:35-85, tests/unit/services/productExtractor.test.ts:45-120
PriceExtractor
The PriceExtractor specializes in parsing financial data. It is designed to handle currency symbols and calculate discounts dynamically.
| Feature | Logic Description |
|---|---|
| Currency Detection | Scans price text for symbols (e.g., $, €, £) to assign the correct currency code. |
| Numeric Conversion | Strips non-numeric characters and handles various decimal/thousand separators. |
| Discount Calculation | Compares current and original prices to derive a percentage-based discount. |
| Error Handling | Returns a default price object (current: 0) if extraction fails, preventing script crashes. |
Sources: src/content/services/priceExtractor.ts:1-15, tests/unit/services/priceExtractor.test.ts:47-95, tests/unit/utils/dataProcessing.test.ts:5-30
Data Structures
The extractors output data strictly following the project's common type definitions. This ensures that the background services receiving the data can perform analysis without further transformation.
Unified Product Schema
Sources: src/common/types/index.ts:13-35
Platform Configuration & Targeting
The extension defines specific host permissions and content script matches to restrict execution to valid marketplaces.
| Platform | Domain Match Pattern | Default Currency |
|---|---|---|
| Temu | *://*.temu.com/* | USD |
| AliExpress | *://*.aliexpress.com/* | USD |
| DHgate | *://*.dhgate.com/* | USD |
Sources: src/manifest.json:10-18, src/common/types/index.ts:3-8, tests/unit/services/priceExtractor.test.ts:100-110
Implementation Details
The extractors utilize specific DOM querying strategies to ensure reliability across frequently updated marketplace frontends.
// Example of Image Extraction Strategy
private extractImages(): string[] {
const selectors = [
'img[data-role="product-image"]',
'.product-gallery img',
'.product-image img'
];
// ... iteration and normalization logic
}
// Example of Seller ID Extraction from Store Links
const storeLink = document.querySelector('a[href*="store"]');
if (storeLink) {
const storeIdMatch = href.match(/store\/(\d+)/);
if (storeIdMatch) return storeIdMatch[1];
}
Sources: src/content/services/productExtractor.ts:70-85, src/content/services/productExtractor.ts:120-130
Conclusion
Content Scripts and Extractors are vital for the MarketSage ecosystem, bridging the gap between unstructured web content and the platform's intelligent analysis engines. By utilizing a modular service architecture and standardized data types, the system maintains high accuracy in product identification and price tracking across diverse Chinese e-commerce platforms.