String Normalization
Relevant source files
The following files were used as context for generating this wiki page:
The String Normalization system within the beverage-utils library provides a suite of deterministic utilities designed to canonicalize free-text identifiers, resolve country references, and parse inconsistent volume strings. Its primary purpose is to ensure that signal comparisons—such as regions, country aliases, and classification labels—remain consistent regardless of how the input data was originally formatted by a producer.
This system is critical for maintaining "apples-to-apples" comparisons across the CellarNode platform, particularly when dealing with international data that may include varied character encodings, diacritics, or localized naming conventions.
Sources: src/normalize.ts:1-7, src/normalize.ts:25-28
Token Canonicalization
Token canonicalization is handled by the normalizeToken function. It transforms raw strings into a "kebab-case" format suitable for slugs and stable identifiers. The process involves several steps to remove stylistic differences while preserving the semantic meaning of the identifier.
Normalization Logic Flow
The following diagram illustrates the transformation steps applied to a raw string:
The diagram shows the sequential string manipulation used to create deterministic slugs.
Sources: src/normalize.ts:8-18, tests/normalize.test.ts:12-32
Key Features of Token Normalization
| Feature | Description | Example |
|---|---|---|
| Diacritic Removal | Strips accents and marks using NFD normalization. | España -> espana |
| Whitespace Handling | Converts spaces and underscores to single hyphens. | Red Wine -> red-wine |
| Hyphen Sanitization | Collapses multiple hyphens and removes them from ends. | --foo--bar-- -> foo-bar |
| Case Insensitivity | Forces all characters to lowercase. | BRANDY -> brandy |
Sources: src/normalize.ts:8-18, tests/normalize.test.ts:12-32
Country Code Resolution
The resolution system maps various country representations (English names, Spanish names, Swedish names, ISO Alpha-2, and ISO Alpha-3) to a canonical ISO Alpha-2 code. This ensures that country-based ranking and filtering compare standardized codes rather than inconsistent strings.
The resolveCountryCode function utilizes the i18n-iso-countries library and supports multiple locales (en, es, sv).
Sources: src/normalize.ts:20-37
Resolution Strategy
The logic prioritizes direct ISO matches before falling back to localized name lookups and diacritic-insensitive matching.
Sources: src/normalize.ts:38-66, tests/normalize.test.ts:40-69
Volume Parsing
The parseVolumeToLiters function converts free-text or numeric volume strings into a standardized numeric value representing liters. This is essential for the "capacity-fit" signal, allowing for comparisons across different units and packaging formats.
Supported Patterns
The parser handles several complex formats, including unit suffixes and bottle-count multipliers:
| Pattern Type | Description | Examples |
|---|---|---|
| Metric Suffixes | Supports kL, hL, L, mL, cL (case-insensitive). | 5 kL (5000), 750 ml (0.75) |
| Bottle Counts | Parses patterns like [Size][Unit] x [Count] or vice-versa. | bottles 750ml × 4000 (3000) |
| Liters (Aliases) | Recognizes liter, liters, litre, litres. | 5000 liters (5000) |
| Bare Numbers | Assumes numeric input or strings without units are liters. | 330 or 330.0 (330) |
Sources: src/normalize.ts:74-118, tests/normalize.test.ts:74-106
Implementation Detail: Bottle Count Regex
The parser uses a non-capturing group regex to identify bottle sizes and counts simultaneously:
// Example pattern from implementation
const bottleMatch = s.match(
/(?:bottles?\s+)?(\d+(?:\.\d+)?)\s*(ml|cl|l)\s*[×x]\s*(\d+(?:\.\d+)?)|(\d+(?:\.\d+)?)\s*[×x]\s*(\d+(?:\.\d+)?)\s*(ml|cl|l)/,
);
Sources: src/normalize.ts:86-88
Label Formatting and Humanization
While normalization simplifies strings for machine processing, the library also provides utilities to reverse this for display purposes. The formatBeverageLabel function attempts to look up a slug in a LabelMap; if no match is found, it "humanizes" the slug by replacing underscores with spaces and applying title case.
export function formatBeverageLabel(
key: string | null | undefined,
map?: LabelMap,
): string {
if (!key) return "";
if (map?.[key]) return map[key];
return key.replace(/_/g, " ").replace(/\b\w/g, (c) => c.toUpperCase());
}
Sources: src/format.ts:3-11, tests/format.test.ts:13-24
Summary
The string normalization module provides the foundation for data integrity within the beverage-utils ecosystem. By transforming inconsistent human input into canonical tokens (via normalizeToken), standard codes (via resolveCountryCode), and uniform metric values (via parseVolumeToLiters), it enables reliable data processing and comparison across diverse beverage classifications.