Parser Package Implementation
Relevant source files
The following files were used as context for generating this wiki page:
The packages/parser module is responsible for transforming raw UTF-8 chemd source text into a structured, normalized Semantic Abstract Syntax Tree (AST). It serves as the initial phase of the cheMD compilation pipeline, handling three distinct layers of content: YAML frontmatter metadata, standard Markdown prose, and specialized chemical block/inline syntax.
The parser implementation adheres to a strict separation between semantic content (e.g., molecule data) and rendering style (e.g., bond lengths). This ensures that the resulting AST is a pure representation of experimental data, while render-specific configurations are handled by the Render Profile system.
Sources: chemd-v0.1-architecture.md:46-52, chemd-v0.1-spec.md:27-38
Core Parsing Architecture
The parsing process is designed as a multi-stage pipeline rather than a single-pass execution. This separation of concerns allows for independent syntax detection, normalization, and diagnostic attachment.
The diagram above illustrates the sequential stages of the parser pipeline from raw input to normalized AST.
Sources: chemd-v0.1-architecture.md:91-105
Entry Point: parseChemd
The primary API provided by the package is parseChemd(source: string), which returns a ParsedChemdDocument. This document contains the parsed metadata, a list of child nodes (both markdown and semantic), and any diagnostics encountered during the process.
Sources: packages/parser/tests/parser.test.ts:54-106, chemd-v0.1-architecture.md:369-371
Frontmatter Parsing
The parser utilizes the YAML library to extract and validate document-level metadata located at the start of the file between --- fences.
Metadata Validation
The parser enforces strict rules for several core frontmatter keys:
- Required Keys:
id,title, anddatemust be present. If missing, the parser emitsE_MISSING_REQUIRED_FRONTMATTER_KEYand provides default values like "draft-document". - Date Format: Dates must follow the ISO
YYYY-MM-DDformat. The parser validates both the regex pattern and the actual calendar validity (e.g., rejecting Feb 30th). - Render Selection: Extracts
render_profileandrender_overrides. Overrides are validated against a known allowlist of paths likestructure.bondLineWidth.
Sources: packages/parser/src/frontmatter/parse-frontmatter.ts:7-25, packages/parser/src/frontmatter/parse-frontmatter.ts:130-184, packages/parser/src/frontmatter/parse-frontmatter.ts:489-510
Diagnostic Handling in Frontmatter
The frontmatter parser is designed to be resilient; it reports errors but continues parsing the rest of the document.
| Diagnostic Code | Severity | Description |
|---|---|---|
W_INVALID_FRONTMATTER_LINE | warning | Encountered a line that does not follow key-value syntax. |
W_DUPLICATE_FRONTMATTER_KEY | warning | A key appeared multiple times; the last value is used. |
E_INVALID_FRONTMATTER_VALUE | error | Value type mismatch (e.g., non-string for ID) or unsupported nested structures. |
W_NON_ISO_FRONTMATTER_DATE | warning | Date does not match YYYY-MM-DD pattern. |
Sources: packages/parser/src/frontmatter/parse-frontmatter.ts:31-50, packages/parser/src/frontmatter/parse-frontmatter.ts:130-150
Block Parsing and Normalization
Structured blocks use a fenced directive syntax: :::block_name #id. The parser identifies these blocks and transforms their internal key-value pairs into node properties.
Standard Block Types
The parser recognizes several semantic block types defined in the v0.1 specification:
- Molecule: Captures SMILES, names, and roles.
- Reaction: Handles reactants, products, and conditions.
- Result: Standardizes yield, conversion, and purity data.
- Analysis/Sample: Captures analytical metadata and material sourcing.
- Template/Use: Handles reusable markdown fragments and their injection sites.
Sources: chemd-v0.1-spec.md:120-140, packages/parser/tests/parser.test.ts:74-95
Field Normalization
Fields within blocks are normalized according to specific rules:
- List Splitting: Fields like
reactants,products, andconditionsuse a|separator. The parser splits these strings and trims whitespace for each item. - ID Extraction: IDs are introduced with
#in the block header and must match the regex[a-zA-Z][a-zA-Z0-9_-]*. - Pruning: Unknown fields produce a
W_UNKNOWN_FIELDwarning and are not included in the semantic node to prevent "dirtying" the AST.
Sources: chemd-v0.1-spec.md:104-118, packages/parser/tests/parser.test.ts:522-540
Inline Syntax and Tokenization
The parser detects specific inline patterns within Markdown blocks, creating specialized token nodes for the resolver.
Chemical Formulas: :chem[...]
The :chem syntax is used for molecular and ionic formulas. The parser extracts the content within brackets to be rendered via KaTeX/mhchem downstream.
- Example:
:chem[H2O]is parsed into a token with valueH2O.
Sources: chemd-v0.1-spec.md:79-93, packages/parser/tests/parser.test.ts:311-325
References: @id
The parser identifies reference tokens that allow documents to dynamicallly link to other objects or metadata:
- Object Reference:
@rxn-main - Field Reference:
@res-main.yield - Meta Reference:
@meta.project - Alias Reference:
@reaction.temperature(used within templates).
Sources: chemd-v0.1-spec.md:204-222, packages/parser/tests/parser.test.ts:100-106
Position Tracking
For all markdown tokens (references, inline chem, links, inline code), the parser tracks precise source ranges, including start, end, startLine, and startColumn. This is critical for error reporting and UI highlighting.
Sources: packages/parser/tests/parser.test.ts:10-48, packages/parser/tests/parser.test.ts:365-395
Data Structures (AST)
The parser produces a tree structure where the root is a document type.
The class diagram shows the hierarchy of the primary AST nodes generated by the parser.
Sources: chemd-v0.1-spec.md:323-358, chemd-v0.1-architecture.md:154-180
Conclusion
The packages/parser implementation provides the foundational semantic layer for the cheMD project. By strictly separating experimental data from visual styles and providing a robust diagnostic system, it enables a reliable multi-output pipeline (HTML, JSON, SVG, DOCX). Its hybrid approach—combining YAML parsing for metadata, standard Markdown for prose, and custom directive parsing for chemistry—ensures the system remains extensible while maintaining compatibility with standard Markdown editors.
Sources: chemd-v0.1-architecture.md:10-25, chemd-v0.1-spec.md:417-430