Markdown Parsing & AST

Relevant source files

The following files were used as context for generating this wiki page:

• Writer/MarkdownRenderer.swift
• Writer/MarkdownTextView.swift
• Writer/TableFormatter.swift
• Writer/WebView.swift
• Writer/HTMLExporter.swift
• Writer/EditorStore.swift

The Writer application utilizes a dual-approach system for handling Markdown content. It leverages Apple's Markdown framework for structural parsing into an Abstract Syntax Tree (AST) for preview and export, while employing a regex-based engine for real-time syntax highlighting within the editor. This architecture ensures that complex documents are rendered accurately in the WebView while maintaining high performance during text entry in the MarkdownTextView.

Sources: Writer/MarkdownRenderer.swift:105-136, Writer/MarkdownTextView.swift:85-115, CLAUDE.md:20-35

The Rendering Pipeline

The transformation of raw Markdown text into rendered output follows a structured pipeline. The MarkdownRenderer acts as the primary coordinator, first escaping HTML entities to prevent injection and then invoking the Markdown framework's parser.

Structural Flow

The process begins with the Document(parsing:) call, which generates the AST. A custom HTMLVisitor then traverses this tree to produce HTML strings. Finally, post-processing is applied to handle custom syntax not natively supported by the AST parser, such as the ==highlight== syntax.

The diagram above illustrates the sequence from raw input to the final HTML used for previews and exports.
Sources: Writer/MarkdownRenderer.swift:125-136, Writer/MarkdownRenderer.swift:163-174

AST Traversal with HTMLVisitor

The HTMLVisitor struct implements the MarkupVisitor protocol to convert AST nodes into specific HTML elements. It recursively visits nodes and their children to build a continuous string.

Node Handling

The visitor explicitly handles structural, block, and inline elements as defined in the following table:

AST Node Type	HTML Tag / Implementation	Context
Heading	<h1...h6>	Uses heading.level to determine tag.
Paragraph	<p>	Standard block wrapper.
CodeBlock	<pre><code class="language-...">	Extracts language attribute for syntax highlighting.
Link	<a href="...">	Maps destination and child content.
Table	<table>	Processes TableHead, TableBody, TableRow, and TableCell.
ThematicBreak	<hr>	Represented by three or more dashes/asterisks.
Emphasis	<em>	Handles *italic* or _italic_.

Sources: Writer/MarkdownRenderer.swift:14-103

Implementation Detail: List Handling

Ordered and unordered lists are processed by visiting child ListItem nodes.

mutating func visitUnorderedList(_ unorderedList: UnorderedList) -> String {
    let content = unorderedList.children.map { $0.accept(&self) }.joined()
    return "<ul>\n\(content)</ul>\n"
}

mutating func visitListItem(_ listItem: ListItem) -> String {
    let content = listItem.children.map { $0.accept(&self) }.joined()
    return "<li>\(content)</li>\n"
}

Sources: Writer/MarkdownRenderer.swift:65-78

Editor Syntax Highlighting

Unlike the preview pane, the MarkdownTextView does not use the full AST for performance reasons. Instead, MarkdownTextViewInternal uses a set of cached regular expressions to apply styles to the NSTextStorage.

Cached Regex Patterns

To minimize overhead during typing, the editor pre-compiles regex patterns for common Markdown elements.

Highlighting Logic

When the text changes, applyMarkdownStyling() resets the attributes of the entire textStorage and then reapplies styles based on regex matches. The applyHeadingStyles method specifically calculates indentation and font weight based on the number of # characters detected at the start of lines.

Sources: Writer/MarkdownTextView.swift:117-150

Table Detection and Formatting

The project includes a specialized TableFormatter that provides advanced parsing for GFM-style tables. This module is responsible for detecting table ranges and calculating visual widths, especially for Unicode/CJK characters.

Visual Width Calculation

The parser accounts for the fact that CJK (Chinese, Japanese, Korean) characters and emojis occupy two positions in a monospaced layout, whereas standard ASCII characters occupy one.

Character Range	Visual Width	Logic
ASCII / Standard	1	Default increment.
CJK Unified Ideographs	2	0x4E00 to 0x9FFF
Emoji and Symbols	2	0x1F300 to 0x1F9FF
Hiragana / Katakana	2	0x3040 to 0x30FF

Sources: Writer/TableFormatter.swift:147-172

Table Formatting Workflow

Users can trigger the formatAllTables() command via the EditorStore, which uses the following logic:

1. Detection: detectTableRanges(in:) identifies lines containing pipes (|) and valid delimiter rows (dashes and colons).
2. Parsing: parseTableRow(_:) splits rows into cells and trims whitespace.
3. Alignment: calculateColumnWidths(rows:) extracts alignment markers (:---, :---:, ---:) from the delimiter row.
4. Formatting: formatTable(_:) reconstructs the table with consistent padding.

Sources: Writer/TableFormatter.swift:34-77, Writer/TableFormatter.swift:207-238, Writer/EditorStore.swift:348-351

Conclusion

The Markdown Parsing & AST system in Writer balances the need for a rich, standards-compliant preview with a responsive, styled editor. By combining Apple's Markdown framework for the heavy lifting of AST generation with custom regex-based styling for the UI, the application provides a seamless distraction-free environment that supports both standard Markdown and custom extensions like text highlighting and automated table alignment.