The Python Builder Pipeline

Relevant source files

The following files were used as context for generating this wiki page:

• build_story_artifact.py
• diagram_inventory.json
• README.md
• story_template.html
• requirements-build.txt

The Python Builder Pipeline extracts visual assets and structured metadata from the GestaltView_Master_Wiki_v4.0.pdf to generate an interactive, offline-capable static website. It transforms a linear document into a searchable, multi-layered visual story by capturing page regions, rendering Mermaid diagrams, and injecting data into a standalone HTML artifact.

Sources: build_story_artifact.py:2-4, README.md:1-5

Pipeline Architecture and Data Flow

The pipeline operates as a batch processing system that consumes a PDF source and produces three primary outputs: optimized image assets, a JSON inventory, and a self-contained index.html.

System Flow Diagram

The following diagram illustrates the transformation from raw PDF to the final interactive artifact.

The diagram shows the sequence of extraction, asset generation, and final embedding into the static site.
Sources: build_story_artifact.py:348-380, README.md:46-52

Core Components and Extraction Logic

The builder utilizes pymupdf (fitz) for PDF manipulation and PIL for image optimization. It categorizes visuals into three distinct types: Curated Diagrams, Captioned Figures, and Mermaid Source Snippets.

Visual Categories

Category	Description	Source Implementation
Curated Diagrams	13 high-level architecture maps found in the wiki's opening sections.	make_curated_items()
Captioned Figures	39 figures identified by "Figure X: Title" regex patterns throughout the PDF.	make_figure_items()
Mermaid Snippets	Raw Mermaid code blocks extracted and rendered as PNGs.	render_mermaid_items()

Sources: build_story_artifact.py:16-30, build_story_artifact.py:221-230, build_story_artifact.py:255-265, diagram_inventory.json:5-11

Extraction Mechanics

1. Coordinate Mapping: The script identifies section headings and figure captions to calculate bounding boxes (Rects) for cropping.
2. Normalization: It applies character normalization (norm() function) to fix common PDF extraction artifacts like broken ligatures (e.g., "fi", "ffi") and whitespace issues.
3. Image Processing: save_crop() renders PDF page regions at a 2.4x scale and saves them as JPEGs with 91% quality to balance clarity and file size.
4. Mermaid Rendering: The system uses manus-render-diagram to convert text-based Mermaid snippets into PNG assets before embedding.

Sources: build_story_artifact.py:182-198, build_story_artifact.py:210-216, build_story_artifact.py:318-325, README.md:52-54

Data Persistence and Artifact Injection

The final stage of the pipeline centralizes all data for zero-dependency hosting.

The Inventory Schema

The diagram_inventory.json file serves as the ground truth for the application. It maps every visual to its chapter, source page, and technical explanation.

{
  "id": "curated-01",
  "kind": "curated",
  "page": 1,
  "title": "GestaltView at a Glance",
  "asset_path": "story_assets/curated-01.jpg",
  "explanations": {
    "orientation": "...",
    "system": "...",
    "architecture": "..."
  }
}

Sources: build_story_artifact.py:370-385, diagram_inventory.json:105-115

Self-Contained Embedding

To ensure the guide remains hostable without a backend, the pipeline embeds all images as Base64 strings. It replaces the __APP_DATA__ placeholder in story_template.html with the complete JSON payload, including these Base64 strings.

def embed_images(items: list[dict[str, Any]]) -> None:
    for item in items:
        asset = ROOT / item["asset_path"]
        mime = "image/png" if asset.suffix.lower() == ".png" else "image/jpeg"
        item["image_data"] = f"data:{mime};base64," + base64.b64encode(asset.read_bytes()).decode("ascii")

Sources: build_story_artifact.py:339-345, README.md:9-12

Validation and Deployment

The pipeline includes a validation suite to ensure source provenance and integrity.

• Acceptance Tests: The script checks that the final inventory contains exactly 54 visuals (13 curated, 39 figures, 2 Mermaid snippets).
• Environment: Requirements include pymupdf and Pillow. Rebuilding also requires the manus-render-diagram utility.
• Hosting: The resulting index.html requires no frontend build step (no Node.js or API keys) and can be served via simple HTTP servers.

Sources: build_story_artifact.py:361-368, README.md:9-16, README.md:55-60

The Python Builder Pipeline ensures that complex architectural documentation remains accessible and interactive by bridging the gap between static PDF sources and modern web-based visual storytelling.