Markdown Streaming
Relevant source files
The following files were used as context for generating this wiki page:
Markdown Streaming in M31A is a core UI/UX feature that enables the real-time, progressive rendering of AI-generated content within the terminal-based user interface (TUI). It allows users to see responses as they are being generated by Large Language Models (LLMs) across various workflow phases—specifically during discussion, planning, and demonstration generation—improving perceived performance and interactivity.
The system utilizes a lightweight streaming markdown parser and viewport virtualization to maintain a 10fps render rate, ensuring smooth terminal performance even with high-volume text output. This capability is integrated into the Bubble Tea-based TUI, where it handles progressive token emission and rendering updates.
Sources: README.md, internal/engine/workflow/discuss.go:26-64
Architecture and Data Flow
The streaming architecture follows a producer-consumer pattern where the Workflow Engine acts as the producer, receiving stream chunks from the LLM provider, and the TUI acts as the consumer, rendering these chunks progressively.
Stream Lifecycle
- Request Initiation: A workflow phase (e.g.,
runDiscuss) initiates a streaming LLM call usingstreamLLMStreaming. - Chunk Processing: The engine iterates through chunks provided by an iterator.
- Emission: Each valid
Deltachunk is wrapped in aStreamChunkMsgand sent via a message emitter to the TUI. - TUI Update: The TUI receives the chunk, accumulates it into the current message content, and triggers a re-render using its lightweight parser.
The diagram shows the end-to-end flow of markdown tokens from the LLM provider to the terminal viewport.
Sources: internal/engine/workflow/discuss.go:26-64, README.md
Phase-Specific Implementations
Markdown streaming is leveraged across different stages of the seven-phase workflow to handle structured and unstructured content.
Discuss Phase Streaming
During the Discuss phase, the engine generates clarifying questions. It uses a streamLLMStreaming call and accumulates content while simultaneously emitting chunks to the TUI for immediate display. Once the stream ends, the full content is parsed to extract structured questions for the Q&A interface.
Sources: internal/engine/workflow/discuss.go:26-55
Plan and Demonstration Streaming
Streaming is also critical during large-scale operations:
- Chunked Planning: When goals are complex, the engine uses chunked generation to expand waves of tasks, providing progress updates to the TUI during expansion.
- Ship Phase Demonstrations: Final walkthroughs are generated via
streamLLM. While these can be formatted after generation, the underlying engine supports streaming these long narrative documents to avoid long wait times.
Sources: internal/engine/workflow/plan_chunk.go:53-70, internal/engine/workflow/ship.go:497-512
Component Overview
| Component | Responsibility | Relevant Files |
|---|---|---|
| Stream Iterator | Handles the raw connection to LLM providers (OpenRouter/Zen/Nvidia) and yields chunks. | internal/engine/workflow/discuss.go |
| Message Emitter | Bridges the Workflow Engine and the TUI by sending tea.Msg packets. | internal/engine/workflow/discuss.go |
| Markdown Lite | A lightweight parser designed for terminal performance and 10fps rendering. | README.md |
| Viewport Virtualization | Manages large markdown documents to prevent TUI lag during streaming. | README.md |
Sources: README.md, internal/engine/workflow/discuss.go:56-62
Technical Specifications
The streaming system is governed by several performance and safety constraints to ensure terminal stability:
- Render Rate: Capped at 10fps to prevent flickering and excessive CPU usage.
- Chunk Handling: The engine accumulates the
fullContentwhile emittingDeltachunks to ensure that once streaming is complete, the engine has the full context for parsing (e.g., parsing JSON task lists or numbered questions). - Validation: Release scripts verify that the TUI can handle these streams without panics and that the UI components (like the REPL and Plan screens) are correctly registered to receive these messages.
Sources: README.md, internal/engine/workflow/discuss.go:40-64, scripts/validate-release.sh:220-230
Content Structure (Plan Format)
The streamed markdown must follow specific structures for the parser to extract actionable data. For example, the plan format requires specific H2 headers and JSON blocks.
# Implementation Plan Title
## Summary
Narrative describing the approach...
## Task List
```json
[
{ "id": 1, "action": "Create", "description": "..." }
]
Sources: [internal/engine/workflow/prompts/plan-format.md:10-75](https://github.com/eshanized/M31A/blob/18a823a3f07dc7fb4caa9ff6b7e3a461c95a04f4/internal/engine/workflow/prompts/plan-format.md#L10-L75)
## Sequence of a Streaming Interaction
The following diagram illustrates the sequence of messages between the system components during a Discuss phase streaming session.
The sequence shows how chunks are emitted to the TUI immediately while the engine waits for the full stream to perform final parsing logic.
Sources: internal/engine/workflow/discuss.go:34-80
Markdown Streaming is a vital performance feature that enables M31A's terminal interface to remain responsive while generating complex, multi-section implementation plans and walkthroughs. By combining aggressive render-rate limiting with progressive token emission, the system provides a modern AI experience within the constraints of a POSIX terminal.