AI Agent Workflow & Memory
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The AI Agent Workflow & Memory system provides a structured protocol for autonomous agents (such as Claude) to interact with, contribute to, and maintain the mlx-qwen3-asr repository. It defines a "source-of-truth" hierarchy for instructions and a persistent memory architecture designed to capture architectural decisions, failures, and implementation details across sessions.
This system ensures that agent contributions remain consistent with the project's "North Star" goals, such as maintaining a zero-Transformers dependency and optimizing specifically for Apple Silicon Metal performance. Sources: CLAUDE.md:1-25, AGENTS.md:1-12
Agent Operating Guidelines
The project utilizes a tiered instruction system where specific markdown files act as entry points for different agent runtimes. CLAUDE.md serves as the primary contributor guide and the authoritative source for architecture and workflow expectations.
Workflow Hierarchy
- Entry Point: Agents looking for compatibility instructions start at
AGENTS.md. - Primary Guide:
CLAUDE.mdprovides the "North Star" vision and technical constraints. - Memory Access: All non-trivial work must be recorded through the designated memory front door at
docs/memory/operating-memory.md.
Sources: AGENTS.md:3-12, CLAUDE.md:27-30
Memory Architecture & Event Logging
The project employs a structured memory workflow to maintain "institutional memory" of technical decisions. This prevents future agents from repeating failed paths and ensures that architectural rationale is preserved even as the codebase evolves.
Memory Protocol
- Operating Memory: A single "front door" for active guidance (
docs/memory/operating-memory.md). - Event Logs: A chronological record of non-trivial changes stored in
docs/memory/events/YYYY-MM.md. - Distilled Learnings: Patterns that repeat across multiple sessions are promoted to high-level guidance.
Memory Workflow Diagram
The following diagram illustrates the lifecycle of a technical decision within the agent memory system.
Sources: AGENTS.md:14-18, CLAUDE.md:175-188
Technical Constraints and Invariants
Agents must adhere to strict technical "invariants" to ensure the correctness of the MLX reimplementation. These rules prevent silent inference failures that could occur when translating from PyTorch implementations.
| Constraint | Technical Requirement | Rationale |
|---|---|---|
| MRoPE | Interleaved frequency assignment (stride-3) | Essential for temporal/spatial alignment |
| Normalization | Encoder: LayerNorm + Bias; Decoder: RMSNorm | Mixing these causes silent inference breakage |
| Dependencies | No torch or transformers in core path | Native MLX performance and package weight |
| Weights | Transpose Conv2d weights from PyTorch format | MLX uses (out, kH, kW, in) |
Sources: CLAUDE.md:94-110, CLAUDE.md:40-50
Implementation Memory Structure
When logging events, agents are required to include specific metadata to ensure the information is actionable for subsequent sessions.
Minimum Required Fields for Event Logs
- Decision: The final path chosen.
- Reuse next time: How to apply this knowledge to future similar tasks.
- Evidence: The code or benchmark result that validates the decision.
- Root Cause: Documentation of failed paths or "dead ends."
Sources: CLAUDE.md:189-198, scripts/eval_streaming_manifest.py:120-135
Metadata-Rich Git Workflows
The agent workflow mandates a specific git commit format designed to encapsulate the "DNA of the change." This ensures that the git history itself acts as a secondary layer of persistent memory.
Sequence of a Memory-Enriched Commit
Sources: CLAUDE.md:140-160, AGENTS.md:14-18
Seed Prompts
A unique requirement of the agent workflow is the inclusion of a "Seed Prompt" in every significant commit. This is a dense, high-context instruction that provides enough architectural awareness to allow a future agent to reproduce the change from first principles without referring to the original diff. Sources: CLAUDE.md:154-165
Conclusion
The AI Agent Workflow & Memory system transforms the repository into a learning environment. By strictly separating operating memory from chronological event logs and enforcing architectural invariants via CLAUDE.md, the project ensures that its MLX-native implementation of Qwen3-ASR remains performant, standalone, and easy to maintain by distributed AI agents.