System Architecture Overview
Relevant source files
The following files were used as context for generating this wiki page:
Flow is a monorepo-based "personal second brain" and agent platform designed for document intelligence, autonomous task execution, and continuous agent improvement. The system integrates a Next.js frontend with a FastAPI backend, utilizing a LangGraph-powered pipeline for agentic workflows.
The architecture emphasizes observability and self-evolution. It features a continuous improvement loop where agents are evaluated against "Golden Sets" by an LLM judge, and their configurations—stored as versioned "Genomes"—are automatically optimized via A/B testing and prompt rewriting.
Sources: README.md:1-10, AGENTS.md:14-18
Core System Components
The Flow ecosystem is partitioned into high-level service layers orchestrated via Docker Compose.
Monorepo Structure and Deployment
| Path | Role | Technologies |
|---|---|---|
apps/web/ | Frontend User Interface | Next.js 16, Tailwind v4, React 19 |
services/api/ | Backend Logic & Agents | FastAPI, LangGraph, asyncpg |
| Infrastructure | Storage & Processing | PostgreSQL (pgvector), Redis, Qdrant |
Sources: README.md:16-22, AGENTS.md:14-16, apps/web/src/components/marketing/HomeLanding.tsx:16-30
Service Interaction Map
The following diagram illustrates the primary data flow between the user interface, backend services, and persistent storage layers.
Sources: README.md:20-35, AGENTS.md:14-18
Agent Execution Framework
Flow utilizes a specific pipeline architecture for processing user queries, primarily relying on the deer-flow template.
The deer-flow Pipeline
Execution follows a phased approach to ensure high-quality synthesis and accurate citations:
- Planner: Analyzes the user intent and formulates a strategy.
- Worker: Executes tool calls, such as searching long-term memory or performing RAG via pgvector.
- Synthesizer: Compiles the final response, embedding inline citation markers.
Sources: README.md:65-75, AGENTS.md:14-18
Communication via SSE
Run executions stream data to the frontend using Server-Sent Events (SSE). This allows for deterministic streaming of token generation and the delivery of specific frames, such as citations, which the frontend uses to render the Sources panel.
Sources: README.md:65-72, AGENTS.md:21-25, apps/web/src/components/marketing/HomeLanding.tsx:26-30
Agent Genome and Versioning
Agent configurations are managed as immutable snapshots called Genomes. This system allows for safe transitions between configuration states.
| Lifecycle State | Description |
|---|---|
| CANDIDATE | A proposed configuration undergoing A/B testing or awaiting approval. |
| ACTIVE | The current production version used for executions. |
| ARCHIVED | Previous versions retained for history and rollback. |
Sources: README.md:82-95, services/api/scripts/seed_agents_and_datasets.py:461-480
Genome Components
A Genome captures the full behavioral state of an agent:
- Template: The LangGraph structure (e.g.,
react-agent,deer_flow). - System Prompt: Core instruction set.
- LLM Config: Provider, model, and hyperparameters (temperature).
- Tools: Enabled capabilities (Sandbox, Search, Memory).
- Skills: Markdown-based instructions with YAML frontmatter.
Sources: README.md:85-93, services/api/scripts/seed_agents_and_datasets.py:48-60
Knowledge and Retrieval Infrastructure
Flow implements a multi-modal ingestion and retrieval system to support its RAG (Retrieval-Augmented Generation) capabilities.
Ingestion Pipeline
Users can add knowledge via file uploads (.pdf, .docx, .md, .txt), URL crawling, or manual text entry. Ingested content is chunked and embedded for vector search.
Retrieval Logic
The system supports two tiers of retrieval:
- Standard RAG: Uses
pgvectorwith cosine similarity to find the top 8 chunks. - Agentic RAG: An optional LangGraph sub-pipeline using Qdrant for hybrid RRF (Reciprocal Rank Fusion) and BM25 search, including LLM-based query rewriting and Tavily fallback.
Sources: README.md:140-145, README.md:73-75
Continuous Improvement Loop
A core feature of the architecture is the automated self-improvement cycle managed by the ARQ worker.
The Optimization Cycle
- Evaluation: Nightly cron jobs run "Golden Set" items (input/expected pairs) through an LLM judge.
- Prompt Rewriting: If performance falls below a 0.7 pass rate, the
PromptRewriteranalyzes failure rationales to suggest edits. - A/B Testing: Candidates are tested against the current baseline.
- Promotion: If a candidate significantly outperforms the baseline, a human-approvable Proposal is generated to promote it to
ACTIVE.
Sources: README.md:97-118, services/api/scripts/seed_agents_and_datasets.py:192-205
Observability and the Knowledge Graph
Flow instruments every execution via the FlowCallbackHandler, which attaches metadata like workspace_id and execution_id to logs.
Agentic Knowledge Graph
Every execution, tool call, and retrieved document is mapped into a persistent Knowledge Graph. This graph utilizes PageRank and semantic clustering to organize and visualize relationships between entities (agents, skills, executions, and concepts).
Sources: README.md:126-135, apps/web/src/components/marketing/HomeLanding.tsx:16-20, apps/web/src/app/(app)/graph/page.tsx:102-120
Conclusion
Flow's architecture represents a sophisticated integration of modern web technologies and agentic AI patterns. By combining a strictly versioned Genome system with an automated evaluation and improvement loop, the platform provides a deterministic yet evolving environment for agent-based document intelligence. The dual-path retrieval system (pgvector and optional Qdrant) and the holographic knowledge graph visualization ensure that both performance and transparency are maintained as the agent's knowledge base grows.