Chain Indexer & Database
Relevant source files
The following files were used as context for generating this wiki page:
The Chain Indexer and Database system within X3 Atomic Star serves as the primary infrastructure for off-chain data availability, asset auditing, and multi-chain state synchronization. Its purpose is to ingest real-time data from the X3 Chain and external networks (Ethereum, Solana, Bitcoin), providing a high-performance query layer for the desktop application, validator dashboards, and the AI swarm ecosystem.
This system bridges the gap between raw blockchain consensus and actionable intelligence by providing deterministic indexing for cross-VM transfers, validator participation metrics, and off-chain asset relationships. It supports the project's goal of "survivability" by ensuring all chain events are verifiable through deterministic off-chain storage.
System Architecture
The indexing architecture is split into three primary planes: the Blockchain/Runtime Plane, the Swarm Control Plane, and the User/Interface Plane.
The data flow starts at the Substrate node and external adapters, moving through Off-Chain Workers into a unified storage layer.
Sources: X3_END_TO_END_GAPS_MASTER_PLAN.md:126-140, plans/Off-Chain-Assets-Audit-Blockchain-Integration-Epic-Plan.md:118-124
Data Storage & Schema
The system utilizes multiple storage engines depending on the data type and performance requirements. Local storage for high-throughput indexing is handled by RocksDB, while the CRM and AI agent interactions utilize a relational structure (SQLite) within the desktop application.
Off-Chain Storage (RocksDB)
The core off-chain storage layer is designed for bridge data and validator metrics.
| Component | Description |
|---|---|
| Schema Design | Targeted storage for bridge data and transaction proofs. |
| Encryption | Implementation of an encryption layer for off-chain data security. |
| Query API | A high-level API for retrieving indexed off-chain assets. |
| Backup | Procedures for disaster recovery and state restoration. |
Sources: plans/Off-Chain-Assets-Audit-Blockchain-Integration-Epic-Plan.md:127-133
CRM & Agent Database
The desktop application manages a specialized database for the "surgical swarm" of 15 agents, tracking leads, tasks, and RAG (Retrieval-Augmented Generation) document indices.
The CRM schema relates external contact data to internal AI agent tasking and knowledge retrieval.
Sources: apps/x3-desktop/src-tauri/src/crm/agents.rs:556-575, apps/x3-desktop/src-tauri/src/crm/agents.rs:710-725
Off-Chain Indexing (OCI)
The indexing process is governed by Off-Chain Workers (OCW) that specifically target external blockchain state for bridge validation.
Multi-Chain Indexing Workers
The system implements dedicated workers for different chains to ensure cross-VM state agreement.
- Ethereum Indexing: Captures block headers and event logs for Merkle proof validation.
- Solana Indexing: Tracks account state changes and transaction signatures.
- Bitcoin Indexing: Parses OP_RETURN data and Merkle proofs for the BTC bridge reserve.
Sources: plans/Off-Chain-Assets-Audit-Blockchain-Integration-Epic-Plan.md:144-150
Indexing Flow
The sequence ensures that data is only indexed after internal invariants are validated by the worker.
Sources: plans/PHASE_5_IMPLEMENTATION_PLAN.md:120-135
RAG System & Document Indexing
The Indexer includes a Retrieval-Augmented Generation (RAG) system that allows AI agents to query indexed technical documentation.
RAG Indexing Logic
The system scans directory patterns (e.g., **/*.md) and indexes them into the crm_rag_docs table.
- File Discovery: Glob patterns locate relevant documentation files.
- Tokenization: Content is split to calculate token counts for context window management.
- Persistence: Documents are stored with their file path, content, and token counts.
- Query Scoring: A simple keyword-matching algorithm (relevance scoring) ranks documents based on user queries.
Sources: apps/x3-desktop/src-tauri/src/crm/agents.rs:850-890
Integration with X3 Desktop
The indexing system is exposed to the frontend via Tauri IPC commands, allowing the UI to interact with live chain state and the off-chain database.
| Command | Function | Data Source |
|---|---|---|
agents_rag_query | Searches indexed docs for agent context | crm_rag_docs |
query_block | Retrieves block data from X3 Chain | X3 Node RPC |
agents_get_leads | Fetches filtered contact data | crm_lead_funnel |
agents_get_funnel_stats | Aggregates CRM statistics | SQLite (CRM DB) |
Sources: apps/x3-desktop/PHASE_2_INTEGRATION.md:113-130, apps/x3-desktop/src-tauri/src/crm/agents.rs:650-665
Summary
The Chain Indexer & Database provides the "survivable" data backbone of the X3 ecosystem. By combining high-performance RocksDB storage for blockchain state with a flexible SQLite-based CRM for AI agents, the system enables complex multi-chain auditing and intelligent automation. Its integration with Off-Chain Workers ensures that external bridge operations remain verifiable and consistent with the internal X3 Chain state.