Data Normalization Layer
Relevant source files
The following files were used as context for generating this wiki page:
The Data Normalization Layer is a critical architectural component of the MCP Games ecosystem, primarily housed within the mcp-connector service. Its primary purpose is to aggregate, transform, and standardize diverse data retrieved from various Model Context Protocol (MCP) servers—such as GitHub, Linear, Notion, and local filesystems—into a unified format suitable for consumption by the narrative-ai service and the cyoa-engine frontend.
By abstracting the specific schemas of external tools, this layer ensures that the narrative generation engine receives consistent context regardless of the source. This allows for semantic querying where real-world user data (e.g., repository activity, project issues, or documentation) can be injected directly into the storytelling process to create personalized game experiences.
Sources: README.md:41-47, apps/mcp-connector/README.md:5-10
Architecture and Data Flow
The normalization process begins when the mcp-connector receives a request to aggregate context or execute a semantic query. The service interacts with the mcp-sdk to communicate with distributed MCP servers. The raw responses from these servers are passed through the normalization layer to produce standardized QueryResult and context objects.
System Integration Flow
The following diagram illustrates how the Data Normalization Layer sits between external MCP servers and the internal game services:
The Data Normalization Layer processes multi-source data before it reaches the API consumers.
Sources: README.md:15-25, apps/mcp-connector/src/index.ts:36-58
Core Data Structures
Standardization is enforced through shared TypeScript interfaces defined in the shared-types package. These structures define how connections are managed and how query results are represented after normalization.
MCP Connection and Config
The normalization layer tracks the state and capabilities of various servers using the MCPConnection interface.
| Field | Type | Description |
|---|---|---|
id | string | Unique identifier for the MCP server (e.g., 'github'). |
type | enum | The source type: github, linear, notion, filesystem, or custom. |
status | enum | Connection state: connected, disconnected, or error. |
config | MCPConfig | Metadata including serverUrl and supported capabilities. |
Sources: packages/shared-types/src/index.ts:6-21
Normalized Query Results
When a semantic query is executed, the raw output is mapped to a QueryResult<T> structure, ensuring metadata consistency.
export interface QueryResult<T = unknown> {
data: T;
metadata: {
source: string;
timestamp: string;
cached: boolean;
};
}
Sources: packages/shared-types/src/index.ts:44-51
Semantic Querying and Aggregation
The normalization layer supports a "Semantic Query Builder" pattern, allowing services to request data based on intent rather than specific API parameters. The mcp-connector then aggregates this data across multiple providers.
Execution Sequence
The sequence shows how semantic intents are transformed into normalized data for the AI engine.
Sources: packages/mcp-sdk/src/index.ts:63-100, apps/mcp-connector/src/index.ts:36-50
Key API Endpoints
The following endpoints in the mcp-connector manage the flow of normalized data:
GET /api/mcp/connections: Returns a list of available and connected MCP sources.POST /api/mcp/query: Executes a semantic query (e.g., "recent contributions") and returns normalized data.GET /api/mcp/context/:userId: Aggregates and normalizes context from all connected sources for a specific user to be used in narrative generation.
Sources: apps/mcp-connector/src/index.ts:21-58, apps/mcp-connector/README.md:18-28
Integration with Narrative AI
The normalized output is specifically designed to feed into the NarrativeGenerationRequest. This structure allows the AI to receive a mcpData record—a dictionary of normalized strings or objects—without needing to understand the underlying MCP protocol.
| Feature | implementation |
|---|---|
| Context Injection | Injecting normalized MCP data into LLM prompts. |
| Data Normalization | Converting GitHub/Linear artifacts into narrative context. |
| Caching | Storing normalized results using MCP_CACHE_TTL (default 3600s). |
Sources: packages/shared-types/src/index.ts:133-140, apps/mcp-connector/README.md:34, apps/narrative-ai/README.md:5-12
Conclusion
The Data Normalization Layer serves as the "translation office" of the MCP Games monorepo. By utilizing a unified set of interfaces and a centralized connector service, it allows the game engine to treat diverse data sources—like a user's code commits or task list—as standardized narrative fuel. This architecture supports the project's vision of "unrestricted OmniAgents" that can adapt to real-world user contexts seamlessly.
Sources: README.md:7-13, packages/shared-types/README.md:1-15