Database Schema & Seeding
Relevant source files
The following files were used as context for generating this wiki page:The BMSview application relies on MongoDB as its primary database for storing all persistent data, including analysis results, system configurations, and AI-generated feedback. A well-defined schema, supported by strategic indexing, is crucial for application performance, especially for data retrieval and synchronization. Seeding scripts are provided to populate the database with realistic sample data for development and testing purposes.
This document outlines the database collections, their schemas, the indexing strategy, and the available data seeding utilities. For more details on the overall system design, refer to the Architecture Guide.
Database Collections
The application utilizes several collections within the bmsview database (or the database specified by the MONGODB_DB_NAME environment variable) to organize its data.
The following diagram illustrates the main collections and their relationships.
Collection Summary
| Collection | Purpose | Source Files |
|---|---|---|
analysis-results | Stores the results of AI-powered BMS screenshot analysis. | [ARCHITECTURE.md:188](https://github.com/Treystu/BMSview/blob/HEAD/ARCHITECTURE.md#L188) |
systems | Manages registered BMS systems, including their configuration and associated data loggers. | [ARCHITECTURE.md:215](https://github.com/Treystu/BMSview/blob/HEAD/ARCHITECTURE.md#L215) |
history | Legacy collection for analysis history. Queries are often directed here for historical data. | [CODEBASE.md:18](https://github.com/Treystu/BMSview/blob/HEAD/CODEBASE.md#L18), [tests/analyze-endpoint-monthly-query.test.js:6](https://github.com/Treystu/BMSview/blob/HEAD/tests/analyze-endpoint-monthly-query.test.js#L6) |
ai_feedback | Stores structured feedback generated by the AI for application improvement. | [SPRINT_COMPLETE_SUMMARY.md:124-126](https://github.com/Treystu/BMSview/blob/HEAD/SPRINT_COMPLETE_SUMMARY.md#L124-L126) |
ai_operations | Logs detailed records of every AI operation for monitoring and cost tracking. | [MONITORING_IMPLEMENTATION_SUMMARY.md:20-23](https://github.com/Treystu/BMSview/blob/HEAD/MONITORING_IMPLEMENTATION_SUMMARY.md#L20-L23) |
ai_metrics | Stores application-level metrics like accuracy, performance, and cost. | [MONITORING_IMPLEMENTATION_SUMMARY.md:52-55](https://github.com/Treystu/BMSview/blob/HEAD/MONITORING_IMPLEMENTATION_SUMMARY.md#L52-L55) |
anomaly_alerts | Contains system-generated alerts for anomalies like cost spikes or high error rates. | [MONITORING_IMPLEMENTATION_SUMMARY.md:71-74](https://github.com/Treystu/BMSview/blob/HEAD/MONITORING_IMPLEMENTATION_SUMMARY.md#L71-L74) |
feedback_tracking | Tracks the implementation lifecycle of AI-generated suggestions. | [MONITORING_IMPLEMENTATION_SUMMARY.md:90-93](https://github.com/Treystu/BMSview/blob/HEAD/MONITORING_IMPLEMENTATION_SUMMARY.md#L90-L93) |
jobs | Used for managing background job processing tasks. | [scripts/create-indexes.js:10](https://github.com/Treystu/BMSview/blob/HEAD/scripts/create-indexes.js#L10) |
Sources: ARCHITECTURE.md, CODEBASE.md, SPRINT_COMPLETE_SUMMARY.md, MONITORING_IMPLEMENTATION_SUMMARY.md, scripts/create-indexes.js
Schemas
analysis-results Collection
This collection holds the primary output of the BMS analysis.
| Field | Type | Description |
|---|---|---|
_id | ObjectId | Unique identifier for the record. |
analysis | Object | Contains the structured data extracted from the BMS screenshot (e.g., overallVoltage, current, cellVoltages). |
weather | Object | Associated weather data at the time of the analysis (e.g., temp, clouds). |
systemId | String | Optional. The ID of the systems record this analysis is linked to. |
dlNumber | String | Optional. The data logger number associated with the analysis. |
fileName | String | The name of the uploaded screenshot file. |
timestamp | ISODate | The creation timestamp of the record. |
updatedAt | ISODate | The timestamp of the last server update, in UTC. Used for synchronization. |
contentHash | String | SHA-256 hash of the image content, used for deduplication. |
_isDuplicate | Boolean | Flag indicating if the uploaded image was a duplicate. |
needsReview | Boolean | Flag indicating if the extracted data failed validation and may need manual review. |
validationWarnings | Array | A list of warnings generated during data integrity validation. |
validationScore | Number | A quality score (0-100) based on the data validation results. |
Sources: ARCHITECTURE.md:188-213, AUTONOMOUS_SELF_CORRECTION.md:162-175
systems Collection
This collection stores user-registered BMS systems.
| Field | Type | Description |
|---|---|---|
_id | ObjectId | Unique identifier for the system. |
name | String | User-defined name for the system. |
chemistry | String | The battery chemistry (e.g., "LiFePO4", "Li-ion"). |
voltage | Number | The nominal voltage of the system. |
capacity | Number | The total capacity of the system in Amp-hours. |
associatedDLs | Array | An array of data logger numbers (dlNumber) linked to this system. |
location | String | The physical location of the system. |
createdAt | ISODate | The creation timestamp of the record. |
updatedAt | ISODate | The timestamp of the last server update, in UTC. |
Sources: ARCHITECTURE.md:215-229
ai_feedback Collection
This collection stores suggestions from the AI to improve the application.
| Field | Type | Description |
|---|---|---|
id | String | A unique ID for the feedback item (e.g., fb_*). |
timestamp | Date | The creation timestamp. |
systemId | String | The BMS system the feedback relates to. |
feedbackType | String | The type of feedback (e.g., feature_request, bug_report). |
category | String | The area of the application the feedback applies to (e.g., ui_ux, performance). |
priority | String | The priority level (critical, high, medium, low). |
status | String | The workflow status (pending, reviewed, accepted, etc.). |
suggestion | Object | A detailed object containing the title, description, rationale, and implementation details of the suggestion. |
contextHash | String | A SHA-256 hash for deduplication of feedback. |
Sources: SPRINT_COMPLETE_SUMMARY.md:126-159
Database Indexing
To ensure efficient data retrieval, several indexes are defined across the collections. These indexes are critical for performance, especially for features like local-first sync, historical data queries, and duplicate detection.
The scripts/create-indexes.js script is used to programmatically create all necessary indexes in the database.
Index Creation Flow
The script follows a clear process for each collection to ensure indexes are created correctly.
This diagram shows the general workflow of the create-indexes.js script.
Sources: scripts/create-indexes.js
Defined Indexes
| Collection | Index Fields | Type | Purpose |
| --- | --- | --- |
| jobs | { "id": 1 } | Unique | Fast lookup of jobs by their unique ID. |
| jobs | { "status": 1 } | Standard | Efficiently query jobs by their status. |
| jobs | { "createdAt": 1 } | Standard | Sort and query jobs by creation time. |
| history | { "timestamp": 1 } | Standard | Optimize range queries on timestamps. |
| history | { "updatedAt": 1 } | Standard | Support efficient incremental synchronization. |
| history | { "systemId": 1 } | Standard | Quickly find all records for a specific system. |
| history | { "dlNumber": 1 } | Standard | Search for records by data logger number. |
| history | { "contentHash": 1 } | Standard | Used for deduplication lookups. |
| systems | { "name": 1 } | Standard | Query systems by name. |
| systems | { "updatedAt": 1 } | Standard | Support efficient incremental synchronization. |
| ai_feedback | { "priority": 1, "status": 1 } | Compound | Filter feedback by priority and status. |
| ai_feedback | { "systemId": 1, "timestamp": -1 } | Compound | Find recent feedback for a specific system. |
| ai_feedback | { "contextHash": 1 } | Standard | Deduplicate AI-generated feedback. |
Sources: scripts/create-indexes.js:11-81, SPRINT_COMPLETE_SUMMARY.md:162-164
The following code snippet from the index creation script shows how indexes for the history collection are defined.
// scripts/create-indexes.js:27-41
const historyIndexes = [
{ key: { timestamp: 1 }, name: "history_timestamp_idx" },
{ key: { updatedAt: 1 }, name: "history_updatedAt_idx" },
{ key: { systemId: 1 }, name: "history_systemId_idx" },
{ key: { dlNumber: 1 }, name: "history_dlNumber_idx" },
{ key: { contentHash: 1 }, name: "history_contentHash_idx" },
{
key: { "analysis.stateOfCharge": 1 },
name: "history_soc_idx",
sparse: true,
},
// ...
];
Sources: scripts/create-indexes.js:27-41
Data Seeding
For development and testing, the repository includes a script to seed the database with realistic sample data for the monitoring system.
seed-monitoring-data.js
This script populates the monitoring-related collections (ai_operations, ai_metrics, anomaly_alerts, feedback_tracking) with sample documents. This is essential for testing the monitoring dashboard and alert systems without needing to generate real data.
Seeding Process
This diagram outlines the steps performed by the seed-monitoring-data.js script.
Sources: scripts/seed-monitoring-data.js:14-30
The script generates data with realistic variations:
ai_operations: Creates 100 sample operations with a 95% success rate, variable durations, and costs calculated based on actual Gemini pricing.anomaly_alerts: Creates 10 alerts with mixed severity levels.feedback_tracking: Creates 20 feedback items with various statuses.custom_metrics: Creates 50 custom metric entries.
To run the seeder, use the following npm command:
npm run seed:monitoring
Sources: MONITORING_IMPLEMENTATION_SUMMARY.md:330-341, scripts/seed-monitoring-data.js:33-138
Conclusion
The BMSview database is structured to support a feature-rich application, with a schema designed for scalability and performance. The indexing strategy is crucial for fast data retrieval and efficient synchronization, directly impacting user experience. The provided scripts for index creation and data seeding are vital tools for developers to set up, maintain, and test the application in a consistent and realistic environment.