Maintenance Scripts
Relevant source files
The following files were used as context for generating this wiki page:
Maintenance scripts in ZINC-Fusion-V15 serve as the primary verification and hygiene layer for the commodity forecasting system. These scripts ensure project integrity, validate environment configurations, and maintain repository health across Python/ML and Next.js frontend boundaries. They are designed to be executed by both human developers and AI agents as mandatory "completion gates" before merging code or marking tasks as finished.
The scope of these scripts includes database connectivity testing, schema validation, linting, test execution, and repository hygiene (worktree and git integrity management).
Sources: CLAUDE.md:31-33, scripts/verify.sh:6-10, scripts/preflight_check.py:9-11
Project Integrity and Verification Gates
The project utilizes a multi-stage verification process to prevent regressions and environment-specific failures.
Preflight Checks
scripts/preflight_check.py is a comprehensive Python-based diagnostic tool used to verify the local development environment. It inspects project structure, environment variables, virtual environment status, and critical Python imports.
The script performs the following primary checks:
- Structure Validation: Ensures critical files like
AGENTS.md,CLAUDE.md, andprisma/schema.prismaexist, along with core directories. - Environment Variables: Verifies
DATABASE_URLand optional keys likeFRED_API_KEYare present in the.envfile. - Dependency Resolution: Attempts to import core ML and data libraries (
pandas,numpy,psycopg2,sklearn) to confirm the.venvis correctly populated. - Schema & DB Connectivity: Validates that the Prisma schema only contains approved schemas and tests live connectivity to the Postgres database.
Sources: scripts/preflight_check.py:46-68, scripts/preflight_check.py:83-93, scripts/preflight_check.py:118-132, scripts/preflight_check.py:145-195
Verification Gate
scripts/verify.sh acts as the "Single Source of Truth" for quality checks. It is a shell script that aggregates linting, testing, and secret detection. It is categorized into Python, Frontend, and Prisma gates.
| Gate Category | Specific Checks | Purpose |
|---|---|---|
| Python | Ruff lint, pytest, venv check | Ensures code style, imports, and logic are valid. |
| Frontend | ESLint, TypeScript compilation | Validates Next.js/TypeScript code in the frontend/ directory. |
| Prisma | Schema validation, Migration status | Ensures the database schema is in sync and valid. |
| Security | Gitleaks | Detects hardcoded secrets in the codebase. |
Sources: scripts/verify.sh:10-12, scripts/verify.sh:44-71, scripts/verify.sh:77-87
Verification Logic Flow
The following diagram illustrates the execution flow of the verification gate:
The diagram shows how verify.sh sequences through different language-specific tools to provide a final safety report.
Sources: scripts/verify.sh:16-118
Repository Hygiene and Git Integrity
Maintaining a clean repository is critical for AI-assisted development, particularly when managing multiple worktrees and branches.
Worktree and Branch Cleanup
scripts/cleanup_worktrees.sh provides interactive and automated methods to remove stale agent worktrees, merged branches, and temporary stashes. It specifically targets branches created by agents (e.g., claude/, copilot/, agent-).
Key cleanup logic:
- Stale Worktrees: Removes worktrees where the last commit is older than 7 days (in
--automode). - Merged Branches: Deletes local branches already merged into
main. - Stashes: Offers to clear the git stash list to prevent accumulation of temporary changes.
Sources: scripts/cleanup_worktrees.sh:3-7, scripts/cleanup_worktrees.sh:58-86, scripts/cleanup_worktrees.sh:102-120
Git Integrity Validation
scripts/check_git_integrity.sh identifies "ghost" files—files that exist on the local disk but are not tracked by Git. This prevents the "works locally, fails on CI" scenario.
- Exclude Poisoning: Checks
.git/info/excludefor active rules that might be hiding files from the remote repository. - Barrel Export Integrity: Scans
frontend/src/inngest/functions.tsand verifies that every re-exported module is actually tracked by Git. - Ghost TS Detection: Searches for untracked
.tsor.tsxfiles in the frontend source directory.
Sources: scripts/check_git_integrity.sh:6-12, scripts/check_git_integrity.sh:24-38, scripts/check_git_integrity.sh:48-63
Data and Model Auditing
For data-intensive maintenance, the system includes scripts to audit the state of the warehouse and model registry.
Databento Audit
scripts/_deprecated/run_audit_queries.py (and associated SQL in grafana/model_registry_queries.sql) performs deep forensic audits on the analytics.zl_price tables. It checks for:
- Source Distribution: Identifying rows from different providers (e.g., Databento vs. Yahoo Finance).
- Price Discontinuities: Detecting percentage jumps that might indicate bad data ingestion.
- Coverage Gaps: Finding missing 15-minute or daily bars in the time-series data.
- Freshness: Calculating the age of the latest record per source.
Sources: scripts/_deprecated/run_audit_queries.py:53-73, scripts/_deprecated/run_audit_queries.py:108-132, scripts/_deprecated/run_audit_queries.py:175-199
Model Performance Tracking
Grafana-compatible SQL queries allow for ongoing maintenance of the model registry. These scripts monitor:
- Champion Models: Identifying which algorithms (e.g., AutoETS, DeepAR) are currently active for specific horizons.
- MASE/WQL Trends: Tracking model accuracy metrics over time to detect performance degradation.
Sources: grafana/model_registry_queries.sql:13-33, grafana/model_registry_queries.sql:71-85
Conclusion
The maintenance scripts in ZINC-Fusion-V15 form a robust defensive layer. By strictly enforcing environment checks, repository hygiene, and data audits, they allow the system to maintain institutional-grade forecasting accuracy while supporting rapid, agent-assisted development iterations. Execution of verify.sh and preflight_check.py remains a non-negotiable requirement for all contributors.
Sources: AGENTS.md:126-130, scripts/verify.sh:111-125