Engineering Metrics Automation
Relevant source files
The following files were used as context for generating this wiki page:
Engineering Metrics Automation within the eng-platform is a multi-layered system designed to track, analyze, and report on development velocity, team health, and tool effectiveness. The system integrates automated Git history analysis, DORA metrics collection, and proactive team analytics to provide data-driven insights into the engineering lifecycle.
By centralizing metrics collection through GitHub Actions and specialized Model Context Protocol (MCP) servers, the platform ensures that engineering standards—such as conventional commits and deployment frequency—are not only documented but quantitatively measured across projects.
Sources: README.md, CHANGELOG.md:46-52
Metric Collection Framework
The framework utilizes automated scripts and workflows to extract data from various sources, including Git logs, GitHub PRs, and team memory files.
DORA and Velocity Tracking
The platform automates the collection of the four key DORA (DevOps Research and Assessment) metrics to evaluate engineering performance:
| Metric | Definition | Data Source |
|---|---|---|
| Deployment Frequency | Number of successful deployments per week | Git history / PR logs |
| Lead Time for Changes | Median hours from commit to production | Git history |
| Change Failure Rate | Percentage of deployments causing a failure | Git history / Issue logs |
| Time to Restore (MTTR) | Median hours to resolve service failures | Git history / Issue logs |
Sources: workflows/engineering-metrics.yml:71-80, CHANGELOG.md:123-128
The logic for calculating these values resides in scripts/metrics/engineering-velocity.js, which generates both JSON and Markdown reports for historical tracking.
Sources: workflows/engineering-metrics.yml:50-54
Team Analytics MCP
The team-analytics MCP server provides deeper insights into individual and team health. It calculates an Automated Health Score (0-100) based on weighted dimensions:
- Code Quality (35%): Measured by bug fix ratios and commit size.
- Knowledge Sharing (25%): Contributions to
team-memory.json. - Velocity (25%): Commits and lines changed per week.
- Collaboration (15%): PR review patterns and mentoring.
Sources: mcp-servers/team-analytics/README.md:9-15, mcp-servers/team-analytics/README.md:120-135
Automation Workflows
The system uses GitHub Actions to orchestrate metrics collection and stakeholder reporting on a set schedule.
The following diagram illustrates the automated data flow of the engineering metrics pipeline:
The metrics workflow automatically gathers data from Git and the GitHub API, processes it into reports, and distributes findings to Slack and GitHub Issues.
Sources: workflows/engineering-metrics.yml:6-12, workflows/engineering-metrics.yml:104-140
Key Automation Components
| Component | Responsibility | Relevant Files |
|---|---|---|
| Engineering Metrics Workflow | Orchestrates daily collection and weekly digests via Slack/GitHub. | workflows/engineering-metrics.yml |
| Team Sync Check | Validates environment, dependencies, and MCP server health before execution. | scripts/team-sync-check.js |
| Velocity Script | Analyzes git history to calculate lead times and deployment frequency. | scripts/metrics/engineering-velocity.js |
| Registry Auditor | Measures tool adoption and effectiveness via tools-registry.json. | tools-registry.json |
Sources: README.md, CHANGELOG.md:139-144
Tool Experimentation & Registry
The platform maintains a tools-registry.json to track development tools and their impact on engineering metrics. This registry supports a data-driven approach to standardizing tools through A/B testing.
Tracked Metrics per Category
The registry defines specific metrics to measure the success of different tool categories:
- Linting: Code quality score, PR approval rate, violation count.
- Testing: Coverage percentage, runtime, flake rate.
- Git Workflow: Deployment frequency, lead time, cycle time.
- AI Assistants: Velocity, suggestion acceptance rate, bug rate.
Sources: tools-registry.json:28-34, tools-registry.json:237-295
Promotion Criteria
Experimental tools are promoted to "standard" status only when they meet specific quantitative thresholds:
- Min Duration: 8 weeks.
- Improvement: >10% in tracked metrics.
- Statistical Significance: p-value < 0.05.
- Satisfaction: >4.0 developer satisfaction score.
Sources: tools-registry.json:298-307
Reporting and Insights
The automated reporting system generates three primary outputs to ensure visibility for different stakeholders:
1. Weekly Digest Issues
A GitHub issue is created every Monday containing the full Markdown report of the previous week's velocity and DORA ratings (e.g., "Elite", "High", "Medium", "Low").
Sources: workflows/engineering-metrics.yml:95-103
2. Slack Alerts
Critical health alerts and trend summaries are pushed to Slack. The team-analytics server specifically flags "Critical" alerts if health scores drop below 50 or bug rates exceed 25%.
Sources: workflows/engineering-metrics.yml:143-176, mcp-servers/team-analytics/README.md:141-145
3. Step Summaries
Every CI run generates a GitHub Step Summary providing immediate feedback on DORA metrics for that specific analysis window.
Sources: workflows/engineering-metrics.yml:180-192
Summary
Engineering Metrics Automation provides a robust infrastructure for measuring the health and efficiency of the development team. By combining low-level Git analysis with high-level team health scoring and a structured tool experimentation framework, the platform enables continuous improvement based on empirical data rather than subjective assessment. These automated systems ensure that engineering velocity remains visible and manageable as projects scale.
Sources: README.md, mcp-servers/team-analytics/README.md:104-108