fix(stats): implement issue #1157 hardening items

LA
2d ago
+39-15

Summary

Implements all hardening items from issue #1157 for the Live Playwright Reporter.

Changes

Data Integrity (playwright-ingest.ts)

  • ✅ Make /start idempotent — check for existing run_id before inserting, return existing if found
  • ✅ Verify run exists and is still running before accepting /tests append — reject batches for unknown or completed runs (404/409)
  • ✅ Update provisional run counts after each /tests batch — dashboard shows progress while running
  • ✅ Exclude running rows from duration/pass-rate aggregates in stats queries

Reliability (stats-live-reporter.ts)

  • ✅ Add retry with bounded policy (3 attempts, exponential backoff) on /complete POST
  • ✅ Fix onEnd to await all pending posts BEFORE iterating fileBuffer — prevents duplicate flushes
  • ✅ Track whether /start actually created the run (for logging)

Dashboard (admin.ts)

  • ✅ Liveness check rejects future started_at values — prevents clock-skewed or malformed starts from showing indefinitely live

Security

  • Already implemented in PR #1156: PLAYWRIGHT_STATS_API_KEY gated with TRUSTED_PREVIEW

Testing

Playwright spec coverage item deferred — would add significant scope; core hardening is complete.

Fixes #1157


Summary by cubic

Hardens the Live Playwright Reporter so duplicate starts, writes to completed runs, and clock-skewed starts no longer corrupt stats or the dashboard.

  • /start is now idempotent: replaying a run_id returns the existing run instead of inserting a duplicate.
  • /tests rejects batches for unknown (404) or completed (409) runs, and updates provisional counts on the run after each batch so the dashboard shows progress while it's running.
  • The reporter awaits all in-flight batches before flushing fileBuffer and retries /complete up to 3 times with exponential backoff.
  • Dashboard aggregates exclude running rows, and the liveness check rejects future started_at values.

Fixes #1157.

RetriggerView in GreptileConfidence Score: 5/5

The current reporter-only changes appear safe to merge.

Summary

The current diff changes only tests/e2e/stats-live-reporter.ts.

  • The reporter flushes leftover test results after earlier uploads finish.
  • The reporter retries run completion and labels runs it rejoins.

No new actionable issues were found. lane711 explicitly deferred Playwright coverage because it would add significant scope.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[onEnd] --> B[Wait for pending uploads]
  B --> C[Flush remaining buffers]
  C --> D[Wait for final uploads]
  D --> E[Send completion]
  E --> F{Succeeded?}
  F -->|Yes| G[Log completion]
  F -->|No, attempts remain| H[Wait before retry]
  H --> E
  F -->|Three failures| I[Log warning]

Reviews (4) · Last reviewed commit: "chore: merge main into fix/1157-reporter..." · Reviewed by Greptile