fix(provenance): derive final-output provenance instead of declaring it unvouchable

Merged
IC
2mo ago
+96-15

Summary

  • Fixes the live producer behind the TraceStore — displayProjection stream that is now ~95% of provenance log output (105 of 110 lines post-deploy). Every one of those lines is parts: ["finalOutput"], and the stock is real and growing — 30 distinct executions in under 4 hours — so this was never just a fixed backlog of bug-era rows.
  • Root cause: ExecutionEngine.setFinalOutput stamped { complete: false } whenever the final block's state carried no provenance. But an absent block-state envelope is not a verdict — several state writers legitimately omit one, and a subflow sentinel aggregating iteration results is the common case (a loop that ran no iterations has nothing to merge, so setBlockOutput is called without provenance). The affected production workflows end in a loop block, which is exactly this shape.
  • The cost was user-facing, not just log noise. An incomplete final-output envelope makes the display projection withhold finalOutput, so the workflow's author could not see their own run's output in the execution log — on every view, permanently. Each view then re-derived the same verdict, which is the read-path re-fire dominating the stream.
  • The fix: derive from the run registry against the value being described — precisely what the end-of-run path (ensureFinalOutputProvenance) already did, and what every other consumer of a provenance-less block state does (the trace-store block-output reader falls back to the run registry). The two paths now share one deriveFinalOutputProvenance rather than disagreeing: take the block state's exact envelope as a shortcut when present, derive otherwise.
  • Still fails closed: exportCommittedProvenanceForValue returns an incomplete envelope when the registry itself latched, which is the genuinely unvouchable case. That guard has its own regression test.

Type of Change

  • Bug fix

Testing

Two new engine tests: one pins the derivation (a final block whose state carries no provenance yields a complete envelope naming the secret actually present in the output), one pins fail-closed (a latched registry still exports incomplete). Verified the derivation test fails without the fix and passes with it. 5,071 tests green across executor/, lib/logs/, and lib/workflows/; bun run type-check clean; all 36 audits pass.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)