chore: proposal - Reflow target for deployed hosts and a manual preview job

Draft
TH
15h ago
+448-28

Proposal for discussion. Draft, not for merge as it stands. Stacked on #9441 and useless without it. #9441 now admits a pull request preview that opted in, as well as a local stack, so the preview half of this pull request has something to call, but only once the infrastructure change that gives a preview its token is applied. Neither workflow is triggered by any event: one is callable only, the other is manual dispatch, and the one secret they need does not exist. Neither has been run.

Summary

  • mise run reflow:target without --url behaves as before and additionally writes an OpenAPI binding holding the control token, so the two shared flows below run against this worktree's stack. The control surface is opt-in: when GRAM_E2E_CONTROL_TOKEN is not set the task stops and prints how to generate a private token and restart the server.
  • mise run reflow:target --url https://<host> --expect-build <sha> aims the Reflow target at a deployed host instead: a pull request preview that carries the preview-e2e label. A deployed target gets a Browser binding and an OpenAPI binding and no database binding: nothing in CI holds a DSN for a deployed environment.
  • A target holds one credential, the control token. The API key the first revision also wrote is gone: the suite's root no longer reads the public API.
  • mise run reflow:golden --label <tag> narrows every selection (the single run, the shard list and the repeat) to golden flows carrying a tag.
  • .github/workflows/reflow-deployed.yml (workflow_call only) and .github/workflows/reflow-preview.yml (manual dispatch, a pull request number): the deployed job and its preview caller. The first refuses hosts outside the known non-production set, waits for the host to answer, and records its runs on a branch name of their own with no base branch, so a pass from a local-stack run is never carried into a deployed run or the reverse. The second refuses forks and pull requests without both the preview and preview-e2e labels.

The existing reflow-golden.yml job is unchanged.

Motivation

The golden flows run against a stack the CI job boots from the branch, and two shared flows tie all of them to it: the root checks the seed with direct SQL, and the shared sign-in clicks through the dev-idp. With the control surface in #9441 both go through the product instead, and the rest of the suite runs on top of them unchanged apart from one frontmatter line (below). Running them against the image a pull request deploys is the reason for the deployed half.

The two shared flows are proposed as replacements for gram-seed-state and sign-in. They are not saved in the collection and replace nothing yet. Both were run against a local stack through this task.

The sign-in step, in full (the digest pins server/internal/e2econtrol/openapi.yaml as it stands on #9441):

### Sign in
```rfl timeout=120s
api.call_value (
  operationId="e2eSignInLink"
  spec="sha256:886174f8414001a9ffe6bd8bd3e5737f18861921b57dd585f10f5fa054bba89f"
  expect=json(
    {"status":200}
  )
  select="/path"
) -> sign_in_path
open ${sign_in_path}
expect testid=user-menu-trigger visible timeout=50000ms
```

The root, step by step, each a typed call with its expectation in the flow:

  1. e2eTenant reports a build containing the expected commit (waits up to 15 minutes).
  2. e2eReseed returns 204: the seed, its own postflight checks and the member fixtures all completed.
  3. e2eTenant gives the organization slug, project slug, seeded id prefix and build, which become the root's outputs. No slug or id is an input.
  4. It publishes project_path, server_host and mcp_server_slug as the current root does.

The seeded state itself is asserted in the dashboard by the flows that depend on the root.

Known costs and gaps:

  • A flow that uses the new sign-in must declare the api: OpenAPI provider, so each Browser golden flow that signs in gains one frontmatter line.
  • Read against a preview's synthetic tenant, 36 of the 46 Browser golden flows that sign in would run with their own steps unchanged. Of the rest: three hard-code local seed identifiers (a seeded user id, a gateway id, a deployment id) and need to read them from the root; three read SQL in their own steps; one asserts telemetry, which a preview does not seed; three need a look on a real preview (a feature flag, a staff-only panel used as a snapshot mask, the default environment). Snapshot baselines recorded on a local stack will differ on a preview wherever a page shows the organization's name, host or members.
  • With no base branch a deployed run executes every portable flow, in shards, since it cannot borrow passes from local-stack runs.
  • The job runs the pull request's own Reflow tasks while holding the control token, which is why forks are refused. The token's reach is synthetic tenants on opted-in previews.
  • Unverified: reflow ci run <slugs> --golden --label portable (the shard path passes slugs and a label together), and the tool list install_args: node aube jq in the callable workflow.