ci: split the TypeScript tests across parallel jobs

Merged
SA
9h ago
+268-68

Issue for this PR

No issue. PR runs had grown to 40+ minutes.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

The TypeScript job's "Run tests" step was the whole critical path of a PR run: every other job finishes in under 4 minutes, and runners start within seconds. That step ran all ~970 test files in one bun test process, one file after another. It grew from about 6 minutes in June to 36 minutes as the suite went from 392 to 970 files and gained suites that start a fresh CLI process per test.

This PR runs those tests as parallel jobs:

JobRuns
TypeScript (1/4) … (4/4)the main suite, bun test --shard=N/4, still one process per job, so the test/preload.ts leak guard holds
TypeScript (CLI processes, solo)learn/cli.test.ts, the slowest suite (about 6 min)
TypeScript (CLI processes, rest)the bounded subprocess pass (unchanged) and four more CLI suites: learn/bootstrap-cli, tool-cli/cli, rules/host-cli, bridge-merge-e2e
SDK codegenthe codegen round-trip check, moved out unchanged
  • One list of CLI suites: PROCESS_SUITES_SOLO and PROCESS_SUITES at the top of the workflow. The shards build their --path-ignore-patterns from it, and the CLI-process jobs run it, so a suite can't end up in both places or in neither.
  • Shared pass/fail check: script/github/check-bun-test-summary.sh is now used by every job. Bun can segfault on exit after passing, so the summary is parsed instead of the exit code, as before. It also fails on Bun's N error(s) line, which counts errors outside a test. test/ci-bun-test-summary.test.ts tests it, like the existing ci-tsc-whitelist.test.ts.
  • Path filter: the typescript path filter includes the script, so a change to it runs the tests.
  • Required checks: no status checks are required on main, so renaming the TypeScript check doesn't block merges.

Not in this PR: five cliIt suites (providers-logout, run/budget-finalization, run/json-output, run/silent-turn, run/time-budget) are skipped in the main pass and missing from the subprocess list, so CI never runs them. That's already true on main. Adding them may surface real failures, so it gets its own PR. learn/cli.test.ts starting a CLI per test is also left for a separate change.

How did you verify your code works?

  • Local run: I ran each job's exact command on main plus this change.

    • The shards and CLI jobs together cover 971 test files, the same as a plain 4-shard run, with none dropped or doubled.
    • The only failures also fail on plain main on that machine (local MCP config), or passed when re-run alone.
  • Step scripts: I extracted them from the workflow and ran them with a stub bun to confirm the arguments each job passes (shard ignore patterns, solo and rest file lists).

  • CI on a2b16761f1 (before the CLI split): the whole run took 14.8 min, against about 41 before.

    JobTime
    TypeScript (1/4)7.3 min
    TypeScript (2/4)4.1 min
    TypeScript (3/4)6.3 min
    TypeScript (4/4)5.5 min
    CLI processes14.6 min

    The CLI-process job's log showed learn/cli at 5.9 min, the other four suites at 4.9 min, and the subprocess pass at 3.0 min, all run one after another. Splitting it in two should bring a run to about 8 min. This head's CI run will confirm.

  • bun test test/ci-bun-test-summary.test.ts: 10 pass.

Cost side effects

None. This changes CI only. A PR run now uses 7 short runners instead of 1 long one, and total runner minutes rise slightly from repeated setup (about 1.5 min per job).

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

šŸ¤– Generated with Claude Code

https://claude.ai/code/session_01XM3J9mB75wZRnzTSGcW2kc

Summary by CodeRabbit

  • Tests
    • Automated TypeScript tests now run in parallel groups, with command-line process tests handled separately.
    • Results are checked across test runs, and runs fail when summaries are missing, report errors, or show test failures.
    • Automated test jobs now have a 30-minute time limit.
  • Chores
    • SDK code generation setup is included in the automated workflow.
  • No end-user-facing changes are included in this update.