fix(host-service): bound concurrent GitHub lookups in the PR runtime

BR
1d ago
+285-24

Why

With many projects open, the sidebar shows "GitHub unreachable", clears, and shows it again many times a day, while gh works (#8998). Every PR refresh spawned all of its gh calls at once. gh's 10 s timeout runs from spawn, so calls that answer in 0.4 s alone timed out waiting behind each other, and the unreachable gate read those timeouts as an outage. The recovery refresh then repeated the same burst.

What changed

  • Every GitHub lookup in the PR runtime takes one of 8 slots before it spawns gh. A queued lookup's timeout has not started yet.
  • The gate is checked before a lookup queues and again after it gets its slot. In a real outage the first lookups time out and trip it, and the rest fail fast without spawning gh.
  • The Octokit fallback gets the same 10 s deadline as gh, so a hung fallback releases its slot and trips the unreachable hold like a hung gh.
  • Explicit refreshes, such as a PR just created, go to the front of the queue.

Scope

  • The PR runtime only. The shared execGh and its other callers are unchanged.
  • The gate still treats a killed gh as unreachable. With the limit, a timeout means the call really hung.

Tradeoffs

The limit of 8 lookups, so at most 24 gh processes, spreads a full refresh of about 100 projects over tens of seconds instead of one burst. That is well inside the 5-minute refresh interval. A higher limit brings the contention back.

Blast Radius

Every GitHub lookup in the PR runtime now waits for a slot when 8 are running. A user-triggered refresh skips ahead of the background sweep.

Verification

  • New tests run a 96-project refresh against a fake gh in virtual time, with 4 processes at once, 400 ms per call, and a 10 s timeout. Before the fix, a healthy GitHub tripped the gate, and a hanging GitHub spawned 192 processes. After it, the gate stays open, and a hanging GitHub still trips it with fewer spawns than projects.
  • Another test hangs 8 Octokit fallbacks and checks that an explicit refresh still completes. The pull-request runtime suite passes (115 tests), and host-service typecheck and biome are clean.
  • With real gh on one machine, latency grew with concurrency (1 call: 0.4 s, 300 concurrent pulls?head= calls: up to 2.9 s). It did not reach the 10 s timeout there, so the reporter's timeouts on a busier machine are inferred from that trend.

Fixes #8998

Summary by CodeRabbit

  • Bug Fixes
    • Pull request updates now handle slow or unresponsive GitHub responses more reliably. Concurrent lookups are limited, and availability checks respond to successful and failed requests.
    • Timed-out requests no longer prevent workspace refreshes from completing. Repeated failures are detected before additional per-project processes are started.
    • GitHub command-line requests now use a 10-second default timeout, while explicitly configured timeouts are preserved.