fix(desktop): keep agents' browser panes reachable past the hidden-webview cap

Merged
KI
1d ago
+768-280

Summary

  • Agents can now list and use their browser panes after hidden-pane eviction unloaded them. The next command rebuilds the pane in the background.
  • browser open --show now answers with the pane it opened, not another tab's pane.
  • Agent panes get their own hidden-webview cap (6), and agent commands count as use. Agents and the user no longer unload each other's pages.

Why / Context

MAX_HIDDEN_WEBVIEWS = 3 covered the whole app, not one workspace. With agents working in several workspaces, I found these problems in the dev app:

  1. browser list left out unloaded panes, so a workspace with 3 agent tabs listed 0.
  2. Every command on an unloaded pane (eval, screenshot, navigate, console, cdp) failed with No webContents for pane …. Only the user showing that tab brought it back.
  3. Agent commands did not count as use. The page an agent had just used 3 times was the first one unloaded when another agent opened a page.
  4. open --show returned the wrong pane id. Showing the workspace rebuilds an unloaded tab, and the bridge took the first pane to register that was not already loaded.

How It Works

  • Open requests: the renderer reports { requestId, paneId } through browser.resolveOpenRequest, on both the background path and the --show path. The bridge waits for that pane to register. This removes the old snapshot heuristic and the per-workspace open queue that existed only to make the heuristic work. A per-workspace pending cap of 8 stays.
  • Revive: resolvePane in browser-bridge.ts handles every per-pane route and the CDP upgrade.
    • If the pane has no webContents, it sends a wakeBrowser op on the existing pane-layout request channel, which targets one window. The renderer finds the pane in the workspace layout and checks that it is an agent pane (data.agent). Then it calls browserRuntimeRegistry.openBackground.
    • wakeBrowser is a desktop-internal op (RendererPaneLayoutOp). It is not in the shared paneLayoutOpSchema, so the host-service /pane-layout route cannot send it.
    • CDP messages that arrive while a pane is rebuilt are queued and sent once the session attaches.
  • List: /panes?workspaceId merges the layout's agent browser panes, in layout order, with loaded: false for unloaded ones. describePaneLayout now fills PaneLayoutPane.browser { url, agent }. If no window answers, the list falls back to the live panes.
  • Use and caps: every bridge command calls browserManager.markAgentUse, which bumps the pane's lastUsedAt through the browser.onAgentUse subscription. The registry evicts two groups separately: user panes (cap 3, unchanged) and agent panes (cap 6).
  • Persistence: a rebuilt pane persists navigations through the new persistAgentBrowserState. It writes to the mounted layout store when the workspace is shown, else to the saved layout. Background opens use the same util now.

Manual QA

I drove the dev app over CDP with the CLI (superset browser …). Three agent workspaces opened 3 pages each while the user watched a fourth workspace. The same script ran on main and on this branch, and I recorded both runs as before/after films (ask me for the video).

                                              main                      this branch
browser list --workspace $CHECKOUT            ✗ 0 of 3 tabs listed       ✓ 3 of 3, unloaded ones marked
browser eval --pane $CART (unloaded)          ✗ No webContents for pane  ✓ "Cart · Checkout"
eval FAQ ×2, open in another ws, eval FAQ     ✗ No webContents for pane  ✓ agent visits: 3 (no reload)
browser open --show --url …/search/results    ✗ got the /search/shoes pane ✓ the pane showing /search/results

Also checked on this branch:

  • browser screenshot on an unloaded pane returns a real PNG.
  • Raw CDP on an unloaded pane works. The first message, sent before the rebuild finished, got its answer.
  • A closed pane gives No browser pane … in this workspace.
  • A rebuilt pane, shown later by the user, attaches without reloading. Its in-page state was kept.
  • After an app restart, nothing is loaded, but agents can still list and use their old pane ids.
  • Right after the app starts, before any window is ready, a command on an unloaded pane answers 503 ("No signed-in desktop window…"), not 404 "No browser pane". Once the window is ready, the same command succeeds.
  • The agent group never goes over 6 loaded hidden pages.

In-app CDP suite (19/19, no renderer errors)

User tabs were opened through the real UI (trusted clicks and typing). Agents used the CLI, and the registry was checked over CDP.

  • The user group keeps 3 hidden pages. An agent command on the user's loaded pane gets 403; on the user's unloaded pane, 404, and the pane is not rebuilt.
  • 9 agent opens leave the user's 3 hidden pages loaded. The agent group keeps 6. list shows all 9, with 3 marked unloaded.
  • navigate, console, screenshot and raw CDP each work on an unloaded pane. CDP answers in about 70 ms, including the rebuild. Host-service browser.reload works on an unloaded pane too.
  • An open into the workspace the user is viewing returns its own pane. When that pane is unloaded and then navigated by the agent, its tab title updates. A real click on the tab shows the same page without a reload.
  • A rebuilt pane's navigation is saved to the layout of a workspace that is not mounted.
  • open --show answers with the pane it opened.
  • Not driven in the app: import-login, because its native confirm dialog can't be answered safely from CDP. Its allow and refuse paths are covered by browser-bridge.test.ts.
  • Found while testing, not caused by this PR: a background open (no --show) into the workspace the user is viewing becomes the active right-pane tab and shows. That comes from the existing background-open and right-pane code.

Testing

  • bun run typecheck in apps/desktop, packages/host-service, packages/cli and packages/shared
  • bunx biome check on the changed files
  • bun test src/main/lib/browser src/main/lib/pane-layout src/renderer/routes/_authenticated in apps/desktop: 2823 pass
  • New tests. I ran each against the old file, and each fails there:
    • browser-bridge.test.ts: an open answers with its own pane even when a rebuilt pane registers first.
    • browser-bridge.test.ts: list shows unloaded agent panes. eval rebuilds an agent pane, and an unloaded user pane is not rebuilt (404).
    • browser-bridge.test.ts: before a window is ready, an agent is told to retry (503), not that its pane is gone.
    • browserRuntimeRegistry.test.ts: user and agent panes are evicted against separate caps, and agent use keeps a pane loaded.

Design Decisions

  • The renderer reports the pane id, instead of a smarter match on registration events. Any match on registration events can be fooled by a pane that is rebuilt at the same moment. Only the renderer knows which pane it created.
  • Revive goes through the pane-layout request channel, not a new channel. That channel already sends each request to exactly one window, with a timeout and a reply. The new op stays desktop-internal.
  • Separate caps, not one per workspace. One cap per workspace would make memory grow with the number of workspaces. A separate agent cap keeps the total bounded (3 + 6 hidden). Rebuilding makes unloading recoverable for agents.

Known Limitations

  • A rebuilt page reloads, so in-page state is lost (form input, JS state, back/forward history). Cookies persist.
  • The right sidebar's hover-peek panel stays faintly visible after the pointer leaves. I saw this while filming; it is not related to this PR.

Follow-ups

  • #9498 (stacked on this PR) replaces the fixed caps with a memory budget and Chrome-style discard rules.

Compatibility

  • Host-service BrowserPane gains an optional loaded field. Older desktops omit it, and the CLI treats a missing value as loaded.
  • The resolveOpenRequest and onAgentUse procedures are on the desktop's internal tRPC over IPC. Main and renderer ship together.

Summary by cubic

Keeps agents' browser panes reachable after hidden-pane eviction unloads them. Commands on an unloaded pane now rebuild it in the background; browser list shows unloaded agent panes; and agent commands count as use, with a separate hidden-webview cap so agents and the user no longer unload each other's pages.

  • The renderer reports { requestId, paneId } per open request (agent opens only); browser open --show answers with the pane it opened, and /open verifies that pane is an agent pane.
  • browser list merges the workspace layout's agent panes and marks unloaded ones loaded: false (the optional field is absent on older desktops).
  • Every per-pane bridge command and CDP upgrade wakes an unloaded agent pane through the desktop-internal wakeBrowser pane-layout op; CDP messages arriving mid-rebuild are queued (capped at 64 frames / 4 MB, past which the socket closes with 1009) and sent once the session attaches.
  • Agent panes get their own hidden-webview cap of 6 (user panes stay at 3), and every agent command bumps the pane's LRU stamp, so agents and the user no longer evict each other's pages.
  • Rebuilt agent panes persist navigations via the new persistAgentBrowserState helper, which writes to the mounted layout store when the workspace is shown, else to the saved layout.
  • When no window is ready to rebuild a pane yet, commands answer 503 and CDP closes with 1013 (try again later) so the agent retries; a 404 now only means the renderer says the pane is gone, a pane that never finishes loading is a 504, and a route error answers 500 without leaving the request open.

Summary by CodeRabbit

  • New Features
    • Browser commands can restore unloaded agent browser panes when needed, while unloaded user panes remain unavailable.
    • Browser pane listings identify panes that are currently unloaded.
    • Opening browser tabs handles overlapping requests more reliably and returns the pane associated with each request.
    • Hidden browser panes are managed separately for regular and agent sessions, with recently used agent panes less likely to be evicted.
    • Pane layout requests can wake an unloaded agent browser pane.
    • Browser connections wait for panes to wake and recover more reliably during restoration.