Improve browser research reliability and bound context growth
Summary
- replace the generic GPT prompt and long browser skill with a compact browser-research contract
- keep full browser output on disk while limiting model-visible tool output to 8 KiB / 400 lines
- use medium response verbosity for GPT-5.5 and make final-answer delivery explicit
- cap individual browser snippets at two minutes and tell the agent to switch strategy after three no-evidence calls
This stays pure TypeScript/JavaScript. It does not add Python, a notebook runtime, a new browser API, or task-specific benchmark logic.
Evaluation
Internal Bench Hard, all 106 tasks, GPT-5.5, medium reasoning:
| Version | Passes | Agent cost | Observed trace cost |
|---|---|---|---|
| Original BrowserCode baseline | 81/106 | $176.92 | $236.50 |
| Immediate pre-guardrail recipe | 86/106 | $152.76 | $196.66 |
| This branch | 87/106 | $132.26 | $181.02 |
Against the immediate predecessor, this is +1 pass with 13.4% lower agent cost, 18.6% fewer cached input tokens, 7.9% fewer uncached input tokens, and 6.2% fewer output tokens. Browser calls increased 9.9%, so the saving comes from leaner model context rather than less browsing.
The 81/106 comparison spans a larger target evolution and should be read as a product-level comparison, not an isolated causal A/B. The 86 -> 87 comparison used the same eval-platform commit and isolates the final evidence-budget commit.
Verification
- browser-execute tests: 7 passed, 6 skipped
- browser skill tests: 2 passed
- provider transform tests: 354 passed
- filtered package typechecks: 17/17 passed
Summary by cubic
Bounds browser research behavior and model context to improve reliability and cost. Previously we used a generic prompt and surfaced full tool output; now we use a compact browser-research contract, persist the CDP session, cap per-call time at 2 minutes, and limit model-visible tool output to 8 KiB/400 lines while saving the full output to disk.
browser_executenow returns compact context with full output saved. Tool metadata includestruncatedandoutputPathwhen truncation occurs. Screenshots still auto-attach.- Max per-call timeout reduced from 10 minutes to 2 minutes.
- Skill doc rewritten to emphasize short deterministic snippets, event-before-action, evidence discipline, reuse via
await import(...?t=...), and switching strategy after three no-evidence calls. - Prompt specialized for browser research and requires final responses to start with "FINAL ANSWER:"; stresses exhaustive, evidence-backed delivery.
gpt-5.5models use mediumtextVerbosity; tests cover the new default.
Review/Rollout
- Required: If any flows relied on >2 minute snippets, split them into smaller calls.
- Required: If anything parsed full console output from the tool result, read the saved file at
metadata.outputPathor use focused Grep/Read instead. - Update expectations for
gpt-5.5verbosity if tests assertedlow. - Expected impact: +1 pass on internal bench (87/106) with ~13% lower agent cost; savings come from tighter context rather than fewer browser calls.