Add goal-oriented Agent Sandboxes guides
Goal-oriented guides for Agent Sandboxes, stacked on #2658. Bill added UI guides (workbench, profiles, inspect) and hardening on this branch. The base is Samuel's agent-sandboxes branch, so this diff shows only the new work.
New pages
- Build with Agent Sandboxes:
- answer questions that need dozens of tool calls;
- work with large tool results;
- give an agent a whole API (typed module plus pre-run type check);
- pause for human approval and resume later;
- keep work across conversation turns.
- Connect your agent: Pydantic AI (
CodeMode(monty_pool_factory=logfire.agent_sandboxes), pydantic/pydantic-ai#11179) and any agent framework (arun_codetool). - What sandbox code can do: the Python subset, the 50 ms and 20 MiB limits, storage limits, and a "Coming soon" list for pricing, regions, and self-hosting.
- Overview: an Experimental marker, and a table comparing one-tool-call-per-turn, container sandboxes, and Agent Sandboxes.
Changes to #2658's pages
- Correct key-creation path (Agent Sandboxes → Setup → Create execution key), and the real permission label.
- Fixed admonition indentation.
- Raised the
agent-sandboxesextra topydantic-monty-client>=1.3.0, to pick up the 1.3.0 security fix andExternalModule.
Measured, not asserted. The claims come from real-model runs with Pydantic AI on staging-eu (3 runs each, strict grading):
| Scenario | Model | Plain tool calls | Agent Sandboxes |
|---|---|---|---|
| Availability from 1,680 rows of request stats | gpt-6.1-sol | 1/3 correct, 111k input tokens | 3/3, 4.7k |
| same | gpt-6-luna | 0/3, 180k | 3/3 (but it returned raw rows, so no token saving) |
| Deploy/error correlation (2,160 rows) | gpt-6.1-sol | 3/3, 70.5k | 3/3, 4.0k |
| same, 288 rows | gpt-6.1-sol | 3/3, 10.1k | 3/3, 22.6k (no saving at this size) |
| Refund 37 orders with approval | gpt-6.1-sol | 37 approval prompts, 2 model requests (~26.4k tokens) after approval | 1 approval prompt, 0 model calls after approval |
The approval guide's own code (with Bill's plan-matching hardening) ran 5/5 with gpt-6.1-sol. That needed two additions:
- tell the model it is not called again after approval;
- type-check before running.
How it was checked
- Every code block was run. Outputs on the pages are real.
- Hosted runs used the GCP test deployment and staging-eu. Pages that test deployment couldn't serve were run against a local Monty relay, or locally with production limits.
- Approval and Pydantic AI pages were also run end to end with a real model.
- Two independent review passes: adversarial accuracy and newcomer persona.
Before this ships
-
The Pydantic AI page needs a pydantic-ai-harness release that includes #11179.
-
Recheck on staging/prod for:
- parallel host calls under
asyncio.gather: they were nearly serialized on the test deployment; - type-check memory: about 5 MiB is charged when a script imports
asyncio.
Both are reported to the Monty team.
- parallel host calls under
-
sessions.mdhas two claims I could not verify on hosted: resume after the 1-hour connection limit, and how restored limits interact withcheckout()options.
Drafted with Claude (Opus 5.5), with my review and direction.