feat(analyst): add sandboxed python_exec tool

Draft
AN
4d ago
+2682-6

Note: the database first-open race fix in this PR (setup-lock.ts and related files) is also proposed separately as #1541 and will drop out of this PR once that merges.

Issue for this PR

Related to #1537 (no Closes needed for a feat: title).

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Why. The analyst agent denies bash, so it has no general-purpose compute. Parsing, statistics, row-wise text logic and cross-table work that SQL expresses badly have no path today.

What changed. Adds an analyst-only python_exec tool (new module-scope files src/altimate/tools/python-exec.ts and src/altimate/tools/python-sandbox.ts; no new members in any export namespace). Edits in upstream-shared files (agent/agent.ts, tool/registry.ts) are wrapped in altimate_change markers.

Security model:

  • Inputs are scratch tables exported to Parquet; save(df, name) writes DataFrames back to scratch only after a successful call. Python never gets warehouse connections.
  • The child runs with a minimal, explicit environment (PATH, HOME, TMPDIR, locale, a few thread limits). No inherited credentials, tokens or connection strings.
  • Persistent, size-capped, session-private workspace under the session's scratch directory; table names are restricted to plain identifiers (no path separators or schema prefixes); symlinks in the workspace are rejected; exceeding the cap kills the call and clears the workspace.
  • Memory limit (address space, with an RSS watchdog fallback where the OS refuses setrlimit), a clamped wall-clock deadline, process-group kill on timeout/abort. No hang on inherited pipes: the call returns on process exit or at the deadline, kills the process group and every tracked descendant (including ones started with start_new_session=True), then closes the pipes itself, so a descendant holding stdout open cannot block the result.
  • Filesystem confinement, fail-closed. Python may read only the session's Python directory (workspace and per-call files), the interpreter installation (sys.prefix, sys.base_prefix, exec prefixes, stdlib, site.getsitepackages(), sysconfig paths, and directories of native libraries it loads) and the minimal system files an interpreter needs. It may write only the session's Python directory. Subprocesses inherit the same confinement. An installation path that would expose / or the home directory is refused.
    • macOS: sandbox-exec profile that denies file reads and writes outside the allowlist, all network, Apple Events, and Mach services outside a small allowlist (so LaunchServices cannot start unconfined helpers). Path metadata is allowed only on ancestors of allowlisted paths.
    • Linux: bwrap with --unshare-all (network, PID, IPC, user, UTS and cgroup namespaces), --die-with-parent, --new-session, --cap-drop ALL, a fresh /tmp, /dev and /proc, read-only binds of exactly the allowlist (no host-root or home bind), merged-usr symlinks (/bin, /lib64, ...) recreated as symlinks, and a writable bind of the session directory. If bwrap is missing or cannot create namespaces, the tool refuses with a clear message. Landlock alone is not used as a fallback: it does not mediate connect() to pathname Unix sockets (Docker, D-Bus, SSH agent), which can reach host files.
    • Signal isolation (macOS): the profile adds (deny signal) with (allow signal (target self) (target children)), so sandboxed code cannot signal the agent (os.kill(os.getppid(), ...)) or any other host process; it can still manage its own children. Verified on macOS with a test. On Linux the PID namespace already provides this.
    • Stock Ubuntu 24.04: kernel.apparmor_restrict_unprivileged_userns=1 makes unprivileged bwrap fail (loopback: Failed RTM_NEWADDR), so python_exec refuses, which is intended. The refusal now names the AppArmor setting and the remedies (an AppArmor profile for bwrap, or the sysctl, as an administrator's choice), or says to install bubblewrap when it is missing. System settings are never changed automatically.
    • Windows and other platforms: refused; no supported mechanism.
    • Every mechanism must pass a self-test before first use: the sandboxed interpreter must import pandas and duckdb and read its own directory, while both Python and a cat subprocess fail to read a canary file outside the allowlist. A mechanism that fails is never used; if none passes, the tool refuses. Python is never run unconfined.
    • Every result reports the mechanism, e.g. Isolation: filesystem bwrap (...); network isolated.
    • Known residual on macOS: Seatbelt cannot block the KERN_PROCARGS2 sysctl, so sandboxed code can still read the arguments and environment of other processes owned by the same user (checked locally with several profile rules; none blocked it). It cannot read files, use the network or start unconfined helpers. The listing of / itself is readable because the dynamic loader needs it. On Linux the PID namespace hides other processes.
  • Registered only for analyst, and only when a Python with pandas and duckdb is found. MCP, plugin and file tools cannot claim the python_exec id. Bash remains denied for the analyst.

Before: the analyst can only use SQL tools. After: the analyst can materialize data to scratch, run Python over it, and read the saved tables back with SQL.

Flag: ALTIMATE_PYTHON_EXEC=0 disables the tool (it is not registered and calls are refused). ALTIMATE_PYTHON selects the interpreter.

How did you verify your code works?

  • Independent review of this revision (head 511585788f): local Codex CLI could not run (usage limit), so a Sonnet agent reviewed it instead. Its findings were fixed: merged-usr hosts would have made bwrap unusable (symlinks are now recreated), Landlock fallback could reach host services over pathname Unix sockets (removed; Linux refuses without bwrap), the cat self-test leg lacked a positive control (added), and refusals were cached until restart (no longer). It also reported that a Seatbelt sysctl-read rule blocks KERN_PROCARGS2. I could not reproduce that here, so it is listed as a residual above. Earlier review (head da0beea431) findings on host file reach are addressed by this revision.
  • bun test test/altimate/tools/python-exec.test.ts test/agent/python-exec.test.ts test/altimate/tools/tool-schema-anthropic.test.ts test/mcp/lifecycle.test.ts: 91 pass, 0 fail (macOS). test/agent alone: 163 pass; test/tool/registry.test.ts: 14 pass. New coverage: reads of a temp file in another directory, ~/.ssh, /etc/hosts, /etc/hostname and the home directory are denied; cat and /bin/sh subprocesses cannot read outside the allowlist but can read the workspace; pandas/duckdb Parquet round trip inside the sandbox; writes outside the workspace denied; the tool refuses when detection reports no mechanism; Windows/other platforms and Linux without a mechanism are refused; a fake bwrap that runs Python unconfined fails the self-test; exact bwrap argv (allowlist ro-binds, merged-usr symlinks, writable session bind, no host-root bind); macOS profile rules; allowlist rejection of / and home roots; MCP tools cannot claim python_exec.
  • Concurrent first run (database is locked). Starting 3 run --agent analyst processes at once against a fresh data directory failed one or two of them with database is locked (reproduced with ALTIMATE_PYTHON_EXEC=0 too, so it was a latent race that this tool's slower startup exposed, not a python_exec lock). Cause: the core SQLite opener ran PRAGMA journal_mode = WAL before any busy handler existed, and the legacy and core database layers both ran migrations whose deferred read-to-write upgrade fails at once with SQLITE_BUSY. Fix (head 01379b70f4): busy handler installed before the WAL switch, and first-run setup serialized across processes by an exclusive transaction on a sentinel database (packages/core/src/database/setup-lock.ts). Python discovery and the self-test never run inside any database transaction. New test/storage/concurrent-open.test.ts opens both layers from 4 simultaneous processes, 3 rounds; it fails on the previous head (database is locked) and passes now. Reproduction before/after: 3 concurrent CLI runs, 5 of 6 rounds failed at least one process before, 0 of 8 after. test/session 1302 pass; core database-migration 21 pass; typecheck clean.
  • Linux was not executed. These checks ran on macOS only. The bwrap path is covered by argv unit tests and by the fail-closed self-test that runs before first use on a real host; it has not run on a Linux machine yet.
  • bun run typecheck: clean.
  • bun run script/upstream/analyze.ts --markers --base origin/main --strict: reports no upstream-shared files for the main baseline (known v2 baseline limitation); markers were checked by hand.
  • Live check, CLI from source, analyst agent, google-vertex/gemini-3.8-flash, temp project with a 3-row DuckDB table. The Gemini enum-schema fix from #1532 is not on main, so I ran against a throwaway local branch with only that fix applied (not in this PR). The agent called sql_materialize (3 rows to scratch), one python_exec (saved wc), sql_execute on wc, then answered: total word count 9, correlation 0.99919, matching a hand calculation. The run reported network isolated and the RSS-watchdog memory fallback on macOS.
  • An earlier live run, before read confinement, showed the model using python_exec to read host files and run git. That is the gap this revision closes: host reads are now denied by the OS sandbox and the tool refuses where no sandbox can be enforced.

Screenshots / recordings

N/A (no UI change).

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • The analyst can use Python for non-SQL calculations and cross-table analysis with data materialized in the session workspace. Availability depends on Python requirements and secure execution support; the tool is not available to other agents.
  • Security
    • Python runs without network access and is restricted to approved files and the session workspace. Execution has time and memory limits, output is bounded, and workspace storage is capped. If secure confinement is unavailable, Python execution is refused.
    • External tools named python_exec are excluded from tool discovery.
  • Bug Fixes
    • Improved database reliability when multiple processes initialize or open the same database concurrently.

[!NOTE]
High Risk
Introduces sandboxed arbitrary Python execution with complex OS confinement and process cleanup; misconfiguration or sandbox bypass would be security-critical, though the design is fail-closed and analyst-scoped.

Overview
Adds an analyst-only python_exec tool so read-only analysis can run Python (parsing, stats, row logic) after data is in session scratch tables—without bash or warehouse credentials.

Execution exports declared scratch inputs to Parquet, runs user code with altimate_io.load / save, and imports successful save outputs back to scratch. A session-private, size-capped workspace persists across calls; timeouts, memory limits, and process-group cleanup apply on abort or overrun.

OS confinement is mandatory: macOS sandbox-exec or Linux bwrap, with self-test before use; the tool refuses if no mechanism works (e.g. missing bwrap, AppArmor blocking user namespaces). The child gets a minimal env (no inherited secrets), no network, and restricted filesystem access.

Wiring: analyst permission and prompt mention python_exec; the tool registers only when pandas/duckdb Python is found (ALTIMATE_PYTHON_EXEC=0 disables). python_exec is reserved in the tool registry and MCP catalog like answer_commit. Tests cover sandbox behavior, registration scope, and MCP collision avoidance.

Reviewed by Cursor Bugbot for commit 35ef3883a64d1bdcfc49bbc5982a694cb6ccaa08. Bugbot is set up for automated code reviews on this repo. Configure here.