feat(sql): execute generated read queries without model round trips

AN
3h ago
+508-14

Issue for this PR

Closes #1489

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Stacked on feat/analyst-harness-dab-2026-10 (#1441).

Why. Reading across many tables (for example a table set built by a generator query) forced the model to receive the generated SQL, then retype it into sql_execute. For large warehouses the generated SQL is clipped in the preview or moved to a spill file, so the model had to recover and re-emit tens of thousands of characters, which is slow, costly and error-prone.

Root cause. sql_execute required literal query text; there was no way to say "run the SQL produced by this read". This is a missing capability, not a defect. The issue's other items are engine behavior or existing permission policy (DuckDB subquery/lateral/named-parameter limits and EXECUTE IMMEDIATE are engine-side; session settings and macros keep their write permission policy; preview clipping is intentional).

Before -> after. Before: the generated SQL had to pass through the conversation. After: sql_execute accepts query_from: { warehouse, sql }. The source SELECT must return exactly one row and one non-empty string column (not truncated, at most 1 MiB UTF-8). That string is validated as a single read-only SELECT and executed on the target warehouse, without being previewed or retyped. Only the final result is formatted.

  • Exactly one of query / query_from is required; query_from needs an explicit target warehouse.
  • Strict-schema clients that send query_from: null or {warehouse:"", sql:""} (or an empty query) for the unused field are accepted; a half-filled query_from is rejected.
  • Source and generated reads are each permission-checked and routed independently. Generated writes, modifying CTEs, metadata commands and statement batches are rejected (including on scratch). PostgreSQL keeps read-only transactions; limits and deadlines are forwarded.
  • Tool description documents DuckDB query_table([...], true) and batching of at most 128 names per UNION ALL BY NAME to stay within the default expression depth.

Deferred. No measured cost/token improvement (affected sessions unavailable offline); sql_materialize, table globs and remote workspace tools are not extended; the 1 MiB limit applies after retrieval and is not a driver-level transfer bound. Review follow-up (P2, not fixed): a source SELECT with duplicate output column aliases can collapse to one column in drivers that return keyed rows (e.g. SQLite), so the single-column check can be bypassed; the generated SQL is still validated as a read-only SELECT, so this is not a permission bypass.

How did you verify your code works?

  • bun test test/altimate/tools/sql-execute-query-from.test.ts: 51 pass, 0 fail (includes a real DuckDB aggregate over 1,024 synthetic tables with a compact request and no spill, and strict-schema placeholder cases).
  • All test files referencing sql_execute (test/altimate/tools plus carry-forward, feature-discovery, precedence, scratch-sessions, sql-materialize, sql-validation, telemetry, warehouse-failure, session, skill and telemetry suites): 1138 pass, 11 fail. 4 failures are pre-existing, identical at the base (datamate_manager x2, a lintSqlResult large-cache test, one sample_setup test); the remaining sample_setup failures were 5s timeouts under load and pass in isolation (8 pass, 0 fail).
  • bun run typecheck (packages/opencode): pass. git diff --check: pass.
  • Marker check: sql-execute.ts does not exist in the last v1 baseline tag and lives under src/altimate/**, so no upstream markers are needed.
  • Independent Codex review: P1 (strict-schema callers could not represent an unused query_from) fixed with regression tests; P2 above noted.

Screenshots / recordings

N/A (no UI change).

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

šŸ¤– Generated with Claude Code

[!NOTE]
Medium Risk
Extends the SQL execution tool with a two-hop path and stricter read-only validation; changes are guarded by permissions and expanded function denylists but still touch warehouse execution behavior.

Overview
Adds query_from to sql_execute so a compact source SELECT can fetch generated SQL internally and run it on a target warehouse without copying megabytes of SQL through the model or tool preview.

The flow requires exactly one of query or query_from, an explicit target warehouse, and validates both the generator and the returned string as single read-only SELECTs (1 MiB cap, strict one-row/one-column scalar). Source and target each get permission prompts, workspace precedence, and PostgreSQL read_only where applicable; workspace redirects can include the generated SQL up to 40 KiB. readSql in sql-check is tightened to block more callable side effects (Unicode-escaped identifiers, Snowflake SYSTEM$*, PG session/backend/replication helpers). Docs and a large test suite cover placeholders, routing, and a 1k-table DuckDB aggregate.

Reviewed by Cursor Bugbot for commit 9c780d359c86ede1a68a307dd1c6a2bf230c0f0b. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by cubic

Adds a query_from option to sql_execute so the model no longer has to copy generated SQL through the conversation and re-emit it as a literal query. Closes #1489.

Previously, executing a read produced by a generator query forced the model to receive the SQL, then retype it into sql_execute; for large warehouses the SQL got clipped in the preview or moved to a spill file, making recovery slow and error-prone. Now query_from: { warehouse, sql } runs a source SELECT and executes its single SQL string result (up to 1 MiB UTF-8) as a read-only SELECT on the target warehouse. The source SQL and generated SQL are each permission-checked and routed independently; generated writes, modifying CTEs, metadata commands, statement batches, and PostgreSQL session, backend, advisory-lock, logical-slot, and system-control functions are rejected (including Unicode-escaped spellings, SYSTEM$ calls, and WAL replay controls). Only the final result is formatted, so the generated SQL never enters the preview or spill path.

  • Requires exactly one of query or query_from; accepts null or empty placeholders from strict-schema clients but rejects half-filled query_from.
  • Source SELECT must return exactly one row and one non-empty string column without truncation; result is validated as a single read-only SELECT before execution.
  • Preserves read-only transactions on PostgreSQL and forwards limits and deadlines to both reads.
  • Workspace redirects carry the generated SQL so they can still be followed, but refuse redirects whose SQL exceeds 40 KiB; results are titled generically without exposing the SQL.
  • Documents DuckDB query_table([...], true) and batching of at most 128 table names per UNION ALL BY NAME.

Tests: a dedicated suite passes, including a real DuckDB aggregate over 1,024 synthetic tables with a compact request and no spill; all existing suites referencing sql_execute pass (11 failures, all pre-existing at the base).