fix(ai): request readable reasoning summaries for capture

Merged
UR
1d ago
+80-20

Summary

ThinkEx's Rudimint captures contain empty reasoning blocks and provider signatures because chat requests do not ask providers to return readable summaries. Request those summaries in the shared chat provider configuration so future captures can contain text when the model emits it.

  • Request OpenAI detailed reasoning summaries and Google/Vertex thought summaries across primary and fallback routes.
  • Request Anthropic adaptive thinking with summarized display only on routes containing Sonnet 5.5. Leave Haiku thinking untouched.
  • Preserve existing Sol and Gemini effort settings, title generation settings, and hidden reasoning in the chat UI. The shared configuration also applies to chat compaction.

These are provider-exposed summaries, not full internal reasoning. The settings follow the Anthropic AI SDK documentation, Google AI SDK documentation, and Gateway provider options.

Testing

  • 12 focused tests passed; changed-file formatting/lint/type checks and repository tsc --noEmit passed. GitHub validate, CodeQL and react-doctor checks passed.
  • 30 live synthetic scenarios using the existing ThinkEx Infisical profile: 12 before/after tool scenarios across all six distinct picker models, six pinned fallback/provider scenarios, six repeated Vertex before/after scenarios, five Rudimint capture scenarios, and one alternate OpenAI summary-format diagnostic.
  • All 24 tool scenarios completed with the tool called and correct final purchase counts. Pinned checks covered Sonnet, Sol, Haiku, Gemini, Azure and Vertex. These test individual fallback legs; they do not simulate every cross-model outage transition.
  • Verified readable summaries persisted in Rudimint Postgres: Sonnet 1,964 characters, Luna 1,356, Gemini Flash 1,439, Gemini Pro 1,798. Downloaded Sonnet's raw R2 capture and confirmed 166 reasoning deltas totaling the same 1,964 characters.
  • Sol returned 697 reasoning tokens but no readable summary; a separate reasoningSummary: "auto" run also returned none. Summary availability is not guaranteed. Haiku thinking remains disabled.
  • All harness sources and JSON results are local untracked research under research/reasoning-capture/.

Review Notes

Live testing exposed existing routing limits: one Gemini tool continuation received Invalid thought signature from Vertex and recovered through Google. This did not reproduce in three pinned Vertex before/after pairs. One of those enabled-summary runs hit the 30-second BYOK timeout and recovered through system credentials. The Azure Sol BYOK deployment was missing and recovered through system credentials. These are successful responses with degraded routing, not clean provider passes. The small sample cannot establish whether summary settings affect the timeout rate or isolate latency overhead.

The harder capture task was a transport check, not a model-quality pass: Sonnet and Sol found all 16 subsets; Luna and Flash missed one, and Pro exhausted the 4,096-token output budget. Readable summaries still need task verification before use as training data.

The initial Infisical limitation was resolved by selecting the existing ThinkEx login profile per command, without changing the default profile. Local checks used Node 25.8.0; the repository declares Node 24. The repository-wide formatting check reports pre-existing untracked research files, excluded from this PR.

Merging to main deploys the ThinkEx production Worker. No code deployment was performed; capture probes sent explicitly tagged synthetic requests through the existing production capture endpoint.