fix: make inconclusive completion checks advisory
Issue for this PR
Closes #1814
Type of change
- Bug fix
What does this PR do?
Previously, a schema or test tool error could make the completion gate ask for another model turn. Now validators distinguish confirmed findings from checks that could not run. In advisory mode, the latter are sent once with the next ordinary user request and use no retries. Confirmed findings keep the same retry cap. Pending notes are stored locally outside shared message metadata.
For a comparison, enable validators in fresh sessions and set ALTIMATE_VALIDATORS_STRICT=1 in the control arm. Leave it unset in the advisory arm. The switch applies across providers. The coordinator's paired cross-model comparison and final default decision are required before shipping.
This PR is based on main. It overlaps with fix/dbt-validator-false-positives in the dbt test validator. That work addresses spurious results at their source. This change handles checks that still cannot complete.
How did you verify your code works?
- Validator suite and real prompt-loop tests with a stub model. The tests cover red builds, failed tests, schema mismatches, invalid incremental configuration, one-shot delivery, strict mode, and the default retry cap.
- The detached baseline requested three extra turns for the advisory scenario and failed the new result classification checks.
- Repository typecheck, upstream markers, and tracker scan passed. Separate local probes verified private-state persistence across processes and cleanup through both session deletion paths.
- GitHub CI passed on the PR head. https://github.com/AltimateAI/altimate-code/actions/runs/44939294095
- Local Codex review found no P0–P3 issues at
73d08bcfdd9f82ff060388da38060895f1f68dd4. The independent review ran once, and subsequent commits addressed its findings.
Cost side effects
None that scale with user data. Advisory delivery adds a short note to an existing model request, after the normal cache breakpoints.
Screenshots / recordings
Internal model context only.
Checklist
- I have tested my changes locally
- I have not included unrelated changes in this PR
- I checked for cost side effects