feat(llm): hold model-card discovery writes until validation releases

Closed
SE
18d ago
+387-17

Part 4 of 6 of the worker validation gates stack (DEP ai-dynamo/dynamo#15897). Based on part 3.

Summary

Under etcd discovery the frontend routes to any endpoint with a model card, whatever the pod's readiness, so holding /health alone doesn't keep traffic off a worker under validation.

  • DYN_WORKER_VALIDATION_HOLD=card (default): register_model_card returns as usual, but the discovery write waits. When every gate passes, held cards are written in registration order, then the worker reports ready.
  • DYN_WORKER_VALIDATION_HOLD=health: the card is written at once and only /health is held, for deployments that keep traffic off the worker by other means.
  • A held card can be unregistered but not edited: taint updates are refused until release.
  • If a held card can't be written, the worker shuts down so its gates run again on restart; its discovery lease removes any card already written.

The runtime gains a single release hook that runs after the gates pass and before release; a failed hook keeps the worker held.

Validation

A card is written only on release and taint updates are refused while it's held; hold=health writes it at once; a card unregistered while held is never written; a failed release hook keeps the worker held; config rejects HOLD without gates or with an unknown value.

🤖 Generated with Claude Code