fix(segmentation): produce confidence masks

Merged
SH
2d ago
+1013-198

Description

Produce opt-in segmentation confidence masks across the SDK bindings: one byte per source pixel, row-major, quantized to 0–255. Preserve existing defaults, Python positional arguments and absent-output behavior.

Native compatibility

Keep plugin API 14 and the original segmentation options, result, service and operation-table layouts. Confidence uses a separate versioned output and optional operation advertised by a capability flag. Existing engine slots and reserved fields are unchanged.

Confidence requests use the same selected engine and model instance, validate extension headers and payload sizes, and return an explicit unsupported error when the engine lacks the extension. Invalid outputs are freed and reset. Provider leases protect callbacks from unload or replacement; plugin registration and library-handle transfer are atomic. Registry initialization order keeps lifecycle-held leases safe during shutdown, and ONNX rejects nonfinite interpolated confidence logits before quantization.

Core, ONNX and SDK native bridges need updated builds to expose confidence. Existing unrelated ABI-14 private-engine archives do not need a new ABI release for this extension. NeuRT pins, hashes and receipt guards remain unchanged.

Validation

  • Current head 3ec3f22: 31 checks passed, eight skipped, none failed. CI includes native debug/ASan tests, platform builds and SDK checks.
  • Local Python namespace suite: 57 passed. Electron build, all four type-check projects and 241 tests passed; Web build/type checks, Node ESM resolution and 288 tests passed.
  • Targeted native suites, malformed-interface/payload and cleanup checks, concurrent plugin load/unload probes, and ONNX confidence-value probes passed.
  • All four original/new ABI-14 host/plugin pairings passed, including protected memory after legacy-sized options and results.
  • Public C headers compile as C11; public exports and offline prebuilt-pin checks pass.

The standalone binary harness uses logging and model-resolution seams. Private-engine integration and skipped CI lanes remain unverified. A refreshed review of the latest commit and maintainer approval are still pending.

Follow-up issues

  • #1022: Python fake-core support and confidence-byte assertions are included here. Permanent native ONNX confidence-value regression tests remain.
  • #1023: The new direct confidence operation validates mask size and dimensions before Python/Electron copy the output. Permanent malformed-output and error-propagation regression coverage remains.

Both follow-ups remain open and are non-blocking for this PR.

Summary by CodeRabbit

  • New Features
    • Segmentation can optionally return a confidence mask with each pixel’s winning-class probability, encoded as a value from 0 to 255.
    • Confidence-mask output is available in the Electron, Flutter, Kotlin, Python, React Native, Swift, and web SDKs. It is off by default, so results remain unchanged unless requested.
    • The ONNX segmentation engine supports confidence-mask output, with one confidence value for each source-image pixel.