Robustness, platform and performance fixes for the next Netdata vendoring

KT
4h ago
+41549-9649

Summary

Robustness, platform and performance work since the last Netdata vendoring (63919ec, #17), and the source side of the next vendoring. Each change landed as one SOW commit; the SOW files under .agents/sow/done/ carry the evidence.

Bugs fixed (each with a reproduction test that fails before the fix)

  • Rust receive no longer times out after one 100 ms abort-poll slice.
  • UDS packet size capped at the kernel datagram limit.
  • SHM sessions detect a dead peer: servers end the session when the client is gone; clients fail a waiting call with a new disconnected error (C NIPC_ERR_DISCONNECTED, Rust NipcError::Disconnected, Go protocol.ErrDisconnected) within one 100 ms slice instead of the full timeout.
  • Server handshakes are bounded; server stop and shutdown are reliable.
  • Client connects, handshakes and retries are bounded by the call deadline.
  • EINTR is retried; POSIX endpoints take an endpoint lock, so a second server gets address-in-use; transport failures are reported distinctly.
  • SHM regions are created only for sessions that negotiate SHM.
  • POSIX listener sockets are created owner-only without changing the process umask; the endpoint lock and the SHM region file stay usable under a restrictive umask.
  • 32-bit SHM bounds, 32-bit LFS stale cleanup, musl msghdr.
  • POSIX and Windows Named Pipe pipelining clients keep progressing when the socket or pipe buffers fill.

Security

  • Windows pipes and SHM objects are restricted to the server's account, SYSTEM and Administrators (new netipc_win_security.c).
  • Every socket and SHM descriptor is close-on-exec.

Platforms

  • FreeBSD: full support, including the POSIX SHM transport (amd64 and 386 for Go).
  • macOS: runs over a framed AF_UNIX stream baseline (no SHM).
  • Go on all Netdata Linux architectures; C, Rust and Go suites run on every Netdata Linux target in CI.
  • Go modules and pinned CI Go moved to 1.27, matching Netdata; Go adds golang.org/x/sys v0.49.0 (macOS fd-relative bind).

Performance

  • Go hot path: no per-request allocations or wall-clock reads.
  • Same-host A/B against 63919ec (what Netdata runs today):
    • POSIX: geometric mean 1.048 over 345 rows; row-interleaved reruns 1.114; Go rows up to 3.7x.
    • Windows: the Named Pipe pipeline-batch rows are about 10-20% lower, accepted in SOW-0064; snapshot refresh over Named Pipe may be about 5% lower (within this host's noise; accepted in SOW-0050).
  • The committed benchmarks-*.csv are unchanged: both benchmark hosts now run at about half their June numbers for old and new code alike; tracked by SOW-0069.

Tests and CI

  • Correctness suites are deterministic and fast; fuzz, stress and scale suites moved to tests/run-extended-*.sh.
  • CI and scanner fixes before vendoring (DACL test SDDL normalization, CLOEXEC test, armv7 static linking, staticcheck, Semgrep pin).

Validation

  • Linux, FreeBSD 14.3 and 15.1, macOS and Windows suites for C, Rust and Go, plus cross-language interop (details per SOW).
  • Extended fuzz (15 targets) and stress (7 suites) on the candidate.
  • GitHub Actions on 4261919: see the checks on this PR. Known non-code failures: Codacy coverage upload (token permission) and OpenSSF Scorecard (default branch only).
  • Code scanning, Dependabot and secret scanning: no open alerts.

Follow-ups

  • SOW-0068: restore Go coverage to its 90% threshold (88.4% today, 88.3% before this branch).
  • SOW-0069: benchmark host drift and floor re-baseline.

Netdata vendoring PR: netdata/netdata#24633