fix(core): truncate guardrail character limits on character boundaries

KU
1d ago
+112-6

Description

createMaxLengthGuardrail and createInputLengthGuardrail truncate with slice(0, maxCharacters), which counts UTF-16 code units, not characters. When the limit lands between the two halves of a surrogate pair, the truncated string ends with a lone high surrogate. That is not valid text: encoding it as UTF-8 replaces it with U+FFFD (?), so output containing emoji or astral ideographs is silently corrupted at the cut point instead of merely being shortened.

What is the current behavior?

const guardrail = createInputLengthGuardrail({ maxCharacters: 11, mode: "truncate" });
// input: "Great job \u{1F44D} team"
// modifiedInput === "Great job \ud83d"   <- trailing lone high surrogate

The same happens in createMaxLengthGuardrail, both in the non-streaming handler and in the streaming handler (where the next chunk can also emit the now-orphaned low surrogate, because the emitted counter still has room).

What is the new behavior?

A truncateToCharacterLimit() helper backs all truncation sites: if the last code unit it would keep is a high surrogate, it cuts one unit shorter. The streaming path pins guardState.emitted to maxCharacters once a chunk has been trimmed, so no later chunk can emit the orphaned low surrogate.

BMP-only text is unaffected — a result is only ever one code unit shorter than before, and still <= maxCharacters.

Related issue(s) linked

No existing issue for this. gh search issues --repo VoltAgent/voltagent surrogate → [], "maxCharacters truncate" → []. It is a bug fix, so I went straight to the PR; happy to file a tracking issue first if you'd prefer one.

Tests for the changes have been added

Three cases in packages/core/src/agent/guardrails/defaults.spec.ts under describe("truncation keeps characters intact"): the input guardrail, the non-streaming output guardrail (asserting metadata.truncatedTo), and the streaming guardrail via the runner.

Validation

Node v24 / vitest 3.2.4 / Windows, run from packages/core with CI=true.

New tests only, source untouched at upstream main (e9fc599f, defaults.ts blob 95c3a4401b1cd0ba55e49b7815a4c2d7c59231a2):

 Test Files  1 failed (1)
      Tests  3 failed | 16 passed (19)

- Expected:
/[\uD800-\uDBFF](?![\uDC00-\uDFFF])|(?<![\uD800-\uDBFF])[\uDC00-\uDFFF]/
+ Received:
"Great job \ufffd"
 ❯ src/agent/guardrails/defaults.spec.ts:440:37

With the fix applied:

 Test Files  1 passed (1)
      Tests  19 passed (19)

Typecheck and package-wide regression, same command over the whole packages/core suite:

npx vitest run --typecheck src/agent/guardrails/defaults.spec.ts
  Tests  19 passed (19)   Type Errors  no errors

pristine main : Test Files  1 failed | 102 passed (103)  Tests  2 failed | 1266 passed | 2 skipped | 1 todo (1271)
this branch   : Test Files  1 failed | 102 passed (103)  Tests  2 failed | 1269 passed | 2 skipped | 1 todo (1274)

The two failures are identical on both sides and unrelated to this diff — they are Windows-only cases in src/workspace/sandbox/local.spec.ts (EBUSY: resource busy or locked, rmdir ... and spawn C:Usersfzktoolsnodenode.exe ENOENT). Passed count goes 1266 → 1269, exactly the three new tests.

Repo tooling on the changed files:

npx biome check packages/core/src/agent/guardrails/defaults.ts packages/core/src/agent/guardrails/defaults.spec.ts
  Checked 2 files in 12ms. No fixes applied.
npx tsc --noEmit   -> rc=0, no output

Notes for reviewers

  • metadata.truncatedTo for the astral case is now 10 rather than 11, because the cut moves one unit earlier. The <= maxCharacters invariant is preserved either way.
  • defaults.ts is also touched by draft PR #1002, last updated 2026-02-13. Its nearest hunk header is @@ -492, mine end at line 505 in the streaming handler, so a small conflict is possible; the changes themselves are unrelated.
  • No docs change: website/docs/guardrails/built-in.md already describes the option as a maximum number of characters, which is what the code now does.
  • AI disclosure: drafted with an AI coding agent (Qwen3.8-Flash in Qoder). I reviewed the diff line by line and ran every command quoted above myself.

PR Checklist

  • The commit message follows our guidelines
  • Tests for the changes have been added
  • Changesets have been added (.changeset/nervous-pandas-tap.md, @voltagent/core: patch)
  • Docs have been added / updated — not needed, see Notes
  • Related issue(s) linked — none exists, see above

Summary by cubic

Fixes createMaxLengthGuardrail and createInputLengthGuardrail truncating on UTF-16 code units instead of characters, so a limit that lands inside an emoji or astral ideograph no longer emits a lone surrogate (which encodes as � in UTF-8 and corrupts the output).

  • Adds a truncateToCharacterLimit() helper for all truncation paths and pins the streaming emitted counter once a chunk is trimmed, so the orphaned low surrogate can't be emitted by a later chunk.
  • metadata.truncatedTo can now report one less than the requested limit when cutting through a surrogate pair; BMP-only text is unaffected.

Summary by CodeRabbit

  • Bug Fixes
    • Input and output length limits now truncate text at complete character boundaries, preserving emoji and other supplementary characters instead of producing replacement characters.
    • The fix applies to direct guardrail handling and streamed output, including cases where a character spans multiple UTF-16 code units. When text is truncated, the reported output length remains consistent with the enforced limit.