serve: refuse malformed tools arrays and non-boolean parallel_tool_calls with 400

PS
2h ago
+107-7

serve: refuse a malformed tools array or a non-boolean parallel_tool_calls with 400

Problem

chat_prompt() passed body["tools"] into the chat template and the parser unchecked. On the real GLM-5.3-Flash
template this had two failure modes (reproduced on current development over tests/test_serve_api.py's stand-in
engine with the template fixture):

requestbeforeafter
"tools": "notalist"500 (ValueError inside apply_chat_template; _template_ids catches only jinja2.TemplateError)400 `tools` must be an array of function definitions
"tools": [1]500 (same)400 `tools[0]` must be an object
[{"type": "function"}] (no function)200, silently400 `tools[0].function` must be an object
[{"type": "function", "function": {}}] (no name)200, silently400 `tools[0].function.name` must be a non-empty string
parameters: "x"200, silently400 `tools[0].function.parameters` must be an object
description: 7200, silently400 `tools[0].function.description` must be a string
type: "nope"200, silently400 `tools[0].type` must be "function"
"parallel_tool_calls": "false" / 0 / {}200, parallel calls kept400 parallel_tool_calls must be true or false
valid tools + parallel_tool_calls: false200200

The first two violate AGENTS.md invariant 7 (unsupported API fields are refused with a 400 that says why); the silent
cases are worse, since the template renders a different prompt and parallel_tool_calls stayed on.

Change

  • trnrt/serve/parsers.py: tools_error(tools) next to tool_choice_error, returning why a non-conforming array is
    refused. Mirrors vLLM's ChatCompletionToolsParam: type defaults to "function", absent/null is no tools, an
    empty array is allowed. A tool_choice of "none" still renders and parses exactly as before (the tools are
    dropped after validation).
  • trnrt/serve/api.py: chat_prompt() calls tools_error beside tool_choice_error and raises via _bad;
    parallel_tool_calls goes through the existing _flag(body, key, default=True) instead of
    body.get(...) is not False, so "false", 0 and {} are 400 rather than silently True.
  • Tests: tests/test_serve_api.py::test_malformed_tools_and_parallel_tool_calls_are_400 (all shapes above plus a
    valid positive control) and tests/test_glm53_template.py::test_a_malformed_tools_value_is_400_not_a_template_error
    (the 500-producing shapes over the real GLM template, asserting the engine saw no request).
  • CHANGELOG.md entry under Unreleased/Fixed.

Verification

Red/green on the same tests, trees from git archive development:

  • Unpatched tree: 2 failed — the serve test returns 200 for every silent case, and the GLM test gets the
    ValueError from transformers/utils/chat_template_utils.py:525 (TestClient re-raises it; a real server answers
    500 — confirmed with raise_server_exceptions=False in the status matrix above).
  • Patched tree: 2 passed in 4.34s; the full patch applied with patch -p1 to a second pristine archive also passes
    both tests (2 passed in 2.60s), so the diff is self-contained.
  • pytest tests/test_serve_api.py tests/test_glm53_template.py: 31 passed.
  • ruff (check --ignore EXE002, format --check) clean on all four changed files; mypy clean on both source files.
    (EXE002 is only the exFAT mount's file mode; CI checkout is 644.)

Notes

  • Overlap: open PR #36 (serve-tokenize → development, 48 files, still open) also rewrites trnrt/serve/api.py.
    This patch is two localized hunks in chat_prompt; merge order #36-then-this keeps the conflict to those hunks.
    parsers.py is not touched by #36.
  • Scope: host validation only — no traced code (trnrt/kernels/, trnrt/models/, model_runner.py,
    capture.py, config.py, platform.py), no behavior change for conforming requests.

Checklist

  • One logical change
  • Tests come with the change; red on development, green after
  • Lint, format, mypy clean
  • CHANGELOG entry
  • Rebase on development after #36 lands (if it lands first)

🤖 Generated with Claude Code


Summary by cubic

Validates the tools array and parallel_tool_calls flag at the host boundary before they reach the chat template or parser. Non-array tools previously caused a 500 inside the template; malformed entries and non-boolean parallel_tool_calls were silently accepted, changing the rendered prompt and keeping parallel calls on.

  • Adds tools_error() to check each entry against the OpenAI schema: object shape, type, function name ([A-Za-z0-9_-]{1,64}), parameters and parameters.properties objects, description string, and refusal of strict: true.
  • Routes parallel_tool_calls through shared boolean validation, treating an explicit null as invalid as well.
  • Skips tool validation when tool_choice is "none", since the request discards tools exactly as before.
  • Tests cover each failure shape plus accepted controls: valid tools with parallel_tool_calls: false, strict: false, and tool_choice: "none".

Summary by CodeRabbit

  • Bug Fixes
    • Malformed tools values and entries now receive an HTTP 400 response identifying the invalid field, rather than causing a template error.
    • Non-boolean parallel_tool_calls values, including explicit null, are rejected with HTTP 400 instead of being treated as enabled.
    • Malformed tool definitions are ignored when tool_choice is set to "none".
    • Valid tool definitions with parallel_tool_calls set to false and strict set to false continue to be accepted.