serve: refuse malformed tools arrays and non-boolean parallel_tool_calls with 400
serve: refuse a malformed tools array or a non-boolean parallel_tool_calls with 400
Problem
chat_prompt() passed body["tools"] into the chat template and the parser unchecked. On the real GLM-5.3-Flash
template this had two failure modes (reproduced on current development over tests/test_serve_api.py's stand-in
engine with the template fixture):
| request | before | after |
|---|---|---|
"tools": "notalist" | 500 (ValueError inside apply_chat_template; _template_ids catches only jinja2.TemplateError) | 400 `tools` must be an array of function definitions |
"tools": [1] | 500 (same) | 400 `tools[0]` must be an object |
[{"type": "function"}] (no function) | 200, silently | 400 `tools[0].function` must be an object |
[{"type": "function", "function": {}}] (no name) | 200, silently | 400 `tools[0].function.name` must be a non-empty string |
parameters: "x" | 200, silently | 400 `tools[0].function.parameters` must be an object |
description: 7 | 200, silently | 400 `tools[0].function.description` must be a string |
type: "nope" | 200, silently | 400 `tools[0].type` must be "function" |
"parallel_tool_calls": "false" / 0 / {} | 200, parallel calls kept | 400 parallel_tool_calls must be true or false |
valid tools + parallel_tool_calls: false | 200 | 200 |
The first two violate AGENTS.md invariant 7 (unsupported API fields are refused with a 400 that says why); the silent
cases are worse, since the template renders a different prompt and parallel_tool_calls stayed on.
Change
trnrt/serve/parsers.py:tools_error(tools)next totool_choice_error, returning why a non-conforming array is
refused. Mirrors vLLM'sChatCompletionToolsParam:typedefaults to"function", absent/null is no tools, an
empty array is allowed. Atool_choiceof"none"still renders and parses exactly as before (the tools are
dropped after validation).trnrt/serve/api.py:chat_prompt()callstools_errorbesidetool_choice_errorand raises via_bad;
parallel_tool_callsgoes through the existing_flag(body, key, default=True)instead of
body.get(...) is not False, so"false",0and{}are 400 rather than silentlyTrue.- Tests:
tests/test_serve_api.py::test_malformed_tools_and_parallel_tool_calls_are_400(all shapes above plus a
valid positive control) andtests/test_glm53_template.py::test_a_malformed_tools_value_is_400_not_a_template_error
(the 500-producing shapes over the real GLM template, asserting the engine saw no request). CHANGELOG.mdentry under Unreleased/Fixed.
Verification
Red/green on the same tests, trees from git archive development:
- Unpatched tree:
2 failed— the serve test returns 200 for every silent case, and the GLM test gets the
ValueErrorfromtransformers/utils/chat_template_utils.py:525(TestClient re-raises it; a real server answers
500 — confirmed withraise_server_exceptions=Falsein the status matrix above). - Patched tree:
2 passed in 4.34s; the full patch applied withpatch -p1to a second pristine archive also passes
both tests (2 passed in 2.60s), so the diff is self-contained. pytest tests/test_serve_api.py tests/test_glm53_template.py: 31 passed.- ruff (
check --ignore EXE002,format --check) clean on all four changed files; mypy clean on both source files.
(EXE002 is only the exFAT mount's file mode; CI checkout is 644.)
Notes
- Overlap: open PR #36 (
serve-tokenize→development, 48 files, still open) also rewritestrnrt/serve/api.py.
This patch is two localized hunks inchat_prompt; merge order #36-then-this keeps the conflict to those hunks.
parsers.pyis not touched by #36. - Scope: host validation only — no traced code (
trnrt/kernels/,trnrt/models/,model_runner.py,
capture.py,config.py,platform.py), no behavior change for conforming requests.
Checklist
- One logical change
- Tests come with the change; red on
development, green after - Lint, format, mypy clean
- CHANGELOG entry
- Rebase on
developmentafter #36 lands (if it lands first)
🤖 Generated with Claude Code
Summary by cubic
Validates the tools array and parallel_tool_calls flag at the host boundary before they reach the chat template or parser. Non-array tools previously caused a 500 inside the template; malformed entries and non-boolean parallel_tool_calls were silently accepted, changing the rendered prompt and keeping parallel calls on.
- Adds
tools_error()to check each entry against the OpenAI schema: object shape,type, functionname([A-Za-z0-9_-]{1,64}),parametersandparameters.propertiesobjects,descriptionstring, and refusal ofstrict: true. - Routes
parallel_tool_callsthrough shared boolean validation, treating an explicitnullas invalid as well. - Skips tool validation when
tool_choiceis"none", since the request discards tools exactly as before. - Tests cover each failure shape plus accepted controls: valid tools with
parallel_tool_calls: false,strict: false, andtool_choice: "none".
Summary by CodeRabbit
- Bug Fixes
- Malformed
toolsvalues and entries now receive an HTTP 400 response identifying the invalid field, rather than causing a template error. - Non-boolean
parallel_tool_callsvalues, including explicitnull, are rejected with HTTP 400 instead of being treated as enabled. - Malformed tool definitions are ignored when
tool_choiceis set to"none". - Valid tool definitions with
parallel_tool_callsset tofalseandstrictset tofalsecontinue to be accepted.
- Malformed