This pull request introduces a robust mechanism to handle Structured Outputs (Pydantic BaseModel enforcement) for LLMs that do not natively support the response_format parameter (e.g., Deepseek Chat, certain open-source models).

Why

Currently, when attempting structured output using the response_format argument （in client.py）with models that don't support it (like deepseek/deepseek-chat), the call to litellm.acompletion fails with a litellm.BadRequestError:

litellm.BadRequestError: DeepseekException - {"error":{"message":"This response_format type is unavailable now","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}
This issue severely limits the range of models compatible with structured output tasks.

What Changed

The core logic has been refactored into a new class, StructuredOutputHandler, adopting a proven strategy implemented by projects like CrewAI (Kudos to the CrewAI team for pioneering this pattern! 👏).

1. New Class Structure & Schema Injection
   The logic is encapsulated in the StructuredOutputHandler class.

We stop passing the response_format argument to litellm.acompletion.

Instead, the Pydantic Model's schema is parsed, optimized, and injected as a strict System Prompt instruction to guide the LLM's output format (format_messages function).

1. Robust Post-processing and Retries
   We introduce a Converter class that manages the post-processing workflow:

It attempts to parse the raw text output into the target Pydantic BaseModel.

If parsing fails (due to partial or invalid JSON), the logic attempts to extract the JSON robustly (handle_partial_json).

If necessary, it initiates retries (up to 3 attempts) by calling the LLM again, asking it to fix the improperly formatted output.

1. Output Normalization
   The successfully validated Pydantic BaseModel instance is converted to a Python dictionary (model.model_dump()) and replaces the original text content in the final response object.

Test Plan

The easiest way to verify this fix is by testing with a non-natively-supported model like Deepseek.

Set the environment variable DEEPSEEK_API_KEY.

Modify the model parameter in examples/quickstart.py from the current default to deepseek/deepseek-chat.

Run the example.

Expected Result: The program should now successfully execute the structured inference task and return a validated Pydantic-based dictionary, without encountering the BadRequestError related to the response_format parameter.

---

Summary by cubic

Adds a generic structured output path that enforces Pydantic models for LLMs without native response_format support (e.g., Deepseek), and updates the client to fall back to this handler on errors. Structured inference now works across more models without BadRequestError.

• New Features

  • Introduced StructuredOutputHandler to enforce Pydantic BaseModel outputs via system-prompt schema injection.
  • Normalizes schemas (resolves $refs, converts oneOf→anyOf, marks all properties required) for strict output.
  • Adds robust parsing with partial JSON extraction and up to 3 retries; final response content is a model_dump() dict.

• Bug Fixes

  • client.py catches litellm.BadRequestError and routes the call through StructuredOutputHandler.
  • Enables structured outputs on deepseek/deepseek-chat and other non-supporting models.