Server Deployment & Configuration

Relevant source files

The following files were used as context for generating this wiki page:

• mlx_qwen3_asr/server.py
• mlx_qwen3_asr/cli.py
• tests/test_server.py
• CLAUDE.md
• examples/openai-server/README.md
• pyproject.toml

The mlx-qwen3-asr project includes a built-in HTTP transcription server that turns any Apple Silicon Mac into a speech-to-text API endpoint. This server is built using FastAPI and Uvicorn, providing both a native asynchronous job API and an OpenAI-compatible transcription interface. Sources: mlx_qwen3_asr/server.py:1-15, CLAUDE.md:120-125

The deployment is designed for production-grade use on Metal-optimized hardware, featuring backpressure management via an atomic job queue, sliding-window rate limiting, and multi-slot model caching. It supports the 0.6B (fast) and 1.7B (accurate) Qwen3-ASR models with various quantization levels. Sources: CLAUDE.md:16-25, mlx_qwen3_asr/server.py:84-110

Server Architecture

The server utilizes an asynchronous architecture to handle long-running transcription tasks without blocking the main event loop. It maintains a shared application state (_AppState) that includes an inference_lock to ensure that the MLX model, which utilizes Metal for acceleration, processes requests sequentially to avoid memory contention. Sources: mlx_qwen3_asr/server.py:102-114, mlx_qwen3_asr/server.py:165-175

Component Overview

Component	Description
FastAPI App	Handles HTTP routing, request validation, and auth middleware.
Job Queue	An asyncio.Queue with configurable max depth for managing pending tasks.
Background Worker	A sequential processor that pulls jobs from the queue and executes transcription.
TTL Sweeper	Periodically removes expired job results from memory based on job_ttl_sec.
Rate Limiter	A sliding-window limiter keyed by API key to prevent service abuse.

Sources: mlx_qwen3_asr/server.py:118-135, mlx_qwen3_asr/server.py:175-195, tests/test_server.py:113-130

Job Lifecycle Flow

The following diagram illustrates the flow of an asynchronous transcription request from submission to result retrieval.

Configuration Options

Server configuration is managed via the ServerConfig dataclass, which can be populated through CLI flags or environment variables. Sources: mlx_qwen3_asr/server.py:64-78, mlx_qwen3_asr/cli.py:246-275

Primary Configuration Parameters

Flag	Env Var	Default	Description
--host	MLX_ASR_HOST	0.0.0.0	Bind address for the server.
--port	MLX_ASR_PORT	8765	Listening port (range 1-65535).
--api-key	MLX_ASR_API_KEY	None	Comma-separated list of authorized keys.
--model	-	Qwen/Qwen3-ASR-0.6B	HF model ID or local path.
--dtype	-	float16	Model precision (float16, float32, bfloat16).
--max-queue-depth	-	10	Maximum depth of the internal job queue.
--rate-limit	-	60	Max requests per minute per API key.
--job-ttl	-	3600	Retention period (seconds) for completed jobs.

Sources: mlx_qwen3_asr/server.py:64-78, mlx_qwen3_asr/cli.py:246-275, tests/test_server.py:148-165

Deployment & Execution

Installation

The server requires optional dependencies. It is installed using the [serve] extra:

pip install "mlx-qwen3-asr[serve]"

Sources: pyproject.toml:39, CLAUDE.md:25

Starting the Server

The server is launched via the serve subcommand. At least one API key is mandatory for the server to start.

mlx-qwen3-asr serve --api-key my-secret-key --port 8765

Sources: mlx_qwen3_asr/cli.py:242-245, mlx_qwen3_asr/server.py:596-605

Validation Logic

Before startup, the server performs a pre-flight validation of the ServerConfig. It will exit with a SystemExit if:

1. No API keys are provided.
2. Port is outside the range 1-65535.
3. Queue depth, rate limit, or file size limits are less than 1.

Sources: mlx_qwen3_asr/server.py:596-614, tests/test_server.py:168-190

API Endpoints

Native Asynchronous API

This API is recommended for long-form audio to avoid client timeouts during transcription. Sources: examples/openai-server/README.md:58-62

• POST /transcribe: Submits an audio file for processing. Returns a job_id. Sources: mlx_qwen3_asr/server.py:272-330
• GET /jobs/{job_id}: Polls for the status and result of a specific job. Access is restricted to the API key that submitted the job. Sources: mlx_qwen3_asr/server.py:332-359
• GET /health: Returns server status, model info, and queue metrics. Sources: mlx_qwen3_asr/server.py:255-270

OpenAI-Compatible API

The server provides endpoints compatible with the OpenAI SDK, allowing it to act as a drop-in replacement for api.openai.com/v1. Sources: examples/openai-server/README.md:1-15

Error Handling & Security

The server implements specific measures to protect internal system information and manage load:

1. Path Sanitization: Error messages are intercepted by _sanitize_error to strip internal server paths (e.g., /tmp/, /Users/, /home/) before returning them to the client. Sources: mlx_qwen3_asr/server.py:522-530, tests/test_server.py:84-96
2. Backpressure: When the job queue and in-flight synchronous requests exceed max_queue_depth, the server returns a 503 Service Unavailable with a Retry-After header. Sources: mlx_qwen3_asr/server.py:321-329, tests/test_server.py:355-390
3. Authentication: All endpoints except /health require a Bearer token in the Authorization header. Sources: mlx_qwen3_asr/server.py:137-149, tests/test_server.py:218-245

The transcription server provides a robust bridge between high-performance MLX inference and web-based applications, ensuring efficient resource utilization on Apple Silicon hardware while maintaining API compatibility with industry standards. Sources: CLAUDE.md:120-125, mlx_qwen3_asr/server.py:1-10