Server HTTP API & Endpoints

Relevant source files

The following files were used as context for generating this wiki page:

• mlx_qwen3_asr/server.py
• mlx_qwen3_asr/cli.py
• tests/test_server.py
• CLAUDE.md
• examples/openai-server/README.md
• pyproject.toml

The mlx-qwen3-asr server is a built-in HTTP transcription service designed to turn any Apple Silicon Mac into a speech-to-text API endpoint. Built using FastAPI and Uvicorn, the server provides high-performance inference via MLX, supporting both a native asynchronous job-based API and a synchronous OpenAI-compatible transcription interface.

The system is designed for production readiness, featuring sliding-window rate limiting, API key authentication, backpressure management via an atomic job queue, and automatic cleanup of temporary resources. It allows developers to integrate Qwen3-ASR capabilities into external tools and workflows without requiring local model management or complex dependencies.

Sources: mlx_qwen3_asr/server.py:1-15, CLAUDE.md:12-16, pyproject.toml:39

Architecture and Design

The server operates as an asynchronous application centered around a shared state that manages model sessions, job queues, and rate limiting. It uses a sequential worker pattern to ensure that compute-intensive inference tasks are serialized, preventing resource exhaustion on the host machine.

System Components

The architecture is divided into the following primary components:

• FastAPI Application: Manages routing, request validation, and HTTP responses.
• Job Worker: A background task that pulls jobs from an asyncio.Queue and executes transcription using a shared Session object.
• TTL Sweeper: A background routine that prunes completed or failed jobs from memory after a configurable Time-To-Live (TTL).
• Rate Limiter: A sliding-window implementation keyed by API keys to prevent abuse.
• Inference Lock: An asyncio.Lock used to ensure only one transcription process accesses the MLX device at a time.

Sources: mlx_qwen3_asr/server.py:92-125, mlx_qwen3_asr/server.py:145-186, tests/test_server.py:228-251

Data Flow

The following diagram illustrates the lifecycle of a transcription request through the system:

API Endpoints

The server exposes three categories of endpoints: health and monitoring, native asynchronous jobs, and OpenAI-compatible transcription.

Native Asynchronous API

These endpoints are recommended for long-form audio to avoid HTTP timeouts.

Endpoint	Method	Description
/transcribe	POST	Submits an audio file for asynchronous transcription. Returns a job_id.
/jobs/{job_id}	GET	Retrieves the status and result of a specific transcription job.

Sources: mlx_qwen3_asr/server.py:238-333

OpenAI-Compatible API

These endpoints allow the server to act as a drop-in replacement for OpenAI's transcription service.

Endpoint	Method	Description
/v1/models	GET	Lists the currently loaded Qwen3-ASR model.
/v1/audio/transcriptions	POST	Synchronous transcription returning JSON, Text, SRT, or VTT formats.

Sources: mlx_qwen3_asr/server.py:337-430, examples/openai-server/README.md:1-20

Native Request Parameters

The POST /transcribe and POST /v1/audio/transcriptions endpoints support various parameters to control the output:

Parameter	Type	Description
audio / file	File	The audio file to transcribe (AAC, FLAC, M4A, MP3, MP4, WAV, etc.).
language	String	Force a specific language (e.g., "en", "zh").
timestamps	Boolean	(Native) Request word-level timestamps via forced aligner.
context / prompt	String	Provide domain-specific terms to bias transcription.
response_format	String	(OpenAI) One of json, text, verbose_json, srt, vtt.

Sources: mlx_qwen3_asr/server.py:240-245, mlx_qwen3_asr/server.py:355-364, mlx_qwen3_asr/cli.py:15-32

Configuration and CLI

The server is started via the CLI command mlx-qwen3-asr serve. It can be configured using command-line flags or environment variables.

Server Configuration Options

Flag	Env Var	Default	Description
--host	MLX_ASR_HOST	0.0.0.0	Bind address.
--port	MLX_ASR_PORT	8765	Server port.
--api-key	MLX_ASR_API_KEY	(None)	Comma-separated list of valid keys.
--model	-	Qwen/Qwen3-ASR-0.6B	Model ID or path.
--dtype	-	float16	Precision (float16, float32, bfloat16).
--rate-limit	-	60	Max requests per minute per key.
--max-queue-depth	-	10	Max pending jobs before returning 503.
--job-ttl	-	3600	Retention period for job results (seconds).

Sources: mlx_qwen3_asr/server.py:65-79, mlx_qwen3_asr/cli.py:247-285

Authentication and Security

Security is enforced through Bearer Token authentication. All requests to transcription and job endpoints must include an Authorization header.

# Example Authentication Header
headers = {"Authorization": "Bearer your_api_key_here"}

The server also implements path sanitization for error messages to prevent leaking internal server directory structures (e.g., /tmp/, /Users/) in the event of an inference failure.

Sources: mlx_qwen3_asr/server.py:127-141, mlx_qwen3_asr/server.py:476-484, tests/test_server.py:80-92

Sequence of Synchronous vs Asynchronous Jobs

The server handles both synchronous (OpenAI) and asynchronous (Native) requests. Both paths compete for the same inference_lock to ensure hardware safety on Apple Silicon.

Conclusion

The mlx-qwen3-asr server provides a robust, scalable interface for Apple Silicon-based speech recognition. By offering both a native async API and OpenAI compatibility, it bridges the gap between high-performance local inference and standard industry integrations. Its internal architecture ensures hardware efficiency through serialized execution while maintaining service availability via backpressure and rate limiting.