Built-in Transcription Server Overview

Relevant source files

The following files were used as context for generating this wiki page:

• mlx_qwen3_asr/server.py
• examples/openai-server/README.md
• mlx_qwen3_asr/cli.py
• mlx_qwen3_asr/transcribe.py
• tests/test_server.py
• CLAUDE.md

The built-in transcription server is a production-grade HTTP interface that transforms an Apple Silicon Mac into a local speech-to-text API endpoint. It leverages FastAPI and Uvicorn to provide high-performance asynchronous transcription capabilities, featuring support for both a custom job-based API and an OpenAI-compatible interface.

The server is designed for native MLX-optimized inference on Metal, supporting 0.6B and 1.7B Qwen3-ASR models. It manages model lifecycles, handles concurrent request backpressure, and enforces security through API key authentication and rate limiting.

Sources: mlx_qwen3_asr/server.py:1-15, CLAUDE.md:21-25

Server Architecture

The server architecture utilizes a centralized state management system and a sequential job worker to manage inference resources efficiently on Metal.

System Components

• FastAPI Application: Handles HTTP routing, request validation, and OpenAPI documentation.
• Job Queue: An asyncio.Queue that manages pending transcription tasks with configurable backpressure.
• Sequential Worker: A background task that pulls jobs from the queue to ensure inference is performed sequentially, preventing GPU memory exhaustion.
• Model Holder (Session): Manages the Qwen3ASRModel lifecycle and performs the actual MLX inference.
• Inference Lock: An asyncio.Lock that serializes access to the model during transcription.

Data Flow Diagram

The following diagram illustrates how an audio transcription request is handled through the system, from initial submission to final result retrieval.

API Endpoints

The server exposes two distinct sets of endpoints: a native asynchronous job API and an OpenAI-compatible synchronous API.

Native Job API

Endpoint	Method	Description
/health	GET	Returns server status, uptime, model info, and queue depth.
/transcribe	POST	Submits an audio file for asynchronous processing. Returns a job_id.
/jobs/{job_id}	GET	Retrieves the status and result of a specific job.

OpenAI-Compatible API

Endpoint	Method	Description
/v1/models	GET	Lists the currently loaded model ID.
/v1/audio/transcriptions	POST	Synchronous transcription compatible with OpenAI SDKs.

Sources: mlx_qwen3_asr/server.py:250-450, examples/openai-server/README.md:1-50

Configuration and Security

The server is configured via the ServerConfig dataclass, which can be populated through CLI flags or environment variables.

Configuration Parameters

Flag	Env Var	Default	Description
--host	MLX_ASR_HOST	0.0.0.0	Bind address for the server.
--port	MLX_ASR_PORT	8765	Server port.
--api-key	MLX_ASR_API_KEY	(empty)	Comma-separated list of authorized keys.
--max-queue-depth	-	10	Max pending jobs before returning 503.
--rate-limit	-	60	Max submissions per minute per key.
--job-ttl	-	3600	Seconds to retain completed job results.

Sources: mlx_qwen3_asr/server.py:58-71, mlx_qwen3_asr/cli.py:277-315

Security Measures

1. Bearer Authentication: All endpoints (except /health) require a valid API key provided in the Authorization: Bearer <key> header.
2. Rate Limiting: A sliding-window rate limiter tracks requests per API key to prevent abuse.
3. Job Isolation: Only the API key that submitted a job is permitted to retrieve its results via the /jobs endpoint.
4. Error Sanitization: Internal file paths (e.g., /tmp/) are stripped from error messages to prevent information leakage.

Sources: mlx_qwen3_asr/server.py:80-100, mlx_qwen3_asr/server.py:380-385, mlx_qwen3_asr/server.py:504-512

Implementation Details

Transcription Lifecycle

When a transcription is requested, the server performs the following steps:

1. Validation: Checks file size (default max 2GB) and audio duration limits.
2. Persistence: Writes the uploaded UploadFile to a temporary local file prefixed with mlx_asr_.
3. Inference: Calls transcribe_async, which offloads the MLX pipeline to a background thread to avoid blocking the event loop.
4. Cleanup: Automatically deletes temporary files after job completion or failure.

# mlx_qwen3_asr/server.py:165-179
async def _run_transcription(job: Job) -> dict:
    """Run transcription via Session.transcribe_async (thread offload)."""
    session = state.session
    async with state.inference_lock:
        result = await session.transcribe_async(
            job.temp_path,
            language=job.language,
            context=job.context,
            return_timestamps=job.timestamps,
            return_chunks=True,
        )
    return _result_to_dict(result)

Sources: mlx_qwen3_asr/server.py:165-179, mlx_qwen3_asr/server.py:295-310

Job State Machine

Jobs move through a specific lifecycle represented by the JobStatus enum:

Usage Example

To start the server with an auto-generated API key:

export MLX_ASR_API_KEY="$(openssl rand -hex 16)"
mlx-qwen3-asr serve --api-key "$MLX_ASR_API_KEY"

Clients can then interact with the server using standard tools like curl:

curl http://localhost:8765/v1/audio/transcriptions \
  -H "Authorization: Bearer $MLX_ASR_API_KEY" \
  -F "file=@recording.wav" \
  -F "model=Qwen/Qwen3-ASR-0.6B" \
  -F "response_format=json"

Sources: examples/openai-server/README.md:1-40

Conclusion

The Built-in Transcription Server provides a robust, secure, and compatible way to deploy Qwen3-ASR models on Apple Silicon. By combining a native asynchronous job system with OpenAI compatibility, it serves both specialized high-volume workflows and existing ecosystem integrations, ensuring efficient resource utilization through managed concurrency and backpressure.